An internet of things data stream intelligent management system based on data processing

Abstract

The application discloses an Internet of Things data stream intelligent management system based on data processing, and relates to the technical field of digital information transmission.The Internet of Things data stream intelligent management system comprises an edge dynamic compression module, a quantum encryption communication module, an Internet of Things communication management and control module, a cloud multi-modal fusion detection module and a cross-protocol equipment cooperation module.The edge dynamic compression module comprises a space-time slicing processing unit and an edge temporary storage unit, and the quantum encryption communication module comprises a miniature quantum key generation unit and an encryption transmission unit.The application sets the edge dynamic compression module and the quantum encryption communication module, improves the edge side data compression precision and the adaptability of quantum encryption technology to edge equipment, and thus solves the problems of high data redundancy of existing Internet of Things edge nodes, difficulty in deploying quantum encryption technology in edge equipment and insufficient data security in narrowband transmission scenarios.

Description

Technical Field

[0001] This invention relates to the field of digital information transmission technology, and in particular to an intelligent management system for Internet of Things (IoT) data streams based on data processing. Background Technology

[0002] The IoT data flow intelligent management system integrates various types of data collected by sensing layer devices to achieve closed-loop management of data from collection and access, cleaning and filtering, storage optimization, real-time analysis to intelligent decision-making and feedback control. Ultimately, it improves the data utilization efficiency of the IoT system, reduces operation and maintenance costs, and ensures the intelligence and reliability of device collaboration and business operations.

[0003] Currently, there are several issues with intelligent management systems for IoT data streams, including high data redundancy at edge nodes, difficulty in deploying quantum encryption technology on edge devices due to module size and adaptability issues, and insufficient data security in narrowband transmission scenarios. In addition, existing systems often adopt a single-module optimization model, with a lack of collaborative connection between modules in data flow, which can easily lead to high data transmission latency and security gaps. Abnormal data is difficult to trigger collaborative processing by devices, resulting in insufficient overall management efficiency and closed-loop control capabilities.

[0004] Therefore, an IoT data stream intelligent management system based on data processing is proposed to solve the above problems. Summary of the Invention

[0005] The main objective of this invention is to provide an intelligent management system for IoT data streams based on data processing, in order to solve the problems mentioned in the background above.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: an IoT data stream intelligent management system based on data processing, the IoT data stream intelligent management system including an edge dynamic compression module, a quantum encryption communication module, an IoT communication control module, a cloud multimodal fusion detection module, and a cross-protocol device collaboration module;

[0007] The edge dynamic compression module is used to compress the data stream collected by IoT terminal devices to optimize data transmission efficiency, including a spatiotemporal slicing processing unit and an edge temporary storage unit.

[0008] The quantum encryption communication module is used to encrypt and transmit the compressed data stream to ensure communication security, and includes a micro quantum key generation unit and an encryption transmission unit;

[0009] The IoT communication management module is used to manage the communication protocol adaptation, data stream transmission scheduling and device communication status management between IoT devices, including a dynamic protocol conversion unit, an intelligent resource scheduling unit and a device remote communication management unit.

[0010] The cloud-based multimodal fusion detection module is used to identify anomalies in the data stream transmitted to the cloud to ensure the validity of communication data. It includes a multimodal feature fusion unit and an anomaly determination unit.

[0011] The cross-protocol device collaboration module is used to realize the linkage control of IoT devices based on communication protocol adaptation, including an abnormal instruction generation unit and a protocol adaptation distribution unit.

[0012] Preferably, the spatiotemporal slicing processing unit of the edge dynamic compression module includes a dynamic period adjuster and a difference filter;

[0013] The dynamic cycle regulator uses adaptive frequency sensing technology to adjust the acquisition cycle according to the data stream change frequency to match the data transmission requirements.

[0014] The difference filter performs difference calculations on the data streams collected by adjacent sensors of the same type, retains the data stream segments with excessive differences, and transmits the selected data stream segments to the edge temporary storage unit.

[0015] Preferably, the edge temporary storage unit of the edge dynamic compression module includes an embedded storage chip;

[0016] The embedded storage chip establishes a communication connection with the spatiotemporal slicing processing unit through a data interface, receives and stores compressed data stream segments output by the spatiotemporal slicing processing unit, and transmits the stored compressed data stream segments to the quantum encryption communication module when the embedded storage chip detects the transmission ready signal of the quantum encryption communication module.

[0017] Preferably, the miniature quantum key generation unit of the quantum encrypted communication module includes an integrated quantum optical chip and a key synthesis circuit;

[0018] The integrated quantum optical chip is used to generate and detect quantum states. The key synthesis circuit generates an encryption key based on the quantum state detection result. The encryption key is used to encrypt data stream transmission to improve communication security.

[0019] Preferably, the encrypted transmission unit of the quantum encrypted communication module includes a narrowband communication module and a quantum encryption chip;

[0020] The quantum encryption chip is communicatively connected to the micro quantum key generation unit and uses the generated encryption key to encrypt data stream segments.

[0021] The narrowband communication module transmits the encrypted data stream to the IoT communication management module.

[0022] Preferably, the dynamic protocol conversion unit of the IoT communication management and control module includes a communication frame parser and a conversion rule generator;

[0023] The communication frame parser scans the communication frame structure features of the access device and extracts protocol identification information;

[0024] The conversion rule generator matches the built-in multi-protocol library to generate protocol conversion rules, thereby achieving communication compatibility between heterogeneous devices.

[0025] Preferably, the intelligent resource scheduling unit of the IoT communication management and control module includes a scene parameter collector and a reinforcement learning scheduler;

[0026] The scene parameter collector acquires the device operating parameters;

[0027] The reinforcement learning scheduler dynamically adjusts the reward function weights and allocates computing resources and communication bandwidth based on the running parameters.

[0028] Preferably, the device remote communication control unit of the IoT communication control module includes a device identity registration unit and a firmware remote updater;

[0029] The device identity registration device records the unique hardware identifier and communication protocol type of the access device;

[0030] The firmware remote updater pushes firmware update packages adapted to the device's communication protocol to the device via an encrypted communication link.

[0031] Preferably, the multimodal feature fusion unit of the cloud-based multimodal fusion detection module includes a cross-type feature extractor and an attention weight assigner;

[0032] The anomaly determination unit of the cloud-based multimodal fusion detection module includes a feature vector processor and an anomaly result output device;

[0033] The cross-type feature extractor extracts feature vectors from numerical data streams, device appearance image data streams, and device operation log text data streams.

[0034] The attention weight allocator determines the feature vector weights corresponding to different anomaly types based on historical anomaly data samples, and transmits the weighted feature vectors to the feature vector processor of the anomaly determination unit.

[0035] The feature vector processor receives the weighted feature vector and performs an anomaly detection algorithm on the feature vector;

[0036] The anomaly result outputter receives the computation result from the feature vector processor, generates an anomaly identification result containing the anomaly type and associated device identifier, and transmits the anomaly identification result to the cross-protocol device collaboration module.

[0037] Preferably, the abnormal instruction generation unit of the cross-protocol device collaboration module includes an abnormal result receiver and a control instruction encoder;

[0038] The protocol adaptation and distribution unit of the cross-protocol device collaboration module includes a multi-protocol communication module;

[0039] The anomaly result receiver receives the anomaly identification results output by the cloud-based multimodal fusion detection module and extracts the anomaly type and associated device identifier;

[0040] The control command encoder generates control commands adapted to the associated device based on the anomaly type and the associated device identifier;

[0041] The multi-protocol communication module receives the device control command output by the abnormal command generation unit, converts the command into the communication protocol format supported by the corresponding terminal device, and then transmits it to the terminal device.

[0042] The present invention has the following beneficial effects:

[0043] 1. This invention utilizes an adaptive frequency sensing technology to adjust the data acquisition cycle in the spatiotemporal slicing processing unit of the edge dynamic compression module, and a difference filter to retain valid data stream segments with excessive differences. Combined with the integrated quantum optical chip of the quantum encryption communication module to generate quantum keys, and a narrowband communication module to transmit encrypted data, this invention achieves dynamic and precise compression of edge-side data streams and lightweight quantum encrypted transmission. Compared with existing technologies where edge compression often uses a fixed acquisition cycle, resulting in redundant data, and quantum encryption modules are large and difficult to adapt to edge devices, and narrowband transmission is not deeply integrated with quantum encryption, this invention can improve the data compression accuracy at the edge and the adaptability of quantum encryption technology to edge devices. Therefore, it can solve the problems of high data redundancy in existing IoT edge nodes, difficulty in deploying quantum encryption technology on edge devices, and insufficient data security in narrowband transmission scenarios.

[0044] 2. The IoT communication management module of this invention comprises a dynamic protocol conversion unit that parses communication frames from access devices to extract protocol identifiers and matches them with multi-protocol libraries to generate conversion rules; an intelligent resource scheduling unit that adjusts reward functions based on reinforcement learning to allocate computing resources and communication bandwidth; a cloud-based multimodal fusion detection module that extracts numerical, image, and log data stream features and allocates attention weights to identify anomalies; and a cross-protocol device collaboration module that generates control commands for the appropriate protocol and sends them to the terminal. This achieves heterogeneous device communication compatibility, dynamic resource optimization, and multimodal data anomaly linkage processing. Compared with existing technologies, this can improve heterogeneous device communication adaptability, resource scheduling rationality, anomaly detection accuracy, and processing timeliness. Therefore, it can solve the problems of poor heterogeneous device communication compatibility, unreasonable resource allocation, high false negative and false positive rates in cloud-based anomaly detection, and the inability to quickly link devices across protocols to handle anomalies after they occur in existing IoT systems.

[0045] 3. This invention integrates an edge dynamic compression module, a quantum encrypted communication module, an IoT communication control module, a cloud-based multimodal fusion detection module, and a cross-protocol device collaboration module to construct an end-to-end closed-loop management architecture for IoT data streams. This architecture enables integrated processing of data streams from terminal acquisition and compression, encrypted transmission, communication control, cloud anomaly identification, to device linkage and control. Compared with existing technologies that often employ single-module optimization and lack collaborative connection between modules, this invention improves the coherence and overall efficiency of IoT data stream management. Therefore, it can solve the problems of high data transmission latency, security gaps, and the inability to quickly trigger device collaborative processing due to independent module operation in existing IoT systems. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the overall system architecture of the present invention;

[0047] Figure 2 This is a schematic diagram of the edge dynamic compression module architecture of the present invention;

[0048] Figure 3 This is a schematic diagram of the quantum encrypted communication module architecture of the present invention;

[0049] Figure 4 This is a schematic diagram of the IoT communication control module architecture of the present invention;

[0050] Figure 5 This is a schematic diagram of the cloud-based multimodal fusion detection module architecture of the present invention;

[0051] Figure 6 This is a schematic diagram of the cross-protocol device collaboration module architecture of the present invention. Detailed Implementation

[0052] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0053] Example 1, please refer to Figure 1 and Figure 2 As shown: An intelligent management system for IoT data streams based on data processing, comprising an edge dynamic compression module, a quantum encrypted communication module, an IoT communication control module, a cloud-based multimodal fusion detection module, and a cross-protocol device collaboration module;

[0054] The edge dynamic compression module is used to compress the data stream collected by IoT terminal devices to optimize data transmission efficiency, including a spatiotemporal slicing processing unit and an edge temporary storage unit.

[0055] The spatiotemporal slicing processing unit of the edge dynamic compression module includes a dynamic period adjuster and a difference filter;

[0056] The dynamic cycle regulator uses adaptive frequency sensing technology to adjust the acquisition cycle according to the frequency of data stream changes in order to match data transmission requirements.

[0057] The difference filter performs difference calculations on the data streams collected by adjacent sensors of the same type, retains the data stream segments with differences exceeding the limit, and transmits the selected data stream segments to the edge temporary storage unit.

[0058] The edge temporary storage unit of the edge dynamic compression module includes an embedded storage chip;

[0059] The embedded storage chip establishes a communication connection with the spatiotemporal slicing processing unit through a data interface, receives and stores compressed data stream segments output by the spatiotemporal slicing processing unit, and transmits the stored compressed data stream segments to the quantum encryption communication module when the embedded storage chip detects the transmission ready signal of the quantum encryption communication module.

[0060] Furthermore, the dynamic period regulator calculates the optimal acquisition period through multi-dimensional parameter collaboration, ensuring that data acquisition neither overloads edge computing power due to excessively short intervals nor causes critical data loss due to excessively long intervals. The formula for calculating the optimal acquisition period of the dynamic period regulator is as follows: ;

[0061] In the formula, T opt The optimal acquisition period, measured in milliseconds, is the final acquisition interval output by the dynamic period regulator. It directly determines the sensor data acquisition frequency and its value is constrained by the upper limit of the computing power of the edge device. The minimum threshold is 50ms and the maximum threshold is 2000ms.

[0062] T base The basic acquisition period, in milliseconds (ms), is a scenario-specific preset value for the corresponding sensor and is strongly tied to the sensor type. For example, T represents an industrial vibration sensor. base =100ms, Household Temperature and Humidity Sensor T base =500ms, automatically allocated based on the device's network registration information, not a universal default value;

[0063] α is a frequency adjustment coefficient, ranging from 0.3 to 0.8. It is dynamically assigned according to the functional importance of the sensor in the IoT architecture. For core control sensors, α is 0.7 to 0.8, and for environmental monitoring sensors, α is 0.3 to 0.4. It is used to control the degree of influence of the data flow change frequency on the acquisition cycle.

[0064] Δf avg The average data stream change rate over the first N periods is expressed in Hz, where N is the system's preset number of statistical periods (N=5), balancing real-time performance and statistical accuracy. The calculation method is as follows: ;

[0065] In the formula, f i The sampling frequency of the data stream in the i-th cycle;

[0066] f max The maximum data stream change rate of the sensor, in Hz, is determined by the system based on the sensor hardware parameters and the upper limit of the computing power of the edge device.

[0067] In terms of parameter configuration, the basic acquisition period for different types of sensors is automatically assigned based on the identity information when the device is registered with the network, rather than a uniform value is set. For example, industrial vibration sensors need to monitor the operating status of equipment in real time, and small vibration changes may be associated with potential equipment failures. Therefore, the basic acquisition period is set to a lower value to capture high-frequency changes. The environmental parameters monitored by household temperature and humidity sensors fluctuate smoothly in the short term. Excessive acquisition frequency will generate a large amount of redundant data. Therefore, the basic acquisition period is set to a higher value to balance data effectiveness and resource consumption.

[0068] The frequency adjustment coefficient is dynamically adjusted based on the functional importance of the sensor in the IoT architecture, forming a differentiated periodic response strategy. Core control sensors directly participate in equipment operation and regulation. For example, data changes from pressure sensors in industrial production lines and current sensors in smart grids may directly affect system safety and operational stability. Therefore, a higher adjustment coefficient is assigned to them, making the impact of data flow change frequency on the acquisition cycle more significant. When data fluctuations occur, the acquisition cycle can be shortened quickly, improving monitoring sensitivity. Environmental monitoring sensor data has a smaller impact on the core operation of the system, so a lower adjustment coefficient is assigned to them to reduce frequent adjustments to the acquisition cycle caused by small data fluctuations and reduce the computing power consumption of edge devices.

[0069] In terms of data change rate processing, the dynamic cycle regulator calculates the average change rate by statistically analyzing the sampling frequency of the data stream in the previous 5 cycles. This avoids interference from fluctuations in the sampling frequency of a single cycle on the cycle calculation, ensuring the accuracy of the statistical results, and also prevents data lag due to too many statistical cycles, thus meeting the real-time requirements.

[0070] The difference filter first identifies and matches all adjacent sensors of the same type using a pre-stored device deployment map in the IoT system. This ensures that only data from sensors that are spatially close and monitor the same object are processed, avoiding meaningless difference calculations due to different sensor types or excessive distance. Subsequently, it performs real-time difference calculations on the successfully matched sensor data streams and determines whether a data segment needs to be retained based on a preset difference threshold. The threshold setting is dynamically adjusted according to the sensor type and application scenario. For example, the difference threshold setting for industrial vibration sensors is stricter to capture minute vibration differences in the equipment, while the difference threshold setting for household temperature and humidity sensors is more lenient to reduce unnecessary filtering triggers.

[0071] After the filtering is completed, the difference filter only transmits the high-value data stream segments to the edge storage unit. Compared with the traditional method of fully compressing the data stream of a single sensor, this filtering mechanism based on spatial correlation can more accurately remove redundant data. While reducing the amount of data, it can also indirectly achieve the preliminary identification of abnormal data, providing convenience for subsequent data processing and analysis. It has the dual functions of data compression and data preprocessing.

[0072] The edge storage unit is used to connect the spatiotemporal slicing processing unit and the quantum encryption communication module. In terms of hardware selection, the embedded storage chip is smaller and consumes less power, which can meet the strict requirements of edge devices for space and energy consumption. At the same time, the embedded chip with high-speed read and write capabilities is selected to ensure that the data stream segments output by the spatiotemporal slicing processing unit can be received quickly, avoiding data accumulation due to insufficient storage speed.

[0073] In terms of communication and data scheduling logic, the embedded storage chip establishes a stable connection with the spatiotemporal slicing processing unit through the industrially common SPI or I2C data interface. The low latency of the interface ensures that the compressed data can be quickly transmitted to the temporary storage unit. The status signals of the quantum encryption communication module are received in real time through the UART communication protocol. When a transmission ready signal is detected, such as when the module completes the establishment of the encryption channel or the previous batch of data transmission is completed, the embedded storage chip will immediately start the data transmission process and push the temporarily stored compressed data stream fragment to the quantum encryption communication module. If no ready signal is detected, the data will continue to be stored securely until the module is ready.

[0074] This state-triggered data transmission mechanism effectively avoids transmission failures or bandwidth waste caused by modules not being ready in the traditional continuous transmission attempt mode. At the same time, the embedded storage chip automatically clears the corresponding storage block after the data transmission is completed, releasing storage space to receive new compressed data, ensuring that the edge temporary storage unit is always in a high-efficiency operating state, so that data will not be lost due to storage overflow, nor will the reception of new data be affected by storage block occupation.

[0075] Example 2, please refer to Figure 1 and Figure 3 As shown: An IoT data stream intelligent management system based on data processing, a quantum encryption communication module is used to encrypt and transmit compressed data streams to ensure communication security, including a micro quantum key generation unit and an encryption transmission unit;

[0076] The miniature quantum key generation unit of the quantum encrypted communication module includes an integrated quantum optical chip and a key synthesis circuit;

[0077] An integrated quantum optical chip is used to generate and detect quantum states. A key synthesis circuit generates an encryption key based on the quantum state detection results. The encryption key is used to encrypt data stream transmission to improve communication security.

[0078] The encrypted transmission unit of the quantum encrypted communication module includes a narrowband communication module and a quantum encryption chip;

[0079] The quantum encryption chip communicates with a micro quantum key generation unit and uses the generated encryption key to encrypt data stream segments.

[0080] The narrowband communication module transmits the encrypted data stream to the IoT communication management module.

[0081] Furthermore, the integrated quantum optical chip employs silicon-based photonic integration technology, with a size of 8mm×5mm×1mm to meet miniaturization requirements. The integrated quantum optical chip's function is to generate and detect quantum states. It includes a built-in quantum light source array, an optical modulator, and a single-photon detector array. The quantum light source array is based on quantum dots and outputs single photons at a wavelength of 1550nm. The optical modulator is a Mach-Zehnder interferometer structure, capable of modulating the polarization and phase states of single photons. The polarization states include four types: horizontal, vertical, 45°, and 135°, while the phase states include two types: 0 and π, generating a total of six quantum states for key generation. The single-photon detector array consists of four avalanche photodiodes, with a detection efficiency of no less than 85% and a dark count rate of no more than 1×10⁻⁶. -6 Every second, the quantum state detection results are converted into 1 and 0 electrical signals and transmitted to the key synthesis circuit after being cooled to -10℃ to reduce noise.

[0082] The integrated quantum optics chip also incorporates a temperature compensation and optical power stabilization module. When the temperature fluctuation exceeds ±0.5℃, the heating element or heat sink automatically adjusts the temperature. When the optical power fluctuation exceeds ±5%, the power is adjusted by a drive current ranging from 10mA to 50mA to ensure quantum state stability.

[0083] The key synthesis circuit is connected to the integrated quantum optical chip via a high-speed LVDS interface with a transmission rate of no less than 1Gbps. The function of the key synthesis circuit is to generate encryption keys based on the detection results, and it also has optimization, storage and anti-leakage capabilities. It adopts a Xilinx Artix-7 series FPGA with no less than 50,000 logic units to achieve high-speed data processing.

[0084] The key synthesis circuit's anti-leakage design includes multiple aspects. First, the OTP memory stores the algorithm firmware to prevent tampering. The SRAM temporarily stores the key and automatically clears it after power failure. In the event of a voltage attack or laser attack, a key destruction mechanism is triggered to overwrite the key data with random scrambled characters.

[0085] The encrypted transmission unit receives the key from the micro quantum key generation unit, encrypts the compressed data stream, and transmits it to the IoT communication control module via narrowband communication. The quantum encryption chip is manufactured using CMOS technology and measures 5mm×5mm×0.8mm. It uses an improved AES-256 algorithm for encryption and automatically updates the key every 1024 bytes of encrypted data, with an update delay of no more than 1ms to prevent security risks. The data segment length can be adjusted to 512 bytes to reduce latency.

[0086] The quantum encryption chip has a highly targeted anti-attack design. It prevents timing attacks by using random delays of 1ns to 10ns, prevents power consumption attacks by using current flattening in the range of ±2μA, and sets up a key cache isolation area to physically isolate the key from the storage area of ​​other data to avoid leakage.

[0087] The narrowband communication module is based on LoRa technology and operates in the Sub-1GHz frequency band, with options for 433MHz, 868MHz, or 915MHz. The transmission rate ranges from 1kbps to 250kbps. In densely populated areas, a high rate is used to reduce data accumulation, while in sparsely populated areas, a low rate is used to achieve a maximum transmission distance of 10km. Its core function is to transmit encrypted data streams and supports multi-node communication and dynamic channel adjustment.

[0088] The protocol stack of the narrowband communication module has been optimized. The physical layer adopts FHSS frequency hopping spread spectrum technology with a hopping interval of 200kHz and a hopping rate of 10 times per second to resist single-channel interference. The data link layer adopts CSMA / CA carrier sense multiple access technology to reduce communication conflicts between nodes. Before sending data, the module listens for channel idle status. If the channel is idle, it sends immediately. If the channel is busy, it waits for a random time of 10ms to 100ms before listening again. The network layer supports a star topology. Each module can directly communicate with the gateway node of the IoT communication management module as a terminal node. It also supports a node sleep-wake mechanism. When the module is not transmitting data, it automatically enters sleep mode with a sleep current of no more than 1μA. When there is a data transmission requirement, it is woken up by an external interrupt signal with a wake-up time of no more than 50ms, which greatly reduces the power consumption of IoT devices.

[0089] Example 3, please refer to Figure 1 and Figure 4 As shown: An IoT data stream intelligent management system based on data processing, the IoT communication control module is used to control the communication protocol adaptation between IoT devices, data stream transmission scheduling and device communication status management, including a dynamic protocol conversion unit, an intelligent resource scheduling unit and a device remote communication control unit;

[0090] The dynamic protocol conversion unit of the IoT communication management and control module includes a communication frame parser and a conversion rule generator;

[0091] The communication frame parser scans the communication frame structure characteristics of the access device and extracts protocol identification information;

[0092] The conversion rule generator matches the built-in multi-protocol library to generate protocol conversion rules, enabling communication compatibility between heterogeneous devices.

[0093] The intelligent resource scheduling unit of the IoT communication management and control module includes a scene parameter collector and a reinforcement learning scheduler;

[0094] The scene parameter collector acquires the device's operating parameters;

[0095] The reinforcement learning scheduler dynamically adjusts the weights of the reward function and allocates computing resources and communication bandwidth based on the running parameters.

[0096] The device remote communication control unit of the IoT communication control module includes a device identity registration unit and a firmware remote updater;

[0097] The device identity registration device records the unique hardware identifier and communication protocol type of the access device;

[0098] The firmware remote updater pushes firmware update packages adapted to the device's communication protocol to the device via an encrypted communication link.

[0099] Furthermore, the dynamic protocol conversion unit addresses the communication compatibility issues of heterogeneous devices. The communication frame parser and conversion rule generator work together to ensure real-time and accurate protocol conversion, adapting to the protocol extension requirements of the IoT operating system. First, the communication frame parser scans the communication frames of the access devices to extract protocol identification information, and also has the ability to handle abnormal frames. It adopts a two-stage mechanism of field scanning and feature matching, with a scanning rate of up to 1000 frames per second, supporting mainstream IoT frame formats. During the scan, abnormal frames are detected and buffered for retransmission. If the number of retries is exceeded, the device is marked as abnormal and reported. The protocol identifier is extracted, including version number, port number, and data type code. It has multiple built-in mainstream protocol identification rules and can extend new protocols through the remote configuration interface of the IoT operating system, with efficient extension response. At the same time, the frame data is preprocessed, the byte order is unified, and long frames are marked for easy subsequent processing.

[0100] The conversion rule generator generates conversion rules based on parsed protocol identifiers and a built-in multi-protocol library. It supports dynamic adjustment of strategies. The multi-protocol library covers industrial low-power short-range and emerging IoT protocols. It obtains update packages from the IoT operating system cloud weekly via an encrypted link, ensuring that updates do not interrupt the conversion. The rule generation process involves three steps: protocol matching, conversion logic construction, and optimization. It combines device type and data priority to generate rules and optimizes historical data to reduce the conversion failure rate.

[0101] The intelligent resource scheduling unit optimizes the allocation of computing resources and communication bandwidth. The scene parameter collector and reinforcement learning scheduler respond to the resource allocation instructions of the IoT operating system to ensure high-priority data transmission. First, the scene parameter collector acquires the data transmission and resource usage parameters of the devices to provide a basis for scheduling. The scene parameter collector adopts a distributed acquisition architecture and dynamically adjusts the acquisition frequency according to the device type. In industrial scenarios, the acquisition accuracy of control devices is improved, while in home scenarios, the frequency of inactive devices is reduced. The acquired data is preprocessed, invalid parameters are filtered, and alarms are reported to reduce device resource usage. The data is compressed and transmitted to the scheduler.

[0102] The reinforcement learning scheduler allocates resources based on collected parameters using reinforcement learning algorithms. Unlike fixed-rule scheduling, it adopts a triple framework of state, action, and reward, dynamically adjusting the weight of the reward function according to the scenario. In industrial scenarios, it focuses on transmission latency and packet loss rate, while in home scenarios, it focuses on resource utilization and energy consumption. The scheduling decision senses the system status every 100 milliseconds, predicts the optimal action through an improved DQN algorithm, and optimizes the results after resource allocation by statistical feedback, forming a closed-loop mechanism. At the same time, it has a built-in resource conflict arbitration mechanism to allocate resources according to device priority, ensuring the needs of critical equipment.

[0103] The device remote communication management unit realizes device identity management and firmware maintenance. Relying on the encryption mechanism of the IoT operating system, it ensures transmission security and meets the secure transmission requirements of the H04L field. First, the device identity registration device records the device's multi-dimensional hardware identifier and communication protocol type to establish an encrypted identity file. The hardware identifier includes MAC address, chip serial number, factory code, and system unique ID to improve anti-counterfeiting capabilities. The file is stored in an AES-256 encrypted database, which supports the storage and efficient query of a large amount of device information. When a device connects, it goes through a three-step process of identity submission and file establishment. After successful verification, the file is synchronized to other units. The device identity is checked regularly, and the permissions of abnormal devices are restricted and the operation and maintenance are notified. The firmware remote updater pushes the firmware update package with the appropriate protocol through the TLS1.3 encrypted link and uses modular compression to reduce bandwidth consumption.

[0104] Example 4, please refer to Figures 1 to 5 As shown: An IoT data stream intelligent management system based on data processing, the cloud multimodal fusion detection module is used to identify anomalies in the data stream transmitted to the cloud to ensure the validity of communication data, including a multimodal feature fusion unit and an anomaly determination unit;

[0105] The multimodal feature fusion unit of the cloud-based multimodal fusion detection module includes a cross-type feature extractor and an attention weight assigner;

[0106] The anomaly detection unit of the cloud-based multimodal fusion detection module includes a feature vector processor and an anomaly result output device;

[0107] The cross-type feature extractor extracts feature vectors from numerical data streams, device appearance image data streams, and device operation log text data streams.

[0108] The attention weight allocator determines the feature vector weights corresponding to different anomaly types based on historical anomaly data samples, and transmits the weighted feature vectors to the feature vector processor of the anomaly determination unit.

[0109] The feature vector processor receives the weighted feature vectors and performs an anomaly detection algorithm on the feature vectors.

[0110] The anomaly result outputter receives the computation results from the feature vector processor, generates an anomaly identification result containing the anomaly type and associated device identifier, and transmits the anomaly identification result to the cross-protocol device collaboration module.

[0111] Furthermore, the multimodal feature fusion unit is responsible for extracting effective features from different types of cloud data streams and dynamically allocating weights to provide accurate fused data for anomaly detection. The cross-type feature extractor and the attention weight allocator are linked through the cloud data bus of the IoT operating system. The cross-type feature extractor synchronously processes three types of cloud data streams: numerical, image, and text, extracts feature vectors, and preprocesses them to eliminate format differences and noise. For numerical data streams, time-series statistics and trend feature extraction are used to calculate four statistical features, such as the mean within a 5-minute sliding window, and two trend features, such as the trend slope, to form a 6-dimensional vector and normalize it to the 0-1 interval.

[0112] For the device appearance image data stream, an improved MobileNetV3 lightweight CNN is used to extract visual features, adapting to the lightweight computing power requirements of the cloud in the IoT operating system. The image is first preprocessed by grayscale conversion, 5×5 Gaussian filtering, and 224×224 size scaling. Then, features are extracted through 16 convolutional layers and 4 pooling layers, and a 256-dimensional vector is output to avoid consuming too many cloud computing resources.

[0113] For the device operation log text data stream, word embedding and semantic feature extraction are used. After preprocessing, it is converted into 50-dimensional word vectors by Word2Vec, and then semantic features are captured by LSTM to output 100-dimensional vectors. Furthermore, the time deviation of the three types of data streams is controlled within 100 milliseconds through the IoT operating system timestamp synchronization mechanism to ensure the time alignment of feature vectors.

[0114] The attention weight allocator dynamically determines the feature weights for different anomaly types based on historical anomaly data samples, improving the targeting of fusion and supporting weight updates. The weights are determined based on the anomaly sample library of the past year, covering three types of anomalies: data over-range, equipment appearance damage, and log error codes. The contribution of each feature is labeled. The weight allocation first matches the historical anomaly types of the equipment to determine the basic weights, and then adjusts them according to the real-time data integrity. If the integrity of a certain type of data is lower than 90%, the weight is reduced by 20% and assigned to the high integrity modality. After the weights are determined, the three types of vectors are weighted and fused to generate a 362-dimensional vector, which is transmitted to the anomaly judgment unit through the high-speed cloud channel of the IoT operating system.

[0115] The anomaly detection unit performs anomaly identification operations on the fused feature vectors and pushes the results to the cross-protocol device collaboration module. First, the feature vector processor receives the fused vectors and first constructs 100 trees through an isolated forest for preliminary screening. Anomalies with an anomaly score ≥0.6 are marked as suspected anomalies, taking ≤50 milliseconds. Then, a pre-trained SVM model is used to accurately determine the suspected anomalies, with the highest probability ≥80% to confirm the anomaly type. Otherwise, it is marked as pending confirmation. Every 7 days, the model parameters and thresholds are updated based on newly added data in the cloud. If the anomaly identification accuracy is lower than 90%, the model is retrained in the background. At the same time, the anomalies and pending confirmation vectors are encrypted and stored for 1 year for subsequent training and source tracing.

[0116] The anomaly output device receives the calculation results, generates a result containing the device identifier of the anomaly type and processing suggestions, encrypts and pushes it to the collaboration module and issues tiered alarms. The result includes the anomaly identification time, device identifier, anomaly type confidence level, data source, and supplementary information according to the anomaly type, such as supplementing the numerical range for data exceeding the range and supplementing the coordinates for external damage. At the same time, it tracks the processing results of the collaboration module and synchronizes them to the IoT operating system device status database. Misjudged data is fed back to the processor to optimize the algorithm, forming a closed loop of anomaly identification, processing, feedback and optimization.

[0117] Example 5, please refer to Figures 1 to 6 As shown: An IoT data stream intelligent management system based on data processing, the cross-protocol device collaboration module is used to realize the linkage control of IoT devices based on communication protocol adaptation, including an abnormal instruction generation unit and a protocol adaptation and distribution unit.

[0118] The abnormal instruction generation unit of the cross-protocol device collaboration module includes an abnormal result receiver and a control instruction encoder;

[0119] The protocol adaptation and distribution unit of the cross-protocol device collaboration module includes a multi-protocol communication module;

[0120] The anomaly result receiver receives the anomaly identification results output by the cloud-based multimodal fusion detection module and extracts the anomaly type and associated device identifier;

[0121] The control command encoder generates control commands adapted to the associated devices based on the anomaly type and the associated device identifier;

[0122] The multi-protocol communication module receives the device control commands output by the abnormal command generation unit, converts the commands into the communication protocol format supported by the corresponding terminal device, and then transmits them to the terminal device.

[0123] Furthermore, the abnormal instruction generation unit is responsible for receiving and parsing the abnormal identification results from the cloud and generating matching instructions. The abnormal result receiver and the control instruction encoder are linked through the instruction data bus of the IoT operating system to ensure timely and accurate instruction generation. First, the abnormal result receiver receives the abnormal identification results from the cloud, extracts key information and verifies its validity, and also has the ability to cache and retransmit requests.

[0124] The abnormal result receiver adopts an encrypted reception and hierarchical parsing mechanism. It receives results through a TLS 1.3 encrypted link at a rate of 500 messages per second, verifies the RSA 2048-bit digital signature in real time, and requests retransmission up to 2 times if the signature fails. After parsing, it extracts the abnormal type, associated device identifier, abnormal confidence level, and abnormal occurrence time, and verifies the device existence and online status with the device identity database of the IoT operating system. If the device is offline, the result is temporarily stored in the cache and re-verified periodically.

[0125] The control command encoder generates adapted commands based on the anomaly type and device identifier, combined with the device control logic library of the IoT operating system. It has the ability to standardize the format and verify the legality. The control logic library adopts a three-dimensional mapping design of anomaly type, device type and control action, and supports remote expansion of new logic through the IoT operating system. The command generation is divided into three steps: logic matching to determine the control action and parameters, encoding according to the device protocol format, and verifying whether the parameters are within the device's safety range and adjusting them to the safety threshold. At the same time, the control command encoder supports command version compatibility, automatically adapts to different protocol versions of the same device, and adds a unique ID and timestamp to each command for easy tracking.

[0126] The protocol communication module adopts a hardware interface plus software protocol stack design. The hardware integrates Ethernet, LoRa, ZigBee, Wi-Fi, and 4G / 5G interfaces, while the software has 15+ built-in IoT protocol stacks that can be expanded through firmware updates. Its core functions are protocol conversion and delivery. First, the target protocol is determined from the IoT operating system's device identity library by using the instruction device identifier. If the instruction format does not match, the algorithm is called to convert it, and then the protocol is delivered through the corresponding interface. The multi-protocol communication module also has a status monitoring and retransmission optimization mechanism. After delivery, if the device response timeout occurs, the protocol settings are followed. If there is no response, the protocol type is retransmitted. If the retransmission fails, a backup path is switched.

[0127] In this invention, an IoT data stream intelligent management system based on data processing is provided. After the IoT terminal device collects the data stream, it transmits it to the edge dynamic compression module. The spatiotemporal slicing processing unit adjusts the acquisition frequency through a dynamic period adjuster. The difference filter performs difference calculation on adjacent sensor data of the same type, retains the valid segments with differences exceeding the threshold, and the compressed data is temporarily stored in the embedded storage chip.

[0128] When the embedded storage chip detects the transmission ready signal of the quantum encryption communication module, it transmits the temporary data to the quantum encryption chip. The micro quantum key generation unit generates quantum states through an integrated quantum optical chip and generates a 256-bit or 512-bit encryption key through a key synthesis circuit. After the quantum encryption chip encrypts the data with the key, it is transmitted to the IoT communication control module by the narrowband communication module.

[0129] The dynamic protocol conversion unit of the IoT communication management module extracts the data protocol identifier through the communication frame parser, and the conversion rule generator generates conversion rules by matching multiple protocol libraries. The intelligent resource scheduling unit dynamically allocates computing resources and communication bandwidth based on the device operating parameters obtained by the scene parameter collector through the reinforcement learning scheduler. The device remote communication management unit synchronously accesses the device identity information and protocol type.

[0130] After the data is transmitted to the cloud, the cross-type feature extractor of the cloud multimodal fusion detection module extracts feature vectors from the numerical data stream, the device appearance image data stream, and the device operation log text data stream, respectively. The attention weight allocator assigns feature weights based on historical abnormal samples. The weighted feature vectors are processed by the feature vector processor to identify anomalies through the isolated forest-SVM algorithm. The anomaly result outputter generates anomaly results containing associated device identifiers and transmits them to the cross-protocol device collaboration module.

[0131] During the coordinated control phase, the abnormal result receiver of the cross-protocol device coordination module extracts the abnormal type and device identifier. The control command encoder, in conjunction with the device control logic library, generates control commands that adapt to the protocol. The multi-protocol communication module converts the commands into a protocol format supported by the terminal device and sends them out. After the terminal device executes the commands, it provides feedback on the execution status, forming an end-to-end closed-loop management of data stream acquisition, compression, encryption, control, detection, and regulation.

[0132] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. An Internet of Things data stream intelligent management system based on data processing, characterized in that, The Internet of Things data stream intelligent management system comprises an edge dynamic compression module, a quantum encryption communication module, an Internet of Things communication management module, a cloud multi-modal fusion detection module and a cross-protocol device cooperation module. The edge dynamic compression module is used for compressing the data stream collected by the Internet of Things terminal device to optimize the data transmission efficiency, and comprises a space-time slicing processing unit and an edge temporary storage unit. The space-time slicing processing unit of the edge dynamic compression module comprises a dynamic cycle adjuster and a difference filter. The dynamic cycle adjuster adopts an adaptive frequency sensing technology to adjust the collection cycle according to the data stream variation frequency to match the data transmission demand. The difference filter performs a difference operation on the data streams collected by adjacent sensors of the same type, retains the data stream segments whose difference exceeds a certain value, and transmits the selected data stream segments to the edge temporary storage unit. The quantum encryption communication module is used for encrypting the compressed data stream for transmission to ensure communication security, and comprises a miniature quantum key generation unit and an encryption transmission unit. The miniature quantum key generation unit of the quantum encryption communication module comprises an integrated quantum optical chip and a key synthesis circuit. The integrated quantum optical chip is used for generating and detecting quantum states, and the key synthesis circuit generates an encryption key based on the quantum state detection result, which is used for data stream transmission encryption to improve communication security. The Internet of Things communication management module is used for managing and controlling the communication protocol adaptation, data stream transmission scheduling and device communication state management among Internet of Things devices, and comprises a dynamic protocol conversion unit, an intelligent resource scheduling unit and a device remote communication management unit. The dynamic protocol conversion unit of the Internet of Things communication management module comprises a communication frame analyzer and a conversion rule generator. The communication frame analyzer scans the communication frame structure characteristics of the access device and extracts the protocol identification information. The conversion rule generator matches the built-in multi-protocol library to generate protocol conversion rules to realize heterogeneous device communication compatibility. The cloud multi-modal fusion detection module is used for identifying abnormalities of the data stream transmitted to the cloud to ensure the validity of the communication data, and comprises a multi-modal feature fusion unit and an abnormality judgment unit. The multi-modal feature fusion unit of the cloud multi-modal fusion detection module comprises a cross-type feature extractor and an attention weight distributor. The abnormality judgment unit of the cloud multi-modal fusion detection module comprises a feature vector processor and an abnormality result outputter. The cross-type feature extractor extracts feature vectors from numerical data streams, device appearance image data streams and device running log text data streams. The attention weight distributor determines the feature vector weights corresponding to different abnormal types according to historical abnormal data samples, and transmits the weighted feature vectors to the feature vector processor of the abnormality judgment unit. The feature vector processor receives the weighted feature vectors and performs abnormality recognition algorithm operation on the feature vectors. The abnormality result outputter receives the operation result of the feature vector processor, generates an abnormality recognition result containing the abnormal type and the associated device identifier, and transmits the abnormality recognition result to the cross-protocol device cooperation module. The cross-protocol device coordination module is used for linkage regulation and control of the Internet of Things device based on communication protocol adaptation, and includes an abnormal instruction generation unit and a protocol adaptation issuing unit.

2. The data processing based IoT data stream intelligent management system according to claim 1, characterized in that, The edge temporary storage unit of the edge dynamic compression module includes an embedded storage chip; The embedded storage chip is in communication connection with the space-time slice processing unit through a data interface, receives and stores the compressed data stream segments output by the space-time slice processing unit, and transmits the stored compressed data stream segments to the quantum encryption communication module when the embedded storage chip detects a transmission ready signal of the quantum encryption communication module. 3.The data processing based IoT data stream intelligent management system according to claim 1, characterized in that, The encryption transmission unit of the quantum encryption communication module includes a narrowband communication module and a quantum encryption chip; The quantum encryption chip is in communication connection with the micro quantum key generation unit, and uses the generated encryption key to encrypt the data stream segments; The narrowband communication module transmits the encrypted data stream to the Internet of Things communication management and control module.

4. The data processing based IoT data stream intelligent management system according to claim 1, characterized in that, The intelligent resource scheduling unit of the Internet of Things communication management and control module includes a scene parameter collector and a reinforcement learning scheduler; The scene parameter collector acquires device operation parameters; The reinforcement learning scheduler dynamically adjusts the reward function weight based on the operation parameters, and allocates computing resources and communication bandwidth.

5. The data processing based IoT data stream intelligent management system according to claim 1, characterized in that, The device remote communication management and control unit of the Internet of Things communication management and control module includes a device identity register and a firmware remote updater; The device identity register records the unique hardware identity and communication protocol type of the access device; The firmware remote updater pushes a firmware update package adapted to the communication protocol of the device to the device through an encrypted communication link.

6. The data processing based IoT data stream intelligent management system according to claim 1, wherein, The abnormal instruction generation unit of the cross-protocol device coordination module includes an abnormal result receiver and a control instruction encoder; The protocol adaptation issuing unit of the cross-protocol device coordination module includes a multi-protocol communication module; The abnormal result receiver receives the abnormality recognition result output by the cloud multi-modal fusion detection module, extracts the abnormality type and associated device identifier; The control instruction encoder generates a control instruction adapted to the associated device according to the abnormality type and associated device identifier; The multi-protocol communication module receives the device control instruction output by the abnormal instruction generation unit, converts the instruction into a communication protocol format supported by the corresponding terminal device, and transmits the instruction to the terminal device.

Images