Self-adaptive real-time streaming data rapid processing method and system and storage medium
By employing an adaptive sliding window and hierarchical data fusion approach, the speed and privacy issues in multimodal heterogeneous real-time streaming data processing are addressed, achieving efficient and secure data processing that adapts to the real-time streaming data needs of different scenarios.
Patent Information
- Application Number
- CN202511071806.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-31
- Publication Date
- 2025-12-09
AI Technical Summary
Existing technologies are unable to effectively process multimodal heterogeneous real-time streaming data, resulting in slow processing speed, data loss, asynchrony, spatiotemporal differences, and high risks of privacy leakage, failing to meet the comprehensive requirements of real-time performance, accuracy, and resource utilization.
An adaptive sliding window mechanism is used for preprocessing and time synchronization. Data is acquired through multimodal sensors, hierarchical data fusion and dynamic task allocation are performed, and an improved Kalman filter and attention-gated recurrent network are combined to achieve spatiotemporal consistency correction of data and perform privacy protection processing.
It improves processing efficiency, reduces latency and resource consumption, ensures data integrity and privacy, and adapts to the real-time streaming data processing needs of different scenarios.
Smart Images
Figure CN121098809A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of data processing technology, specifically relating to an adaptive real-time streaming data fast processing method, system, and storage medium. Background Technology
[0002] With the rapid development of IoT technology, the demand for real-time streaming data processing is increasing. In application scenarios such as smart homes and smart cities, real-time streaming data generated by sensors is characterized by multimodality, heterogeneity, and high speed. Traditional data processing methods often struggle to meet the comprehensive requirements of real-time performance, accuracy, and resource utilization.
[0003] For example, in a home environment, real-time streaming data includes visual and acoustic data streams, which have different structural characteristics and typically require different processing methods. Visual data streams usually require sequential object detection, image segmentation, and subsequent fusion; acoustic data streams require A / D and D / A conversion. Traditional processing methods are inefficient and cannot meet the demands of real-time processing. Existing real-time streaming data processing technologies face significant challenges in terms of processing speed and accuracy when dealing with large-scale real-time streaming data, and have the following shortcomings:
[0004] 1. Most traditional methods, systems, and devices rely on single-threaded or inefficient processing mechanisms during data processing, which cannot effectively achieve parallel processing of multimodal data, resulting in slow processing speeds. In particular, when processing large-scale real-time streaming data, the system response time is unacceptable.
[0005] 2. Existing feature extraction and data fusion methods are mostly limited to processing visual and acoustic data separately, lacking an effective combination of the two, resulting in data loss, asynchrony, spatiotemporal differences, and low processing efficiency.
[0006] 3. For certain application scenarios, there is a certain amount of private information. Uploading it to the cloud for processing carries a certain risk of leakage, while keeping it locally for processing cannot guarantee processing efficiency.
[0007] Therefore, when faced with multimodal heterogeneous real-time streaming data, how to improve processing efficiency while ensuring data accuracy and integrity is an urgent problem to be solved in data processing technology under artificial intelligence scenarios. Summary of the Invention
[0008] To address the aforementioned problems, this invention provides an adaptive real-time streaming data fast processing method, system, and storage medium to resolve the issues in the prior art.
[0009] To achieve the aforementioned objectives, this invention proposes an adaptive real-time streaming data fast processing method, which includes:
[0010] The acquired real-time streaming data is preprocessed, spatially aligned, and time-synchronized using a sliding window mechanism to obtain a set of feature vectors. The feature vector set is then subjected to hierarchical data fusion, and processing tasks are dynamically allocated based on the fusion results.
[0011] Furthermore, the specific methods are as follows:
[0012] S1. Continuously acquire real-time streaming data of the target space, including environmental parameters, equipment status, and biological characteristics, through multimodal sensors; establish an adaptive sliding window and dynamically preprocess the acquired real-time streaming data;
[0013] S2. Perform time alignment and time synchronization on the dynamically preprocessed real-time streaming data to generate a set of feature vectors with a unified spatiotemporal reference.
[0014] S3. In the first fusion stage, the feature vector set is corrected for spatiotemporal consistency, and in the second fusion stage, the multidimensional state tensor of the living space is output through the attention-gated recurrent network.
[0015] S4. Based on the confidence score of the multidimensional state tensor of the living space and the resource load index of the edge node, dynamically allocate data processing tasks to the local computing unit or the cloud server.
[0016] Furthermore, the window length W of the adaptive sliding window in step S1 is determined according to the following formula:
[0017]
[0018] Where β is a scaling factor that is dynamically adjusted according to the sensor type, with a value range of [0.5, 2.0], V is the data arrival rate, C is the number of available processor cores, and B is the current network bandwidth.
[0019] Furthermore, in step S2, a coordinate system mapping based on depth matrix transformation is performed on the visual data stream in the real-time streaming data, and a time-frequency domain feature transformation is performed on the acoustic data stream to generate a set of feature vectors with a unified spatiotemporal reference.
[0020] Furthermore, in step S3, the first fusion stage employs an improved Kalman filter to correct the spatiotemporal consistency of physical quantity measurements; the process noise matrix Q of the improved Kalman filter is dynamically configured as follows:
[0021] Q ij =σ i ×σ j ×ρ ij
[0022] Where σ i ρ represents the calibration error coefficient of the i-th type of sensor.ij The correlation index represents the data correlation between two types of sensors, i and j, which is obtained by calculating the covariance matrix of historical data.
[0023] Furthermore, the specific method for coordinate system mapping is as follows:
[0024] A reference coordinate system based on the building floor plan of the target space is established, and the sensor data are converted to a unified coordinate space through the pre-calibrated sensor spatial parameter matrix H∈R^(4×4).
[0025] Furthermore, it also includes methods for handling abnormal states, as follows:
[0026] A dynamic baseline model based on a sliding time window is constructed, and a multi-level response mechanism is triggered when the Mahalanobis distance between the fused data and the baseline continuously exceeds the third threshold.
[0027] Furthermore, in step S2, privacy protection processing is required for the real-time streaming data, as detailed below:
[0028] Differential privacy coding is performed on human feature regions in the visual data stream, and feature decoupling is performed on the speech content in the acoustic data stream.
[0029] The present invention also provides an adaptive real-time streaming data fast processing system for implementing the above method, the system comprising:
[0030] The multi-source heterogeneous data acquisition module, composed of multimodal sensors, is used to acquire real-time streaming data of the target space, including environmental parameters, equipment status, and biological characteristics.
[0031] The data processing module is used to dynamically preprocess, time-align, and time-synchronize real-time streaming data, generating a set of feature vectors with a unified spatiotemporal reference.
[0032] The hierarchical fusion module is used to perform hierarchical data fusion on the feature vector set and dynamically allocate processing tasks based on the fusion results.
[0033] The present invention also provides a computer storage medium storing program instructions for performing the above-described methods.
[0034] Compared with the prior art, the present invention has the following beneficial effects:
[0035] 1. Through the adaptive sliding window mechanism, the processing strategy can be dynamically adjusted for different artificial intelligence application scenarios and real-time streaming data types, thereby improving the flexibility and adaptability of the invention.
[0036] 2. Through hierarchical data fusion and dynamic task allocation, efficient processing of large-scale real-time streaming data is achieved, reducing processing latency and resource consumption;
[0037] 3. It can perform privacy protection processing on real-time streaming data, effectively protecting users' privacy information, preventing privacy leaks, and improving security. Attached Figure Description
[0038] Figure 1 A flowchart of the steps of an adaptive real-time streaming data fast processing method of the present invention;
[0039] Figure 2 The system architecture diagram of the adaptive real-time streaming data fast processing system in this invention. Detailed Implementation
[0040] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0041] Example 1
[0042] This embodiment provides an adaptive real-time streaming data fast processing method, such as... Figure 1 As shown, the method is as follows: the acquired real-time streaming data is preprocessed, spatially aligned, and time-synchronized using a sliding window mechanism to obtain a set of feature vectors; the feature vector set is then subjected to hierarchical data fusion, and processing tasks are dynamically allocated based on the fusion results.
[0043] The specific method is as follows:
[0044] S1. Continuously acquire real-time streaming data of the target space, including environmental parameters, equipment status, and biological characteristics, through multimodal sensors; establish an adaptive sliding window and dynamically preprocess the acquired real-time streaming data;
[0045] The window length W of the above adaptive sliding window is determined according to the following formula:
[0046]
[0047] Where β is a scaling factor that is dynamically adjusted according to the sensor type, with a value range of [0.5, 2.0], V is the data arrival rate, C is the number of available processor cores, and B is the current network bandwidth.
[0048] S2. Perform time alignment and time synchronization on the dynamically preprocessed real-time streaming data to generate a set of feature vectors with a unified spatiotemporal reference.
[0049] Specifically, coordinate system mapping based on depth matrix transformation is performed on the visual data stream in real-time streaming data, and time-frequency domain feature transformation is performed on the acoustic data stream to generate a set of feature vectors with a unified spatiotemporal reference.
[0050] The specific method for mapping the above coordinate system is as follows:
[0051] Establish a reference coordinate system based on the building floor plan of the target space, and transform the data of each sensor to a unified coordinate space through the pre-calibrated sensor spatial parameter matrix H∈R^(4×4);
[0052] In addition, real-time streaming data also needs to undergo privacy protection processing, as follows:
[0053] Differential privacy coding is performed on human feature regions in the visual data stream, and feature decoupling is performed on speech content in the acoustic data stream;
[0054] S3. In the first fusion stage, an improved Kalman filter is used to correct the spatiotemporal consistency of physical quantity measurements; the process noise matrix Q of the improved Kalman filter is dynamically configured as follows:
[0055] Q ij =σ i ×σ j ×ρ ij
[0056] Where σ i ρ represents the calibration error coefficient of the i-th type of sensor. ij The correlation index represents the data correlation index between two types of sensors, i and j, which is obtained by calculating the covariance matrix of historical data.
[0057] S4. Based on the confidence score of the multidimensional state tensor of the living space and the resource load index of the edge node, dynamically allocate data processing tasks to the local computing unit or the cloud server.
[0058] When network latency exceeds a second threshold, a lightweight inference engine is activated to generate a set of control instructions. This lightweight inference engine includes:
[0059] The pre-trained device control decision tree model receives the fused multidimensional state tensor at its input layer and generates a set of control instructions that matches the environmental control device protocol at its output layer. The control instruction set is then distributed to the target device via the MQTT protocol.
[0060] In addition to steps S1 to S4 above, the abnormal state handling methods are as follows:
[0061] Construct a dynamic baseline model based on a sliding time window. When the Mahalanobis distance between the fused data and the baseline continuously exceeds a third threshold, a multi-level response mechanism is triggered, which may include:
[0062] Level 1 Response: Emergency adjustment of local device status;
[0063] Level 2 Response: Generate an encrypted alarm message and upload it to the regulatory platform;
[0064] Level 3 response: Activate backup sensors to verify data.
[0065] Following the methods provided in the above embodiments, streaming data processing tests were conducted in an experimental environment with a scale of 100 nodes based on an Apache Flink 3.0 cluster. The various indicators and comparisons are as follows:
[0066]
[0067]
[0068] As can be seen from the comparison of the data in the table above, the streaming data processing method provided in this embodiment significantly improves data processing efficiency while effectively ensuring data integrity compared with existing processing methods.
[0069] Example 2
[0070] This embodiment is an adaptive real-time streaming data fast processing system, used to implement the method involved in Embodiment 1, such as... Figure 2 As shown, the system includes:
[0071] The multi-source heterogeneous data acquisition module, composed of multimodal sensors, is used to acquire real-time streaming data of the target space, including environmental parameters, equipment status, and biological characteristics.
[0072] The data processing module is used to dynamically preprocess, time-align, and time-synchronize real-time streaming data, generating a set of feature vectors with a unified spatiotemporal reference.
[0073] The hierarchical fusion module is used to perform hierarchical data fusion on the feature vector set and dynamically allocate processing tasks based on the fusion results.
[0074] Example 3
[0075] This embodiment provides a computer storage medium that stores program instructions, which, when executed, control the device where the computer storage medium is located to perform the above-described method.
[0076] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.
[0077] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.
[0078] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0079] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this patent should be determined by the appended claims.
Claims
1. An adaptive real-time streaming data fast processing method, characterized in that: The method involves preprocessing, spatially aligning, and temporally synchronizing the acquired real-time streaming data using a sliding window mechanism to obtain a set of feature vectors. The feature vector set is subjected to hierarchical data fusion, and processing tasks are dynamically allocated based on the fusion results.
2. The adaptive real-time streaming data fast processing method according to claim 1, characterized in that: The specific method is as follows: S1. Continuously acquire real-time streaming data of the target space, including environmental parameters, equipment status, and biological characteristics, through multimodal sensors; establish an adaptive sliding window and dynamically preprocess the acquired real-time streaming data; S2. Perform time alignment and time synchronization on the dynamically preprocessed real-time streaming data to generate a set of feature vectors with a unified spatiotemporal reference. S3. In the first fusion stage, the feature vector set is corrected for spatiotemporal consistency, and in the second fusion stage, the multidimensional state tensor of the living space is output through the attention-gated recurrent network. S4. Based on the confidence score of the multidimensional state tensor of the living space and the resource load index of the edge node, dynamically allocate data processing tasks to the local computing unit or the cloud server.
3. The adaptive real-time streaming data fast processing method according to claim 2, characterized in that: The window length W of the adaptive sliding window in step S1 is determined according to the following formula: Where β is a scaling factor that is dynamically adjusted according to the sensor type, with a value range of [0.5, 2.0], V is the data arrival rate, C is the number of available processor cores, and B is the current network bandwidth.
4. The adaptive real-time streaming data fast processing method according to claim 2, characterized in that: In step S2, coordinate system mapping based on depth matrix transformation is performed on the visual data stream in the real-time streaming data, and time-frequency domain feature transformation is performed on the acoustic data stream to generate a set of feature vectors with a unified spatiotemporal reference.
5. The adaptive real-time streaming data fast processing method according to claim 2, characterized in that: The first fusion stage in step S3 employs an improved Kalman filter to correct the spatiotemporal consistency of physical quantity measurements; the process noise matrix Q of the improved Kalman filter is dynamically configured as follows: Q ij =s i ×s j ×r ij Where σ i ρ represents the calibration error coefficient of the i-th type of sensor. ij The correlation index represents the data correlation between two types of sensors, i and j, which is obtained by calculating the covariance matrix of historical data.
6. The adaptive real-time streaming data fast processing method according to claim 4, characterized in that: The specific method for coordinate system mapping is as follows: A reference coordinate system based on the building floor plan of the target space is established, and the sensor data are converted to a unified coordinate space through the pre-calibrated sensor spatial parameter matrix H∈R^(4×4).
7. The adaptive real-time streaming data fast processing method according to claim 1, characterized in that: It also includes methods for handling abnormal states, as follows: A dynamic baseline model based on a sliding time window is constructed, and a multi-level response mechanism is triggered when the Mahalanobis distance between the fused data and the baseline continuously exceeds the third threshold.
8. The adaptive real-time streaming data fast processing method according to claim 4, characterized in that: In step S2, privacy protection processing is also required for the real-time streaming data, as detailed below: Differential privacy coding is performed on human feature regions in the visual data stream, and feature decoupling is performed on the speech content in the acoustic data stream.
9. An adaptive real-time streaming data fast processing system, used to implement the method as described in any one of claims 1 to 8, characterized in that: The system includes: The multi-source heterogeneous data acquisition module, composed of multimodal sensors, is used to acquire real-time streaming data of the target space, including environmental parameters, equipment status, and biological characteristics. The data processing module is used to dynamically preprocess, time-align, and time-synchronize real-time streaming data, generating a set of feature vectors with a unified spatiotemporal reference. The hierarchical fusion module is used to perform hierarchical data fusion on the feature vector set and dynamically allocate processing tasks based on the fusion results.
10. A computer storage medium, characterized in that: The computer storage medium stores program instructions for performing the method according to any one of claims 1 to 8.