Streaming data processing method and device

Abstract

The invention discloses a streaming data processing device and method, and the device comprises: a data collection module which is used for collecting the streaming data of at least two data streams,and transmitting the streaming data of each data stream to a data generation module; a data generation module which is used for receiving the streaming data sent by the data acquisition module, integrating the streaming data of the at least two data streams into one or more streaming data tuples, and sending the streaming data tuples to the data output module; and a data output module which is used for outputting the streaming data according to the streaming data tuples integrated by the data generation module. According to the invention, the stream data of each data stream is collected, the stream data of all the data streams is integrated into one or more stream data tuples, and the stream data is output according to the stream data tuples, so that a data set with relatively aligned timecan be obtained, the problem of data time misalignment is solved, the stream data can be directly used for calculation, the calculation efficiency is improved, and therefore, a more rigorous calculation result can be obtained.

Description

technical field [0001] The invention relates to the field of computer technology, in particular to a stream data processing method and device. Background technique [0002] With the rise of the Internet of Things and stream computing technology, more and more sensor data are connected to the stream computing platform for real-time analysis and processing. If it is necessary to calculate the data of multiple sensors at a certain time point, it is difficult to accurately align the sensor data according to a certain time point due to the inconsistency of the acquisition time and acquisition cycle of different sensor data acquisitions. [0003] For example, there are sensors A and B. A starts from 0 seconds and collects data every 10 seconds, while B starts from the 5th second and collects data every 20 seconds. The data is entered in Table 1 below. [0004] Table 1 [0005] time(s) 0 5 10 15 20 25 30 35 40 45 Sensor A A1 A2 A3 A4 A5 ...

AI Technical Summary

Problems solved by technology

[0009] It can be seen that the data A1 of sensor A at the 0th second is aligned with the data B1 of sensor B at the 5th second, and each subsequent data is misplaced in turn. If the data is directly accumulated as an input stream, it will cause the problem of data time misalignment, which cannot be used directly. calculations and analytical calculations based on two-dimensional tables, or lead to imprecise calculation results

Images