Window-based method for missing streaming data

Abstract

The invention belongs to the technical field of information retrieval and its database structure, and discloses a window-based stream data missing processing method, comprising: inputting a data stream containing missing data to be processed; judging whether the stream data is missing; dynamic sliding window evaluation ;Window locking; Incomplete information processing; Output the complete data flow result after processing. By providing a sliding window for streaming data, the window size is dynamically adjusted according to the size of the data stream; the missing data is filled by using the internal time and space characteristics of the data, and the missing data of the streaming data is filled. The invention has the characteristics of high classification accuracy, low runtime complexity, high computing resource utilization rate, etc., and effectively overcomes the problems of single mode and high time-space complexity in the existing processing methods.

Description

technical field [0001] The invention belongs to the technology and field of wireless communication, information retrieval and database structure thereof, and in particular relates to a window-based stream data missing processing method. Background technique [0002] At present, the existing technologies commonly used in the industry are as follows: Sensor networks are usually used to monitor physical or environmental conditions in different locations. After the network acquires data, it will be transmitted to the data center for analysis and display. In the node environment, usually due to the limitations of the sensor network's own hardware and software, coupled with the impact of various unfavorable factors such as weather, man-made, electromagnetic radiation, and unstable transmission channels during transmission to the center, the sensor network In the process of monitoring and obtaining information, the nodes in the network will have missing data, and the data received...

AI Technical Summary

Problems solved by technology

[0005] (1) The three-stage method is used to fill in a single form, which cannot solve the problem of multiple missing fillings, and cannot meet the low-latency conditions required for streaming data processing

[0006] (2) Use the classifier to classify and use the classified data to fill in the missing data. This method cannot solve the classification problem of data with a relatively large relationship between internal data attributes

[0007] (3) The existing technology is mainly aimed at data filling of static data sets. When faced with the missing processing problem of streaming data, its method has high time complexity and space complexity in calculation, and does not solve the flow control problem of data flow

The data flow collected in the sensor network will be missing due to the influence of sensor accuracy, sensitivity and some factors in data transmission. In the process of data center analysis and display, the missing data will reduce the amount of useful information. At the same time, the central processing and Information display is difficult, so missing handling of streaming data is important

Images