Stream processing method and system for online filling of missing network data

Abstract

The invention discloses a stream processing method for online filling of missing data in the network, comprising: during the process of stream processing the missing data in the network, using a feature extractor and a gating cycle unit to implicate the monitoring data matrix sequence in the previous period The time and space information of the network can be extracted, and the context vector that retains the effective information in the historical data can be obtained, and it can be combined with the spatial feature vector information corresponding to the missing network data at the current moment, and the joint vector has been input into the pre-trained missing data generation model to obtain the current monitoring data matrix after filling in the missing data of the network. The stream processing method and system for online filling of missing network data provided by the present invention combines the spatial feature vector corresponding to the missing network data with the context vector related to the previous historical data to integrate the spatiotemporal information of the network data in the previous period, and can Effectively improve the accuracy of online filling of missing data in the network.

Description

technical field [0001] The invention belongs to the technical field of computer networks, and more specifically relates to a stream processing method and system for online filling of network missing data. Background technique [0002] With the rapid development of communication technology, the scale of the network that needs to be maintained is also increasing. In the process of network operation and maintenance, it is necessary to measure network monitoring data for subsequent tasks such as anomaly detection, root cause analysis, and traffic prediction. However, the cost of performing network-wide measurement on a large-scale network is very high, and frequent measurement will also occupy a large amount of network resources. [0003] Studies have shown that network monitoring data has temporal correlation and spatial correlation, so the current mainstream processing method is to measure part of the data and restore the unmeasured data according to the correlation. Since t...

AI Technical Summary

Problems solved by technology

[0007] In view of the above defects or improvement needs of the prior art, the present invention provides a stream processing method and system for online filling of missing data in the network. The matrix is ​​used as input, and each sequence needs to train parameters from scratch, which leads to the technical problem of greatly increasing the time cost of filling, and the existing machine learning-based filling method does not train a filling model with fixed parameters through historical data. It is applicable to the technical problems of filling network data, as well as the technical problems of slow convergence speed of the model due to failure to consider the characteristics of network data conforming to the heavy-tailed distribution, and the defect of unbalanced gradient update in the process of training the model, resulting in the final The technical problem of low filling accuracy

Images