Important information sorting method and device for traffic analysis based on deep learning, and storage medium

Abstract

The invention provides an important information sorting method and device for traffic analysis based on deep learning and a readable storage medium. Public traffic symbols and traffic content are used for training to generate a BERT model, repeated documents in the same kind are compared through MD5, and redundant junk traffic and repeated traffic are filtered out through an iterative dichotomy method; specific separators are obtained in a statistical mode for small flow, the flow is cut into flow sections, and every two flow sections are compared; if the comparison similarity is high, only one section is taken as the main flow; and if the comparison results are different, re-analysis is carried out for different flow sections, in a single flow section, an N-Gram algorithm is adopted to extract a flow piece sequence and carry out frequency statistics, and finally, the flow piece with the highest occurrence frequency is framed out to carry out integral sorting. And finally obtaining an important information sorting result of the traffic. The problems of high time consumption, low efficiency and no key point caused by mass flow analysis are effectively solved.

Description

technical field [0001] The present invention relates to the technical field of text mining, in particular to a method, device and storage medium for sorting important information based on deep learning for traffic analysis. Background technique [0002] With the rapid development of network technology and computer technology, the Internet has become an important way for people to obtain external information and communicate with each other. However, in recent years, traffic has become complicated, concealed, difficult to analyze, and so on. Therefore, the analysis of Internet traffic is very important at this time, and how to obtain useful information through Internet traffic is not open. At present, various artificial intelligence, big data and other technical fields are also emerging. Big data realizes the value of Internet traffic, and realizes the extraction of important information of traffic through massive data combined with deep learning analysis methods, and improve...

AI Technical Summary

Problems solved by technology

[0004] In order to solve the problems existing in the prior art that the analysis of massive traffic is time-consuming, inefficient, and unable to identify key content, the following technical solutions are proposed in the embodiments of the present invention:

Images