Method for extracting, analyzing and searching network flow and content

Abstract

The invention discloses a method for extracting, analyzing and searching network flow and content. The method comprises the following steps: shunting original flow into n data processing queues; independently processing an original data message of each data processing queue by the data processing queue, performing protocol recognition and filtration on the message and performing conversation recombination on TCP (Transmission Control Protocol) flow in the message; performing protocol resolving and decoding on a recombined TCP conversation and extracting out structured data information therein; and as for key information specified by requirements, performing searching labeling in data content extracted by a content resolving and extracting module based on a multimode matching algorithm or a search engine technology, and submitting labeling results to a searching labeling information database, thereby providing searching labeling results for multiple modes of applications. The method can be used for solving the problems of repeated data packets, serial number zero adjustment and the like in the TCP conversation recombination, realizing the character labeling for the original flow, and ensuring that a user can acquire effective information conveniently.

Description

technical field [0001] The invention relates to the technical field of the Internet, in particular to a method for performing content analysis and key information retrieval and labeling on network traffic. Background technique [0002] With the development of Internet technology, network information security has become the focus of the industry. On the one hand, as large as the national network security supervision department, as small as families and individuals, they all need to maintain the stability of the network system, monitor network information, and prevent the spread of illegal or unsafe information; on the other hand, network devices based on content analysis or filtering And network security products also urgently need more effective and comprehensive testing before they can be put into use. There are two main problems at present: first, the data transmitted in the network is very complex, and the amount of information is explosively increasing, and the binary d...

AI Technical Summary

Problems solved by technology

An existing network traffic restoration method such as figure 1 As shown, although it realizes the analysis and restoration of traffic, it has shortcomings: first, the range of restored data is too wide, the amount of information is large, and the key information that users are concerned about cannot be effectively retrieved; second, the integrity of the original data is required Very high, if any data packet is lost in a session, the session will not be able to restore the application layer file, so the information in the traffic cannot be fully restored in actual use; in addition, after the traffic data is processed and restored, only the web page can be obtained Files, audio / video files, document files, binary files and other conventional types of application layer files, subsequent applications can only extract information based on a large number of application layer files, as regular data for monitoring, the traffic analysis information cannot be fully utilized

Images