Methods, apparatus, equipment and storage media for extracting network analysis data

Abstract

This application discloses a method, apparatus, electronic device, and computer-readable storage medium for extracting network analysis data, belonging to the Internet field. The method includes: capturing a data packet to be detected from a device port and determining the network protocol type of the data packet; determining a data identifier bit of the data packet to be detected based on the network protocol type of the data packet from a preset first correspondence; determining a target analysis process template corresponding to the data identifier bit of the data packet from a preset second correspondence; and analyzing the content to be detected in the data packet to be detected, starting from the data identifier bit, according to the steps in the target analysis process template. This solves the problem in related technologies where the large scale and complexity of network traffic data result in a high data parsing load and long processing time, leading to low data parsing efficiency and the inability to quickly extract network traffic data.

Description

Technical Field

[0001] This application belongs to the Internet field, and specifically relates to a method, apparatus, electronic device, and computer-readable storage medium for analyzing network data. Background Technology

[0002] Passive eavesdropping refers to monitoring and capturing network traffic during transmission without interfering with or modifying it, in order to obtain relevant information such as traffic patterns, communication modes, application usage, and security events. Compared to active eavesdropping, passive eavesdropping has advantages such as not affecting normal network operation, not causing resentment among network users, and not violating network compliance.

[0003] Methods for extracting passively monitored network traffic data typically require specialized packet capture and analysis tools, as well as a dedicated storage system. Packet capture tools are used to capture data packets from network transmission ports, packet analysis tools are used for deep parsing, protocol identification, and application identification of the packets, and storage systems are used to store and manage the data packets. The performance and efficiency of these tools and systems directly affect the quality and speed of network traffic data extraction.

[0004] However, existing methods for extracting network traffic data from passive monitoring are inefficient because the scale and complexity of network traffic data require complete parsing of different data packets each time, consuming a lot of computing resources and time, and thus failing to extract network traffic data quickly. Summary of the Invention

[0005] This application aims to provide a method, apparatus, electronic device, and computer-readable storage medium for analyzing network data, at least to solve the problems of high data parsing load and long processing time in the process of extracting network analysis data.

[0006] In a first aspect, embodiments of this application disclose a method for analyzing network data, including:

[0007] Capture the data packets to be detected from the device's port and determine the network protocol type of the data packets to be detected; the data packets to be detected contain the content to be detected, as well as other content besides the content to be detected;

[0008] From the preset first correspondence, the data identifier bit of the detected data packet is determined according to the network protocol type of the detected data packet; the first correspondence is used to store the correspondence between the network protocol type and the data identifier bit; the data identifier bit is used to characterize the starting position of the content to be detected in the detected data packet;

[0009] From the preset second correspondence, a target analysis process template corresponding to the data identifier bit of the detected data packet is determined; the second correspondence stores the correspondence between the data identifier bit and the analysis process template; the analysis process template is used to record the specific steps for analyzing the content to be detected in the detected data packet, starting from the data identifier bit, under the corresponding network protocol type.

[0010] Following the steps in the target analysis process template, starting from the data identifier bit of the detected data packet, the content to be detected in the detected data packet is analyzed.

[0011] Secondly, embodiments of this application also disclose a network data analysis device, the device comprising:

[0012] The capture module is used to capture the data packets to be detected from the device's port and determine the network protocol type of the data packets to be detected; the data packets to be detected contain the content to be detected, as well as other content besides the content to be detected.

[0013] The first relationship module is used to determine the data identifier bit of the detected data packet from a preset first correspondence relationship according to the network protocol type of the detected data packet; the first correspondence relationship is used to store the correspondence between network protocol types and data identifier bits; the data identifier bit is used to characterize the starting position of the content to be detected in the detected data packet;

[0014] The second relationship module is used to determine the target analysis process template corresponding to the data identifier bit of the detected data packet from a preset second correspondence relationship; the second correspondence relationship stores the correspondence between the data identifier bit and the analysis process template; the analysis process template is used to record the specific steps of analyzing the content to be detected of the detected data packet, starting from the data identifier bit, under the corresponding network protocol type.

[0015] The data analysis module is used to analyze the content to be detected in the data packet, starting from the data identifier bit of the data packet to be detected, according to the steps in the target analysis process template.

[0016] Thirdly, embodiments of this application also disclose an electronic device, including a processor and a memory, wherein the memory stores a program or instructions that can run on the processor, and the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0017] Fourthly, embodiments of this application also disclose a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0018] In summary, in this embodiment, by determining the data identifier of the detected data packet from a preset first correspondence based on the network protocol type of the detected data packet, and by determining the target analysis process template corresponding to the data identifier of the detected data packet from a preset second correspondence, the starting position of the content to be detected in the detected data packet can be quickly located. Furthermore, different analysis process templates are used according to different network protocol types, achieving customized analysis of the content to be detected in the detected data packet. Based on the target analysis process template, the content to be detected in the detected data packet is analyzed starting from the data identifier, and the required network traffic data can be extracted without affecting the normal operation of the network. Therefore, the method based on this embodiment does not require complete parsing of each data packet when analyzing network data, solving the problem in related technologies where the large scale and complexity of network traffic data leads to a large data parsing load, long processing time, and low data parsing efficiency, making it impossible to quickly extract network traffic data. Attached Figure Description

[0019] In the attached diagram:

[0020] Figure 1 This is a flowchart illustrating the steps of a network data analysis method provided in this embodiment;

[0021] Figure 2 A flowchart illustrating the steps of another network data analysis method provided in the application embodiments;

[0022] Figure 3 This is an architecture diagram of a network data analysis method provided in an embodiment of this application;

[0023] Figure 4 This is a program logic diagram of a network data analysis method provided in an embodiment of this application;

[0024] Figure 5 This is a block diagram of a network data analysis device provided in an embodiment of this application;

[0025] Figure 6 This is a block diagram of an electronic device provided in one embodiment of this application;

[0026] Figure 7 This is a block diagram of an electronic device according to another embodiment of the present application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0029] Figure 1 This embodiment provides a method for analyzing network data.

[0030] The method may include the following steps:

[0031] Step 101: Capture the data packets to be detected from the device's port and determine the network protocol type of the data packets to be detected; the data packets to be detected contain the content to be detected, as well as other content besides the content to be detected.

[0032] In one embodiment of this application, data packets to be detected are captured from the device's port, and the network protocol type of the data packets to be detected is determined. The data packets to be detected refer to data packets acquired in real time from the network for network data analysis. The network protocol type refers to the network communication rules followed by the data packets, such as Transmission Control Protocol / Internet Protocol (TCP / IP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), etc.

[0033] For example, during traffic monitoring of a server, the monitored port is 1, and the network protocol type is TCP / IP. 1000 data packets are captured from port 1 of the router, each packet being 1KB in size. According to the TCP / IP protocol specification, the first 20 bytes of each packet are the IP header, followed by the TCP header and data. Therefore, the network protocol type is TCP / IP.

[0034] Step 102: Determine the data identifier bit of the detected data packet from the preset first correspondence relationship according to the network protocol type of the detected data packet; the first correspondence relationship is used to store the correspondence between network protocol type and data identifier bit; the data identifier bit is used to characterize the starting position of the content to be detected in the detected data packet.

[0035] In one embodiment of this application, the data identifier bit of the detected data packet is determined from a preset first correspondence relationship based on the network protocol type of the detected data packet. The first correspondence relationship is a data structure used to store the correspondence between different network protocol types and data identifier bits. The data identifier bit refers to a set of binary bits in the data packet used to mark the starting position of the content to be detected.

[0036] For example, in the embodiment of step 101, the corresponding data identifier bit can be found from the preset first correspondence, based on the network protocol type TCP / IP. Assuming the data identifier bit corresponding to the TCP / IP protocol in the first correspondence is 01100010, then the starting position of the content to be detected in the detected data packet can be determined based on this data identifier bit.

[0037] Step 103: Determine the target analysis process template corresponding to the data identifier bit of the detected data packet from the preset second correspondence relationship; the second correspondence relationship stores the correspondence between the data identifier bit and the analysis process template; the analysis process template is used to record the specific steps of analyzing the content to be detected of the detected data packet, starting from the data identifier bit, under the corresponding network protocol type.

[0038] In one embodiment of this application, a target analysis process template corresponding to the data identifier bit of the detected data packet is determined from a preset second correspondence. The second correspondence is a data structure used to store the correspondence between different data identifier bits and analysis process templates. The analysis process template is an algorithm or program used to record the specific steps for analyzing the content to be detected in the detected data packet, starting from the data identifier bit, under the corresponding network protocol type.

[0039] For example, in the embodiment of step 102, the corresponding target analysis process template is found from the preset second correspondence based on the data identifier bit (01100010). Assume that the analysis process template corresponding to data identifier bit 01100010 in the second correspondence is a program, and includes the following data analysis process: Input: The data packet to be detected; starting from the data identifier bit, read the TCP header and data; parse the TCP header to extract key information, such as source port, destination port, sequence number, and acknowledgment number; parse the TCP data to extract key information, such as application layer protocol and application layer data; select an appropriate analysis method based on the application layer protocol; analyze the application layer data to obtain raw data; perform further processing based on the raw data, such as statistics, classification, filtering, encryption, and compression; and output the analysis results.

[0040] Step 104: Following the steps in the target analysis process template, start analyzing the content to be detected in the data packet from the data identifier bit of the data packet being detected.

[0041] In one embodiment of this application, the content to be detected in the data packet is analyzed starting from the data identifier bit of the data packet being detected, according to the steps in the target analysis process template. The target analysis process template is an algorithm or program used to record the specific steps of analyzing the content to be detected in the data packet being detected, starting from the data identifier bit, under the corresponding network protocol type. The analysis result refers to the network traffic data extracted from the data packet being detected, such as application layer protocols, application layer data, raw data, etc.

[0042] For example, in the embodiment of step 103, an analysis process template corresponding to the identifier 01100010 has been obtained. In this step, the detected data packet is analyzed and processed starting from the identifier 0110010 position through this analysis process template to obtain the final analysis result.

[0043] In summary, in this embodiment, by determining the data identifier of the detected data packet from a preset first correspondence based on the network protocol type of the detected data packet, and by determining the target analysis process template corresponding to the data identifier of the detected data packet from a preset second correspondence, the starting position of the content to be detected in the detected data packet can be quickly located. Furthermore, different analysis process templates are used according to different network protocol types, achieving customized analysis of the content to be detected in the detected data packet. Based on the target analysis process template, the content to be detected in the detected data packet is analyzed starting from the data identifier, and the required network traffic data can be extracted without affecting the normal operation of the network. Therefore, the method based on this embodiment does not require complete parsing of each data packet when analyzing network data, solving the problem in related technologies where the large scale and complexity of network traffic data leads to a large data parsing load, long processing time, and low data parsing efficiency, making it impossible to quickly extract network traffic data.

[0044] Figure 2 Another method for analyzing network data provided in the application embodiments, referring to Figure 2 The method may include the following steps:

[0045] Step 201: Capture sampling data packets from the device's port, and determine the data identifier and network protocol type of the sampling data packets based on the sampling data packets; the sampling data packets contain the content to be detected, as well as other content besides the content to be detected.

[0046] In one embodiment of this application, sampled data packets are captured from the device's port, and the data identifier bits and network protocol type of the sampled data packets are determined based on the sampled data packets. The sampled data packets refer to a subset of data packets randomly selected from the network, used to establish a first correspondence and a second correspondence. The data identifier bits refer to a set of binary bits in the data packet used to mark the starting position of the content to be detected. The network protocol type refers to the network communication rules followed by the data packets, such as TCP / IP, HTTP, FTP, etc.

[0047] For example, during traffic monitoring of a server, the port is 80 and the network protocol type is HTTP. 100 sample data packets are captured from port 80, each packet being 1KB in size. According to the HTTP protocol specification, the first 20 bytes of each packet are the TCP header, and the following bytes are the HTTP header and body. Therefore, the data identifier bit is a set of binary bits starting from the 21st byte, used to represent the start position of the HTTP header and body. Based on the content of the sampled data packets, the specific value of the data identifier bit is determined, such as 01010101. The network protocol type (HTTP) and the data identifier bit (01010101) of the sampled data packets are stored in the first correspondence.

[0048] Optionally, in step 201, before determining the data identifier bit and network protocol type of the sampled data packet based on the sampled data packet, the following sub-steps are included:

[0049] Sub-step 2011: Extract key data from the sampled data packet to obtain a simplified sampled data packet; the key data includes one or more of the following: the type of command stored in the data packet, the parameters of the command, the status code of the response, authentication information, and duplicate information.

[0050] In one embodiment of this application, key data can be extracted from the sampled data packet before data processing to obtain a simplified sampled data packet. The key data refers to one or more of the following stored in the data packet: command type, command parameters, response status code, authentication information, and duplicate information. The simplified sampled data packet contains only the key data, used to reduce data volume and improve analysis efficiency.

[0051] For example, during traffic monitoring of a server, the port is 22, and the network protocol is Secure Shell (SSH). 100 sample packets are captured from port 22, each packet being 1KB in size. According to the SSH protocol specification, the first 5 bytes of each packet contain the packet length, padding length, and packet type; the following bytes contain the padding data and payload. Therefore, the key data is the packet type and payload, which include information such as the command type, command parameters, response status code, authentication information, and duplicate information. The key data is extracted from the sample packets to obtain a simplified sample packet. The simplified sample packet is no larger than 100 bytes and contains only the binary string of the packet type and payload.

[0052] Optionally, step 201, determining the sampled data packet, specifically includes the following sub-steps:

[0053] Sub-step 2012: Obtain the offset of the sampled data packet; the offset is used to characterize the number value of the sampled data packet during data transmission.

[0054] In one embodiment of this application, the offset of the sampled data packet is first obtained. The offset refers to the number value of each data packet during data transmission, used to indicate the order and position of the data packets. The offset can be an integer, a timestamp, a sequence number, or other form of identifier.

[0055] For example, in monitoring the network traffic of a mobile phone, the port is 443 and the network protocol is HTTP. 100 sample data packets are captured from port 443, each packet being 1KB in size. According to the HTTP protocol specification, the first 5 bytes of each packet contain the packet length, padding length, and packet type; the following bytes contain the padding data and payload. Therefore, the offset is a packet type value used to indicate the type and order of the packets. To obtain the offset of the sampled data packets, the 5th byte of each packet is read, resulting in an integer between 0 and 255, which is used as the offset value.

[0056] Sub-step 2013: Combine the offset with the data of the sampled data packet to obtain the offset data packet of the sampled data packet, and determine the feature code of the offset data packet based on the offset data packet.

[0057] In one embodiment of this application, the offset is combined with the data of the sampled data packet to obtain an offset data packet of the sampled data packet, and the feature encoding of the offset data packet is determined based on the offset data packet. The offset refers to the number value of each data packet during data transmission, used to indicate the order and position of the data packets. The data refers to key data stored in the data packet, such as the command type, command parameters, response status code, authentication information, and duplicate information. The offset data packet is a data packet composed of the offset and the data, used to increase the identifiability and distinguishability of the data packet. The feature encoding refers to a method for encoding the offset data packet, used to generate a unique identifier representing the type and content of the offset data packet.

[0058] For example, in monitoring the traffic of a printer, the port is 9100, and the network protocol is Internet Printing Protocol (IPP). 100 sample data packets are captured from port 9100, each packet being 1KB in size. According to the IPP protocol specification, the first 8 bytes of each packet contain the version number, opcode, request ID, and status code; the following bytes contain the attribute group, data, and end marker. Therefore, the data contains the key information stored in the packet, such as the opcode, request ID, status code, attribute group, data, and end marker. The offset of the sample data packet is obtained by reading the 9th byte of each packet, resulting in an integer between 0 and 255, which serves as the offset value. The offset is combined with the data to obtain the offset data packet. The offset data packet is no larger than 101 bytes and contains a binary string of the offset and data. Based on the offset data packet, the feature code of the offset data packet is determined. The feature code can be a hash function that encodes the binary string of the offset data packet, generating a fixed-length hexadecimal string as the feature code value.

[0059] Optionally, sub-step 2013 includes the following sub-steps:

[0060] Sub-step 20131: Perform an XOR operation between the offset and the data of the sampled data packet, and use the sampled data packet after the XOR operation as the offset data packet.

[0061] In one embodiment of this application, an XOR operation is performed between the offset and the data of the sampled data packet, and the sampled data packet after the XOR operation is used as the offset data packet. The offset refers to the number value of each data packet during data transmission, used to indicate the order and position of the data packets. The data refers to key data stored in the data packet, such as the command type, command parameters, response status code, authentication information, and duplicate information. The XOR operation is a logical operation used to compare two binary bits; if they are the same, it outputs 0; if they are different, it outputs 1. The offset data packet is a data packet composed of the result of the XOR operation between the offset and the data, used to increase the randomness and security of the data packet.

[0062] For example, in monitoring traffic from a camera, the port is 554, and the network protocol is Real-Time Streaming Protocol (RTSP). 100 sample data packets are captured from port 554, each packet being 1KB in size. According to the RTSP specification, the first four bytes of each packet contain the version number, packet type, and packet length, while the following bytes contain the packet body. Therefore, the data consists of key information such as the packet type, packet length, and packet body stored within the packet. The offset of the sampled data packet is obtained by reading the fifth byte of each packet, resulting in an integer between 0 and 255, which serves as the offset value. An XOR operation is performed between the offset and the data to obtain the offset data packet. The offset data packet is no larger than 101 bytes and contains a binary string representing the result of the XOR operation between the offset and the data.

[0063] Optionally, sub-step 2013 includes the following sub-steps:

[0064] Sub-step 20132: Use the summary data of the offset data packet as the feature encoding of the offset data packet.

[0065] In one embodiment of this application, the summary data of the offset data packet is used as the feature encoding of the offset data packet. The offset data packet refers to a data packet composed of an offset and data, used to increase the identifiability and distinguishability of the data packet. The summary data refers to the result of a summary algorithm applied to the offset data packet, used to generate a short identifier representing the content and characteristics of the offset data packet. The feature encoding refers to a method of encoding the offset data packet, used to generate a unique identifier representing the type and content of the offset data packet.

[0066] For example, in the embodiment of step 20131, the digest data of the offset data packet is used as the feature encoding of the offset data packet. The digest data can be a digest algorithm that digests the binary string of the offset data packet to generate a fixed-length hexadecimal string as the value of the digest data.

[0067] Sub-step 2014: Based on the feature code, search in the preset third correspondence to determine the data identifier bit of the sampled data packet; the third correspondence stores the correspondence between the feature code and the data identifier bit.

[0068] In one embodiment of this application, the data identifier bit of the sampled data packet is determined by searching in a preset third correspondence based on the feature encoding. The feature encoding refers to a method for encoding the offset data packet, used to generate a unique identifier representing the type and content of the offset data packet. The third correspondence is a data structure used to store the correspondence between different feature codes and data identifier bits. The data identifier bit refers to a set of binary bits in the data packet used to mark the starting position of the content to be detected.

[0069] For example, following the embodiment in step 2013, the data identifier bit of the sampled data packet can be determined by searching in a preset third correspondence based on the feature code. Assume that the correspondence between the feature code and the data identifier bit in the third correspondence is: 5a3f2b (feature code) - 01001100 (data identifier bit), 7c8d9e (feature code) - 00100110 (data identifier bit). Then, based on the value of the feature code, the corresponding value of the data identifier bit can be found in the third correspondence and used as the data identifier bit of the sampled data packet.

[0070] Step 202: Store the network protocol type of the sampled data packet and the data identifier bit of the sampled data packet into the first correspondence.

[0071] In one embodiment of this application, the network protocol type and data identifier bits of the sampled data packet are stored in the first correspondence. The first correspondence is a data structure used to store the correspondence between different network protocol types and data identifier bits. The first correspondence can be an array, a hash table, a dictionary, or other form of data structure.

[0072] For example, after the embodiment of step 201, the network protocol type (HTTP) and data identifier (01010101) of the sampled data packet can be stored in the first correspondence.

[0073] Step 203: Generate an analysis process template for analyzing the sampled data packet based on the network protocol type and data identifier bit of the sampled data packet; the analysis process template is used to record the specific steps for analyzing the content to be detected in the sampled data packet, starting from the data identifier bit, under the corresponding network protocol type.

[0074] In one embodiment of this application, an analysis process template for analyzing the sampled data packet is generated based on the network protocol type and data identifier bits of the sampled data packet. The analysis process template is an algorithm or program used to record the specific steps for analyzing the content to be detected in the sampled data packet, starting from the data identifier bits, under the corresponding network protocol type. The analysis process template can be a set of executable programs.

[0075] For example, following the embodiment of step 202, a program can be obtained that executes according to the following process: starting from the data identifier, reading the HTTP header and body; parsing the HTTP header to extract key information, such as the request method, request Uniform Resource Locator (URL), response status code, etc.; parsing the HTTP body to extract key information, such as content type, content length, content encoding, etc.

[0076] Step 204: Store the data identifier bit of the sampled data packet and the analysis process template of the sampled data packet into the second correspondence.

[0077] In one embodiment of this application, the data identifier bits of the sampled data packet and the analysis process template of the sampled data packet are stored in the second correspondence. The second correspondence is a data structure used to store the correspondence between different data identifier bits and analysis process templates. The second correspondence can be an array, a hash table, a dictionary, or other form of data structure.

[0078] For example, after the embodiment of step 203, the data identifier bit of the sampled data packet and the analysis process template of the sampled data packet can be stored in the second correspondence.

[0079] Step 205: Capture the data packets to be detected from the device's port and determine the network protocol type of the data packets to be detected; the data packets to be detected contain the content to be detected, as well as other content besides the content to be detected.

[0080] The method shown in this step has been explained in step 101 and will not be repeated here.

[0081] Step 206: Determine the data identifier bit of the detected data packet from the preset first correspondence relationship according to the network protocol type of the detected data packet; the first correspondence relationship is used to store the correspondence between network protocol type and data identifier bit; the data identifier bit is used to characterize the starting position of the content to be detected in the detected data packet.

[0082] The method shown in this step has been explained in step 102 and will not be repeated here.

[0083] Step 207: Determine the target analysis process template corresponding to the data identifier bit of the detected data packet from the preset second correspondence relationship; the second correspondence relationship stores the correspondence between the data identifier bit and the analysis process template; the analysis process template is used to record the specific steps of analyzing the content to be detected of the detected data packet, starting from the data identifier bit, under the corresponding network protocol type.

[0084] The method shown in this step has been explained in step 103 and will not be repeated here.

[0085] Step 208: Following the steps in the target analysis process template, starting from the data identifier bit of the data packet being detected, analyze the content to be detected in the data packet being detected.

[0086] The method shown in this step has been explained in step 104 and will not be repeated here.

[0087] Optionally, if the identifier does not obtain a target analysis process template corresponding to the data characteristics of the detected data packet, this method further includes step 209:

[0088] Step 209: The detected data packet is used as a sampled data packet, and the process proceeds to the step of determining the data identifier bit and network protocol type of the sampled data packet based on the sampled data packet.

[0089] In one embodiment of this application, if the identifier does not obtain a target analysis process template corresponding to the data characteristics of the detected data packet, the detected data packet is used as a sampling data packet to expand the data in the first correspondence and the second correspondence.

[0090] For example, the first correspondence stores the analysis process templates corresponding to FTP data and HTTP data. During the analysis of the detected data packet, a URL data is obtained. At this time, the corresponding analysis process template cannot be obtained. Therefore, it is necessary to analyze this URL data and store it in the first correspondence and the second correspondence.

[0091] Figure 3 This is an architecture diagram of a network data analysis method provided in an embodiment of this application. Figure 4 This is a program logic diagram of a network data analysis method provided in an embodiment of this application. Figure 3 The monitoring module monitors network traffic by associating with both the policy center and the workgroup. The parser processes the captured data and sends the results back to the monitoring module and the workgroup, while the processor provides hardware and software support to the policy center for policy selection during analysis. Figure 3 Under this architecture, it can be done according to Figure 4 The specific steps for analyzing data packets are as follows:

[0092] In one embodiment of this application, steps S1 to S4 execute the specific processes of steps 101 to 104 in this application. It should be emphasized that the type of data packet is not distinguished here, whether it is a sampled data packet or a data packet to be analyzed.

[0093] In step S1, the protocol type of the data packet is specifically determined. If it matches the recorded protocol type, the data packet is processed. Then, in steps S2, S3, and S4, the offset and status parameters of the data packet are parsed to obtain the identifier bit of the data packet. After obtaining the identifier bit, the data packet can be sent to the subsequent steps to obtain the final analysis result and realize feedback.

[0094] In summary, in this embodiment, by determining the data identifier of the detected data packet from a preset first correspondence based on the network protocol type of the detected data packet, and by determining the target analysis process template corresponding to the data identifier of the detected data packet from a preset second correspondence, the starting position of the content to be detected in the detected data packet can be quickly located. Furthermore, different analysis process templates are used according to different network protocol types, achieving customized analysis of the content to be detected in the detected data packet. Based on the target analysis process template, the content to be detected in the detected data packet is analyzed starting from the data identifier, and the required network traffic data can be extracted without affecting the normal operation of the network. Therefore, the method based on this embodiment does not require complete parsing of each data packet when analyzing network data, solving the problem in related technologies where the large scale and complexity of network traffic data leads to a large data parsing load, long processing time, and low data parsing efficiency, making it impossible to quickly extract network traffic data.

[0095] refer to Figure 5 This application illustrates a network data analysis device 30 provided in an embodiment of the present application. The device 30 includes:

[0096] The capture module 301 is used to capture the data packets to be detected from the port of the device and determine the network protocol type of the data packets to be detected; the data packets to be detected contain the content to be detected, as well as other content besides the content to be detected.

[0097] The first relationship module 302 is used to determine the data identifier bit of the detected data packet from a preset first correspondence relationship according to the network protocol type of the detected data packet; the first correspondence relationship is used to store the correspondence between network protocol types and data identifier bits; the data identifier bit is used to characterize the starting position of the content to be detected in the detected data packet;

[0098] The second relationship module 303 is used to determine the target analysis process template corresponding to the data identifier bit of the detected data packet from a preset second correspondence relationship; the second correspondence relationship stores the correspondence between the data identifier bit and the analysis process template; the analysis process template is used to record the specific steps of analyzing the content to be detected of the detected data packet, starting from the data identifier bit, under the corresponding network protocol type.

[0099] The data analysis module 304 is used to analyze the content to be detected in the data packet, starting from the data identifier bit of the data packet to be detected, according to the steps in the target analysis process template.

[0100] Optionally, the network data analysis device 30 also includes:

[0101] The sampling module is used to capture sampling data packets from the device's port and determine the data identifier and network protocol type of the sampling data packets based on the sampling data packets; the sampling data packets contain the content to be detected, as well as other content besides the content to be detected;

[0102] The first relationship storage module is used to store the network protocol type of the sampled data packet and the data identifier bit of the sampled data packet into the first correspondence relationship;

[0103] The analysis module is used to generate an analysis process template for analyzing the sampled data packets based on the network protocol type and data identifier bits of the sampled data packets. The analysis process template is used to record the specific steps for analyzing the content to be detected in the sampled data packets, starting from the data identifier bits, under the corresponding network protocol type.

[0104] The second relation storage module is used to store the data identifier bits of the sampled data packet and the analysis process template of the sampled data packet into the second correspondence relation.

[0105] Optionally, the sampling module includes:

[0106] An offset quantum module is used to obtain the offset of the sampled data packet; the offset is used to characterize the number value of the sampled data packet during data transmission.

[0107] The feature encoding submodule is used to combine the offset with the data of the sampled data packet to obtain the offset data packet of the sampled data packet, and to determine the feature encoding of the offset data packet based on the offset data packet;

[0108] The identifier confirmation submodule is used to search in a preset third correspondence relationship according to the feature code to determine the data identifier bit of the sampled data packet; the third correspondence relationship stores the correspondence between the feature code and the data identifier bit.

[0109] Optionally, the feature encoding submodule includes:

[0110] The XOR operation unit is used to perform an XOR operation on the offset and the data of the sampled data packet, and use the sampled data packet after the XOR operation as the offset data packet.

[0111] Optionally, the feature encoding submodule includes:

[0112] A summary data unit is used to encode the feature data of the offset data packet as a summary data of the offset data packet.

[0113] Optionally, the network data analysis device 30 includes:

[0114] The key data module is used to extract key data from the sampled data packet to obtain a simplified sampled data packet; the key data includes one or more of the following: the type of command stored in the data packet, the parameters of the command, the status code of the response, authentication information, and duplicate information.

[0115] Optionally, the network data analysis device 30 includes:

[0116] The recording and analysis module is used to treat the detected data packet as a sampled data packet when the identifier bit does not obtain a target analysis process template corresponding to the data characteristics of the detected data packet, and proceed to the step of determining the data identifier bit and network protocol type of the sampled data packet based on the sampled data packet.

[0117] In summary, in this embodiment, by determining the data identifier of the detected data packet from a preset first correspondence based on the network protocol type of the detected data packet, and by determining the target analysis process template corresponding to the data identifier of the detected data packet from a preset second correspondence, the starting position of the content to be detected in the detected data packet can be quickly located. Furthermore, different analysis process templates are used according to different network protocol types, achieving customized analysis of the content to be detected in the detected data packet. Based on the target analysis process template, starting from the data identifier of the detected data packet, the content to be detected in the detected data packet is analyzed, and the required network traffic data can be extracted without affecting the normal operation of the network. Therefore, the method based on this embodiment does not require complete parsing of each data packet when analyzing network data, solving the problem in related technologies where the large scale and complexity of network traffic data leads to a large data parsing load, long processing time, and low data parsing efficiency, making it impossible to quickly extract network traffic data. 。

[0118] Reference Figure 6 The electronic device 500 may include one or more of the following components: processing component 502, memory 504, power supply component 506, multimedia component 508, audio component 510, input / output (I / O) interface 512, sensor component 514, and communication component 516.

[0119] Processing component 502 typically controls the overall operation of electronic device 500, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 502 may include one or more processors 520 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 502 may include one or more modules to facilitate interaction between processing component 502 and other components. For example, processing component 502 may include a multimedia module to facilitate interaction between multimedia component 508 and processing component 502.

[0120] Memory 504 is used to store various types of data to support the operation of electronic device 500. Examples of this data include instructions for any application or method operating on electronic device 500, contact data, phonebook data, messages, pictures, multimedia, etc. Memory 504 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0121] Power supply component 506 provides power to various components of electronic device 500. Power supply component 506 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 500.

[0122] Multimedia component 508 includes an interface that provides an output interface between electronic device 500 and user. In some embodiments, the interface may include a liquid crystal display (LCD) and a touch panel (TP). If the interface includes a touch panel, the interface may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may not only sense the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 508 includes a front-facing camera and / or a rear-facing camera. When electronic device 500 is in an operating mode, such as shooting mode or multimedia mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0123] Audio component 510 is used to output and / or input audio signals. For example, audio component 510 includes a microphone (MIC) used to receive external audio signals when electronic device 500 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 504 or transmitted via communication component 516. In some embodiments, audio component 510 also includes a speaker for outputting audio signals.

[0124] Input / output (I / O) interface 512 provides an interface between processing component 502 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0125] Sensor assembly 514 includes one or more sensors for providing state assessments of various aspects of electronic device 500. For example, sensor assembly 515 may detect the on / off state of electronic device 500, the relative positioning of components such as the display and keypad of electronic device 500, changes in position of electronic device 500 or a component of electronic device 500, the presence or absence of user contact with electronic device 500, orientation or acceleration / deceleration of electronic device 500, and temperature changes of electronic device 500. Sensor assembly 514 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 515 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 514 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0126] Communication component 516 facilitates wired or wireless communication between electronic device 500 and other devices. Electronic device 500 can access wireless networks based on communication standards, such as WiFi, carrier networks (such as 2G, 3G, 4G, or 5G), or combinations thereof. In one exemplary embodiment, communication component 516 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 516 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0127] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to implement the methods provided in the embodiments of this application.

[0128] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 504 including instructions, which can be executed by a processor 520 of an electronic device 500 to perform the above-described method. For example, the non-transitory storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0129] Figure 7This is a block diagram of an electronic device 600 according to another embodiment of the present invention. For example, the electronic device 600 may be provided as a server. (See also...) Figure 7 The electronic device 600 includes a processing component 622, which further includes one or more processors, and memory resources represented by memory 632 for storing instructions, such as application programs, that can be executed by the processing component 622. The application programs stored in memory 632 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 622 is configured to execute instructions to perform the methods provided in the embodiments of this application.

[0130] Electronic device 600 may also include a power supply component 626 configured to perform power management of electronic device 600, a wired or wireless network interface 650 configured to connect electronic device 600 to a network, and an input / output (I / O) interface 658. Electronic device 600 may operate on an operating system stored in memory 632, such as Windows Server™, MacOSX™, Unix™, Linux™, FreeBSD™, or similar.

[0131] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0132] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for analyzing network data, characterized in that, include: Capture the data packets to be detected from the device's port and determine the network protocol type of the data packets to be detected; the data packets to be detected contain the content to be detected, as well as other content besides the content to be detected; From the preset first correspondence, the data identifier bit of the detected data packet is determined according to the network protocol type of the detected data packet; the first correspondence is used to store the correspondence between the network protocol type and the data identifier bit; the data identifier bit is used to characterize the starting position of the content to be detected in the detected data packet; From the preset second correspondence, determine the target analysis process template corresponding to the data identifier bit of the detected data packet; The second correspondence stores the correspondence between the data identifier and the analysis process template; the analysis process template is used to record the specific steps for analyzing the content to be detected in the data packet under the corresponding network protocol type, starting from the data identifier. Following the steps in the target analysis process template, starting from the data identifier bit of the detected data packet, the content to be detected in the detected data packet is analyzed; The method further includes: Capture sampling data packets from the device's port, and determine the data identifier and network protocol type of the sampling data packets based on the sampling data packets; the sampling data packets contain the content to be detected, as well as other content besides the content to be detected; The network protocol type and data identifier bits of the sampled data packet are stored in the first correspondence. Based on the network protocol type and data identifier bits of the sampled data packets, an analysis process template for analyzing the sampled data packets is generated; the analysis process template is used to record the specific steps for analyzing the content to be detected in the sampled data packets, starting from the data identifier bits, under the corresponding network protocol type. The data identifier bits of the sampled data packet and the analysis process template of the sampled data packet are stored in the second correspondence.

2. The method as described in claim 1, characterized in that, The step of determining the data identifier bit of the sampled data packet based on the sampled data packet includes: Obtain the offset of the sampled data packet; the offset is used to characterize the number value of the sampled data packet during data transmission. The offset is combined with the data of the sampled data packet to obtain the offset data packet of the sampled data packet, and the feature code of the offset data packet is determined based on the offset data packet; Based on the feature code, the data identifier bit of the sampled data packet is determined by searching in a preset third correspondence relationship; the third correspondence relationship stores the correspondence between the feature code and the data identifier bit.

3. The method as described in claim 2, characterized in that, The step of combining the offset with the data of the sampled data packet to obtain the offset data packet of the sampled data packet includes: Perform an XOR operation between the offset and the data of the sampled data packet, and use the sampled data packet after the XOR operation as the offset data packet.

4. The method as described in claim 2, characterized in that, Determining the feature encoding of the offset data packet based on the offset data packet includes: The summary data of the offset data packet is used as the feature encoding of the offset data packet.

5. The method as described in claim 1, characterized in that, Before determining the data identifier bits and network protocol type of the sampled data packet based on the sampled data packet, the method further includes: Extract key data from the sampled data packet to obtain a simplified sampled data packet; the key data includes one or more of the following: the type of command stored in the data packet, the parameters of the command, the status code of the response, authentication information, and duplicate information.

6. The method as described in claim 1, characterized in that, The method further includes: If no target analysis process template corresponding to the data characteristics of the detected data packet is obtained for the identifier bit, the detected data packet is treated as a sampled data packet, and the process proceeds to the step of determining the data identifier bit and network protocol type of the sampled data packet based on the sampled data packet.

7. A network data analysis device, characterized in that, The device includes: The capture module is used to capture the data packets to be detected from the device's port and determine the network protocol type of the data packets to be detected; the data packets to be detected contain the content to be detected, as well as other content besides the content to be detected. The first relationship module is used to determine the data identifier bit of the detected data packet from a preset first correspondence relationship according to the network protocol type of the detected data packet; the first correspondence relationship is used to store the correspondence between network protocol types and data identifier bits; the data identifier bit is used to characterize the starting position of the content to be detected in the detected data packet; The second relationship module is used to determine the target analysis process template corresponding to the data identifier bit of the detected data packet from a preset second correspondence relationship; the second correspondence relationship stores the correspondence between the data identifier bit and the analysis process template; the analysis process template is used to record the specific steps of analyzing the content to be detected of the detected data packet, starting from the data identifier bit, under the corresponding network protocol type. The data analysis module is used to analyze the content to be detected in the data packet, starting from the data identifier bit of the data packet to be detected, according to the steps in the target analysis process template. The device further includes: The sampling module is used to capture sampling data packets from the device's port and determine the data identifier and network protocol type of the sampling data packets based on the sampling data packets; the sampling data packets contain the content to be detected, as well as other content besides the content to be detected; The first relationship storage module is used to store the network protocol type of the sampled data packet and the data identifier bit of the sampled data packet into the first correspondence relationship; The analysis module is used to generate an analysis process template for analyzing the sampled data packets based on the network protocol type and data identifier bits of the sampled data packets. The analysis process template is used to record the specific steps for analyzing the content to be detected in the sampled data packets, starting from the data identifier bits, under the corresponding network protocol type. The second relation storage module is used to store the data identifier bits of the sampled data packet and the analysis process template of the sampled data packet into the second correspondence relation.

8. An electronic device, characterized in that, include: Processor; memory for storing processor-executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the method as described in any one of claims 1 to 6.

Images