Method and system for restoring Protobuf format data in network traffic

Abstract

A method and system for restoring Protobuf format data in network traffic, the method comprising: monitoring network traffic data, obtaining quintuple information in the network traffic data; determining the data format of the network traffic data according to the quintuple information; after determining that the network traffic data is Protobuf format data, extracting HTTP payload and URI field, and parsing the HTTP payload by a parsing tool to obtain a parsing result; establishing a mapping relationship between the URI field and Protobuf format data according to the parsing result, and merging the Protobuf format data of the same URI field to establish a database; establishing a regular expression for a definition file and Protobuf format data according to the mapping relationship, and matching a corresponding target parsing file in a pre-established comparison database according to the parsing result; the present invention can match a corresponding target definition file when there is a lack of definition files in the network traffic data, thereby improving data parsing efficiency.

Description

Technical Field

[0001] The present invention relates to the field of physics, and in particular to a network traffic data processing technology, in particular to a method and system for restoring Protobuf format data in network traffic. Background Art

[0002] The Protobuf Buffers protocol was originally developed by Google to solve data serialization and communication problems in its internal systems. Due to its high efficiency and cross-language support, Protobuf is widely used in various applications. Specifically, Protobuf is often used in network communications as a data exchange format, for configuration files, and for transferring structured data between different systems. The key difference between Protobuf and common data serialization formats such as Json is that Json is a text format with its own field number key, while Protobuf is a binary format without its own field number key and requires a .proto definition file for complete parsing. Inferring the content of the key in the absence of a .proto definition file in network traffic is a difficult problem that has not yet been solved in the prior art. Summary of the invention

[0003] The object of the present invention is to provide a method and system for restoring Protobuf format data in network traffic, so as to solve the problem in the prior art that network traffic data cannot be parsed due to the lack of .proto definition files.

[0004] In a first aspect, the present invention provides a method for restoring Protobuf format data in network traffic, comprising: monitoring the network traffic data to obtain quintuple information in the network traffic data; determining the data format of the network traffic data according to the quintuple information; after determining that the network traffic data is Protobuf format data, extracting HTTP payload and URI field, and parsing the HTTP payload through a parsing tool to obtain a parsing result; establishing a mapping relationship between the URI field and the Protobuf format data according to the parsing result, and merging the Protobuf format data of the same URI field to establish a database; establishing a regular expression for a definition file and the Protobuf format data according to the mapping relationship, and matching a corresponding target parsing file in a pre-established comparison database according to the parsing result; wherein the comparison database includes multiple groups of known Protobuf data of known reference definition files, each group of the known Protobuf data has the same hierarchical structure and is classified according to the size of the URI field, and each group of the Protobuf format data of the URI field in the comparison database corresponds to one reference definition file.

[0005] In an implementation of the first aspect, the HTTP payload is parsed by a parsing tool to obtain a parsing result, including: parsing each area field of the HTTP payload according to a standard format to obtain the encoding type wire_type and data domain length Length of the field in each area; determining that the data of the HTTP payload is Protobuf format data when the encoding type wire_type and data domain length Length of each area field meet the requirements; saving the encoding type wire_type, the data domain length Length, the field number field_num and the data value value of each area field obtained after parsing in Json format to obtain a key-value pair, so as to obtain the hierarchical structure of the HTTP payload.

[0006] In an implementation of the first aspect, parsing the HTTP payload by a parsing tool to obtain a parsing result also includes: after determining that the data of the HTTP payload is Protobuf format data, when the Protobuf format data comes with a field number key, taking the field number key as input, combining the encoding format of the data of the HTTP payload, and outputting a corresponding reference definition file.

[0007] In an implementation of the first aspect, matching the corresponding target parsing file in a pre-established comparison database based on the parsing result includes: establishing a regular expression based on the hierarchical structure corresponding to the different Protobuf format data in the database and a reference definition file; obtaining the target field number corresponding to the parsing result based on the regular expression; and determining the target definition file corresponding to the network traffic data based on the target field number and the hierarchical structure of the HTTP payload.

[0008] In an implementation of the first aspect, obtaining the target field number corresponding to the parsing result according to the regular expression includes: according to the hierarchical structure in the parsing result, respectively comparing the data value value at a specific position, and inferring the target field number according to the regular expression and the data value value.

[0009] In an implementation of the first aspect, parsing the HTTP payload by a parsing tool to obtain a parsing result also includes: reporting a parsing error when the encoding type wire_type or the data field length Length of each area field does not meet the requirements, and determining that the start and end positions of the HTTP payload are incorrect or the data is encrypted; when the start and end positions of the HTTP payload are incorrect, adjusting the configuration file to modify the start and end positions of the parsing and then continuing the parsing; when the HTTP payload has data encryption, discarding the current HTTP payload.

[0010] In an implementation of the first aspect, monitoring the network traffic data and obtaining the five-tuple information in the network traffic data include: monitoring the network traffic data in a communication system, and separating the data link layer of the monitored network traffic data to obtain IP network layer data; extracting the source IP address, destination IP address and transmission protocol based on the IP network layer data; separating the IP network layer to obtain transport layer data, so as to extract the source port and destination port corresponding to the transport protocol.

[0011] In an implementation of the first aspect, determining the data format of the network traffic data based on the quintuple information includes: separating the transport layer data to obtain application layer data, and parsing the header bytes of the application layer data; matching and judging the header bytes based on the port and keywords to determine whether the network traffic data is HTTP traffic; after determining that the network traffic data is HTTP traffic, extracting the HTTP header; parsing the HTTP header, and determining the format type of the network traffic data based on the fields in the HTTP header.

[0012] In an implementation of the first aspect, after determining that the network traffic data is Protobuf format data, extracting the HTTP payload and URI field, including: extracting the downlink payload when the network traffic data is a GET request, extracting the downlink payload when the network traffic data is a POST request, and parsing the URI field of the HPPT header at the same time.

[0013] In a second aspect, the present invention also provides a system for restoring Protobuf format data in network traffic, including: a monitoring module, used to monitor the network traffic data and obtain five-tuple information in the network traffic data; a format determination module, used to determine the data format of the network traffic data according to the five-tuple information; a parsing module, used to extract HTTP payload and URI field after determining that the network traffic data is Protobuf format data, and parse the HTTP payload through a parsing tool to obtain a parsing result; a mapping module, used to establish a mapping relationship between the URI field and the Protobuf format data according to the parsing result. system, and merge the Protobuf format data of the same URI field to establish a database; a matching module, used to establish a regular expression for the definition file and the Protobuf format data according to the mapping relationship, and match the corresponding target parsing file in a pre-established comparison database according to the parsing result; wherein the comparison database includes multiple groups of known Protobuf data of known reference definition files, each group of the known Protobuf data has the same hierarchical structure and is classified according to the size of the URI field, and each group of the Protobuf format data of the URI field in the comparison database corresponds to one reference definition file.

[0014] As described above, the method and system for restoring Protobuf format data in network traffic according to the present invention have the following beneficial effects:

[0015] Compared with the prior art, the present invention parses the Protobuf format data with field number key in the network traffic data to obtain the corresponding URI field and hierarchical structure, classifies the Protobuf format data of different hierarchical structures and URI fields and establishes a comparison database, and establishes regular expressions of different Protobuf format data and definition files according to the comparison database. When there is a lack of definition files in the network traffic data later, the target definition file corresponding to the network traffic data is matched in the comparison database according to the regular expression, so as to quickly complete the parsing process of the network traffic data, thereby improving the data parsing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Shown is a flow chart of a method for restoring Protobuf format data in network traffic according to an embodiment of the present invention.

[0017] Figure 2 Shown is a structural block diagram of a system for restoring Protobuf format data in network traffic according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] The following describes the embodiments of the present invention through specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0019] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The illustrations only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0020] See also Figure 1 and Figure 2 . The method and system for restoring Protobuf format data in network traffic described in the present invention parses the Protobuf format data with field number key in the network traffic data to obtain the corresponding URI field and hierarchical structure, and classifies the Protobuf format data of different hierarchical structures and URI fields and establishes a comparison database, and establishes regular expressions of different Protobuf format data and definition files based on the comparison database. When there is a lack of definition files in the network traffic data, the target definition file corresponding to the network traffic data is matched in the comparison database according to the regular expression, so as to quickly complete the parsing process of the network traffic data, thereby improving the data parsing efficiency.

[0021] like Figure 1 As shown, in one embodiment, a method for restoring Protobuf format data in network traffic of the present invention comprises the following steps:

[0022] S101. Monitor the network traffic data and obtain quintuple information in the network traffic data.

[0023] The five-tuple information includes source IP address, destination IP address, source port, destination port and transmission protocol. The five-tuple information is obtained by monitoring network traffic data to facilitate judging the format of network traffic data.

[0024] In one embodiment, monitoring the network traffic data and obtaining quintuple information in the network traffic data includes:

[0025] Monitoring the network traffic data in the communication system, and separating the data link layer of the monitored network traffic data to obtain IP network layer data;

[0026] Extracting the source IP address, the destination IP address and the transmission protocol according to the IP network layer data;

[0027] The IP network layer is separated to obtain transport layer data, so as to extract the source port and the destination port corresponding to the transport protocol.

[0028] Specifically, after the network traffic data arrives at the network port, the network layer protocol number 6 and the transport layer port numbers 80 and 8080 are monitored to collect network traffic data, and the data is parsed to separate the IP network layer data and the transport layer data, thereby extracting the source IP address, destination IP address, source port, destination port and transport protocol to obtain the corresponding five-tuple information.

[0029] S102: Determine a data format of the network traffic data according to the quintuple information.

[0030] In one embodiment, determining the data format of the network traffic data according to the quintuple information includes:

[0031] Separating the transport layer data to obtain application layer data, and parsing header bytes of the application layer data;

[0032] Matching and judging the header bytes according to the port and the keyword to determine whether the network traffic data is HTTP traffic;

[0033] After determining that the network traffic data is HTTP traffic, extracting the HTTP header;

[0034] The HTTP header is parsed, and the format type of the network traffic data is determined according to the fields in the HTTP header.

[0035] Specifically, first separate the transport layer data to obtain the application layer data, parse the header bytes of the application layer data, determine that the network traffic data is HTTP traffic when the header bytes contain "GET" or "POST", and extract the HTTP header, then parse the HTTP header, and determine whether it is Protobuf format data based on whether the content type (content-type) field is "application / x-protobuf" or "application / vnd.google.protobuf". If its content type (content-type) field is "application / x-protobuf" or "application / vnd.google.protobuf", it is determined to be protobuf format data, and the HTTP payload is extracted.

[0036] S103: After determining that the network traffic data is in Protobuf format, extract the HTTP payload and URI field, and parse the HTTP payload using a parsing tool to obtain a parsing result.

[0037] In one embodiment, after determining that the network traffic data is Protobuf format data, extracting the HTTP payload and URI field includes:

[0038] When the network traffic data is a GET request, the downlink load is extracted; when the network traffic data is a POST request, the downlink load is extracted, and the URI field of the HPPT header is parsed at the same time.

[0039] In one embodiment, parsing the HTTP payload by a parsing tool to obtain a parsing result includes:

[0040] Parse each area field of the HTTP payload according to the standard format to obtain the encoding type wire_type and data field length Length of each area field;

[0041] When the encoding type wire_type and the data field length Length of each area field meet the requirements, determine that the data of the HTTP payload is Protobuf format data;

[0042] The encoding type wire_type, the data field length Length, the field number field_num and the data value value of each area field obtained after parsing are saved in Json format to obtain a key-value pair to obtain the hierarchical structure of the HTTP payload.

[0043] In one embodiment, after determining that the data of the HTTP payload is Protobuf format data, when the Protobuf format data comes with a field number key, the field number key is used as input, combined with the encoding format of the HTTP payload data, and the corresponding reference definition file is output.

[0044] Since Protobuf format data generally uses Varints and Zigzag to encode data, the overall structure is divided into several field areas. Each area consists of Tag-[Length]-Value, where Length is optional. Whether Length exists is determined by the value of the Tag.

[0045] Specifically, the Tag occupies one byte, including the 5-bit field number field_num and the 3-bit encoding type wire_type. The field number is used for table lookup. The length Length only exists when the wire_type is 2, and the length is not fixed. At this time, the length of the Value field is determined by the Length field. The encoding type wire_type is only meaningful when the value is 0, 1, 2, or 5. 3 and 4 are already discarded. When parsing specifically, first parse the data according to the usual format. If the encoding type wire_type is out of range, the parsing will report an error. If the wire_type is 2, parse the length Length. If the length Length exceeds the remaining data length, the parsing will report an error. After parsing a field area, try to parse the next field until the data is parsed.

[0046] If the parsing does not report an error, it means that the data in the payload is indeed in Protobuf format. After successful parsing, save the data in Json format and get the key-value pair of field_num and value. For some Protobuf data with built-in keys, directly use these keys as input, combine the encoding format of the data, and output the .proto reference definition file.

[0047] On the other hand, parsing the HTTP payload by a parsing tool to obtain a parsing result further includes:

[0048] When the wire_type encoding type or the data field length Length of each field in the area does not meet the requirements, an error is reported during analysis to determine that the start and end positions of the HTTP payload are incorrect or the data is encrypted;

[0049] When the start and end positions of the HTTP payload are incorrect, the configuration file is adjusted to modify the start and end positions of the parsing and then continue parsing;

[0050] When data encryption exists in the HTTP payload, the current HTTP payload is discarded.

[0051] If the parsing fails, you need to confirm whether the start and end positions of the parsing are accurate, or whether the data is encrypted. The characteristics of Protobuf data are Tag-Length-Value structure and tags in the form of 0x*2, 0x*8, and 0x*A. The first tag is generally 0x08 or 0x0A. If the data is observed to be chaotic and there is no similar tag, it is judged to be encrypted data. At this time, such data is discarded and no parsing is performed. If the protobuf data position is incorrect, adjust the configuration file, modify the start and end positions of the parsing, and then parse it. After the parsing is successful, follow the above steps to parse it.

[0052] S104, establishing a mapping relationship between the URI field and the Protobuf format data according to the parsing result, and merging the Protobuf format data of the same URI field to establish a database.

[0053] Among them, the comparison database includes multiple groups of known Protobuf data of known reference definition files, each group of the known Protobuf data has the same hierarchical structure and is classified according to the size of the URI field, and each group of the Protobuf format data of the URI field in the comparison database corresponds to one reference definition file.

[0054] If two HTTP messages have the same URI field and the HTTP payload has exactly the same Protobuf hierarchical structure, then the two HTTP flows can be considered to have the same business type. For HTTP requests with the same URI, their Protobuf format data is merged and a database is established to reduce the complexity of data storage and processing. The aggregation method is to group by URI and then merge the Protobuf data with the same structure into one table to reduce duplicate fields.

[0055] S105, establishing a regular expression for the definition file and the Protobuf format data according to the mapping relationship, and matching a corresponding target parsing file in a pre-established comparison database according to the parsing result.

[0056] In one embodiment, matching a corresponding target parsed file in a pre-established comparison database according to the parsing result includes:

[0057] Establishing a regular expression according to the hierarchical structure corresponding to the different Protobuf format data in the database and the reference definition file;

[0058] Obtaining a target field number corresponding to the parsing result according to the regular expression;

[0059] The target definition file corresponding to the network traffic data is determined according to the target field number and the hierarchical structure of the HTTP payload.

[0060] In one embodiment, obtaining the target field number corresponding to the parsing result according to the regular expression includes:

[0061] According to the hierarchical structure in the parsing result, the data value value at the specific position is compared respectively, and the target field number is inferred according to the regular expression and the data value value.

[0062] The protobuf format data in the comparison database obtained above and the corresponding reference definition file are used as known data sets to establish regular expression rules. For example, the matching pattern of Chinese mobile phone numbers is "86[0-9]{11}", which indicates that the number starts with 86 and the following 11 digits are in digital form. The matching pattern of Unix timestamps is "16[0-9]{8}".

[0063] Verify the matching effect through cross-validation or separate data sets, test the matching accuracy and other indicators, and if the performance does not meet the requirements, readjust the regular rules. For fields such as timestamps that change over time, the rules should also be adjusted in a timely manner. After a long period of network traffic control and analysis, summarize the knowledge base, form a method for inferring the field number key, and then further feedback the inference process based on the summarized inference method.

[0064] The present invention parses Protobuf format data with field number key in network traffic data to obtain corresponding URI fields and hierarchical structures, classifies Protobuf format data of different hierarchical structures and URI fields and establishes a comparison database, and establishes regular expressions of different Protobuf format data and definition files according to the comparison database. When there is a lack of definition files in the network traffic data, the target definition files corresponding to the network traffic data are matched in the comparison database according to the regular expressions, so as to quickly complete the parsing process of the network traffic data and improve the data parsing efficiency.

[0065] It should be noted that the protection scope of the method for restoring Protobuf format data in network traffic described in the present invention is not limited to the execution order of the steps listed in this embodiment, and all solutions implemented by adding, reducing, or replacing steps in the prior art based on the principles of the present invention are included in the protection scope of the present invention.

[0066] like Figure 2As shown, in one embodiment, the present invention further provides a system for restoring Protobuf format data in network traffic, comprising:

[0067] A monitoring module 201 is used to monitor the network traffic data and obtain quintuple information in the network traffic data;

[0068] A format determination module 202, configured to determine a data format of the network traffic data according to the five-tuple information;

[0069] The parsing module 203 is used to extract the HTTP payload and URI field after determining that the network traffic data is Protobuf format data, and parse the HTTP payload through a parsing tool to obtain a parsing result;

[0070] A mapping module 204 is used to establish a mapping relationship between the URI field and the Protobuf format data according to the parsing result, and merge the Protobuf format data of the same URI field to establish a database;

[0071] A matching module 205 is used to establish a regular expression for the definition file and the Protobuf format data according to the mapping relationship, and match the corresponding target parsing file in a pre-established comparison database according to the parsing result;

[0072] The comparison database includes multiple groups of known Protobuf data of known reference definition files, each group of known Protobuf data has the same hierarchical structure and is classified according to the size of the URI field, and each group of Protobuf format data of the URI field in the comparison database corresponds to one reference definition file:

[0073] It should be noted that the structures and principles of the monitoring module 201, the format determination module 202, the parsing module 203, the mapping module 204 and the matching module 205 correspond one to one with the steps (steps S101 to S105) in the above-mentioned method for restoring Protobuf format data in network traffic, so they are not repeated here.

[0074] In the several embodiments provided by the present invention, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules or units, which can be electrical, mechanical or other forms.

[0075] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention. For example, the functional modules / units in the various embodiments of the present invention may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0076] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0077] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.

[0078] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for restoring Protobuf format data in network traffic, characterized in that: include: Monitoring network traffic data, and obtaining quintuple information in the network traffic data; Determine the data format of the network traffic data according to the quintuple information; After determining that the network traffic data is Protobuf format data, extracting HTTP payload and URI field, and parsing the HTTP payload by a parsing tool to obtain a parsing result; The step of parsing the HTTP payload by using a parsing tool to obtain a parsing result includes: Parse each field of the HTTP payload in a standard format to obtain the encoding type wire_type and data field length Length of each field; When the wire_type encoding type and the data field length Length of each field in the area meet the requirements, determine that the data of the HTTP payload is Protobuf format data; The encoding type wire_type, the data field length Length, the field number field_num and the data value value of each area field obtained after parsing are saved in Json format to obtain a key-value pair, so as to obtain the hierarchical structure of the HTTP payload; The parsing of the HTTP payload by a parsing tool to obtain a parsing result further includes: after determining that the data of the HTTP payload is Protobuf format data, when the Protobuf format data has a field number key, taking the field number key as input, combining the encoding format of the data of the HTTP payload, and outputting a corresponding reference definition file; Establish a mapping relationship between the URI field and the Protobuf format data according to the parsing result, and merge the Protobuf format data of the same URI field to establish a database; Establishing a regular expression for the definition file and the Protobuf format data according to the mapping relationship, and matching a corresponding target parsing file in a pre-established comparison database according to the parsing result; matching a corresponding target parsing file in a pre-established comparison database according to the parsing result includes: Establishing a regular expression according to the hierarchical structure corresponding to the different Protobuf format data in the database and the reference definition file; Obtaining a target field number corresponding to the parsing result according to the regular expression; Determining a target definition file corresponding to the network traffic data according to the target field number and the hierarchical structure of the HTTP payload; Among them, the comparison database includes multiple groups of known Protobuf data of known reference definition files, each group of the known Protobuf data has the same hierarchical structure and is classified according to the size of the URI field, and each group of the Protobuf format data of the URI field in the comparison database corresponds to one reference definition file.

2. The method for restoring Protobuf format data in network traffic according to claim 1, characterized in that: The step of obtaining a target field number corresponding to the parsing result according to the regular expression includes: According to the hierarchical structure in the parsing result, the data values ​​value at corresponding positions are compared respectively, and the target field number is inferred according to the regular expression and the data value value.

3. The method for restoring Protobuf format data in network traffic according to claim 1, characterized in that: The step of parsing the HTTP payload by a parsing tool to obtain a parsing result further includes: When the wire_type encoding type or the data field length Length of each field in the area does not meet the requirements, an error is reported during analysis to determine that the start and end positions of the HTTP payload are incorrect or the data is encrypted; When the start and end positions of the HTTP payload are incorrect, the configuration file is adjusted to modify the start and end positions of the parsing and then continue parsing; When data encryption exists in the HTTP payload, the current HTTP payload is discarded.

4. The method for restoring Protobuf format data in network traffic according to claim 1, characterized in that: The monitoring of the network traffic data and obtaining the five-tuple information in the network traffic data includes: Monitoring the network traffic data in the communication system, and separating the data link layer of the monitored network traffic data to obtain IP network layer data; Extracting the source IP address, the destination IP address and the transmission protocol according to the IP network layer data; The IP network layer is separated to obtain transport layer data, so as to extract the source port and the destination port corresponding to the transport protocol.

5. The method for restoring Protobuf format data in network traffic according to claim 4, characterized in that: The determining the data format of the network traffic data according to the quintuple information includes: Separating the transport layer data to obtain application layer data, and parsing header bytes of the application layer data; Matching and judging the header bytes according to the port and the keyword to determine whether the network traffic data is HTTP traffic; After determining that the network traffic data is HTTP traffic, extracting the HTTP header; The HTTP header is parsed, and the format type of the network traffic data is determined according to the fields in the HTTP header.

6. A system for restoring Protobuf format data in network traffic, characterized in that: include: A monitoring module, used for monitoring network flow data and obtaining quintuple information in the network flow data; A format determination module, used to determine the data format of the network traffic data according to the quintuple information; A parsing module, for extracting HTTP payload and URI field after determining that the network traffic data is Protobuf format data, and parsing the HTTP payload through a parsing tool to obtain a parsing result; The step of parsing the HTTP payload by using a parsing tool to obtain a parsing result includes: Parse each field of the HTTP payload in a standard format to obtain the encoding type wire_type and data field length Length of each field; When the wire_type encoding type and the data field length Length of each field in the area meet the requirements, determine that the data of the HTTP payload is Protobuf format data; The encoding type wire_type, the data field length Length, the field number field_num and the data value value of each area field obtained after parsing are saved in Json format to obtain a key-value pair, so as to obtain the hierarchical structure of the HTTP payload; After determining that the data of the HTTP payload is Protobuf format data, when the Protobuf format data has a field number key, taking the field number key as input, and combining the encoding format of the data of the HTTP payload, outputting a corresponding reference definition file; A mapping module, used to establish a mapping relationship between the URI field and the Protobuf format data according to the parsing result, and merge the Protobuf format data of the same URI field to establish a database; A matching module is used to establish a regular expression for the definition file and the Protobuf format data according to the mapping relationship, and match the corresponding target parsing file in a pre-established comparison database according to the parsing result; the matching of the corresponding target parsing file in a pre-established comparison database according to the parsing result includes: Establishing a regular expression according to the hierarchical structure corresponding to the different Protobuf format data in the database and the reference definition file; Obtaining a target field number corresponding to the parsing result according to the regular expression; Determining a target definition file corresponding to the network traffic data according to the target field number and the hierarchical structure of the HTTP payload; Among them, the comparison database includes multiple groups of known Protobuf data of known reference definition files, each group of the known Protobuf data has the same hierarchical structure and is classified according to the size of the URI field, and each group of the Protobuf format data of the URI field in the comparison database corresponds to one reference definition file.

Images