A method and apparatus for parsing a streaming network protocol

Abstract

The embodiment of the present application provides a method and device for parsing a streaming network protocol, which are used to solve the problems of complex parser and high maintenance difficulty caused by the need to manage protocol caches in streaming protocol parsing. The method includes: preprocessing a data stream by a preprocessing unit, and determining a corresponding target parser based on the protocol type of the data stream pre-parsed by the preprocessing unit, where the data stream includes a plurality of data packets; determining, by a data preparation unit, data preparation conditions corresponding to at least one corresponding protocol parsing logic executed by the target parser based on the target parser corresponding to the current data stream, determining data to be parsed that meets the data preparation conditions from the data packets obtained by the preprocessing unit, and sending the data to the target parser; receiving, by the target parser, the data to be parsed and parsing the data to be parsed.

Description

Technical Field

[0001] This application relates to the field of network communication technologies, and in particular, to a method and apparatus for parsing a streaming network protocol. Background Art

[0002] In the field of network communication technologies, protocol parsing is a very important technology. Protocol parsing is the core input function of network security detection capabilities. With the increasing richness of communication scenarios that require security protection, including but not limited to the Internet, industrial Internet, vehicle Internet, Internet of Things, and 5G, etc., the demand for protocol parsing is also increasing. Streaming network protocol parsing means that in a complex network environment, cross-packet situations may occur, resulting in the content to be parsed not being in the same packet. Therefore, in order to accurately perform protocol parsing, it is necessary to perform parsing based on the data stream. In current common Intrusion Detection and Prevention Service (IDPS) solutions, streaming parsing is achieved by the parser itself caching fields and saving states internally. Therefore, the coding difficulty and complexity of the parser are very high. Summary of the Invention

[0003] Embodiments of this application provide a method and apparatus for parsing a streaming network protocol to solve the problems of high complexity and high maintenance difficulty of the parser caused by the need to manage protocol caches for streaming protocol parsing.

[0004] In a first aspect, embodiments of this application provide a method for parsing a streaming network protocol, including:

[0005] Preprocessing a data stream by a preprocessing unit, and determining a corresponding target parser based on the protocol type of the data stream parsed by the preprocessing unit according to the preprocessing, where the data stream includes multiple data packets;

[0006] Based on the target parser corresponding to the current data stream, a data preparation unit determines data preparation conditions corresponding to at least one corresponding protocol parsing logic executed by the target parser, determines data to be parsed that meets the data preparation conditions from the data packets obtained from the preprocessing unit, and sends it to the target parser;

[0007] Receiving the data to be parsed by the target parser and parsing the data to be parsed.

[0008] Based on the above solution, in the present application, a data preparation unit is provided. The target parser sends the data preparation conditions corresponding to the parsing logic to the data preparation unit, so that the data preparation unit determines the data to be parsed from the received data packets according to the data preparation conditions, and then sends the data to be parsed to the target parser, so that the target parser parses the data to be parsed. The target parser only needs to perform protocol parsing and does not need to perform data caching anymore. The difficulty of writing and maintaining the parser is greatly reduced, and the parser only needs to care about the protocol parsing logic. In addition, based on the above method, it is relatively easy to integrate parsers developed in various languages, as long as the corresponding parsing logic can be expressed, which greatly increases the scalability of the framework.

[0009] In a possible implementation manner, the data to be parsed includes at least one complete data packet obtained from the preprocessing unit, or at least one of partial data in the data packet obtained from the preprocessing unit.

[0010] In a possible implementation manner, determining the data to be parsed that meets the data preparation conditions from the data packets obtained from the preprocessing unit includes:

[0011] When the data to be parsed that meets the data preparation conditions determined from the data packet obtained from the preprocessing unit last time includes partial data in the data packet, determine the position of the last byte in the data to be parsed obtained last time in the data packet;

[0012] Determine the data to be parsed that meets the data preparation conditions from the data after the position in the data packet of the preprocessing unit, and obtain the data to be parsed that currently meets the data preparation conditions.

[0013] In a possible implementation manner, determining the data to be parsed that meets the data preparation conditions from the data packets obtained from the preprocessing unit further includes:

[0014] The data preparation unit receives in real time the data packets preprocessed by the preprocessing unit, and obtains the data preparation conditions corresponding to the protocol parsing logic currently executed from the target parser;

[0015] When the data packets obtained in real time do not meet the data preparation conditions, cache the data packets obtained in real time and the data preparation conditions;

[0016] When it is determined that the data to be parsed that met the data preparation conditions last time is partial data in the data packet, determine the position of the last byte in the data to be parsed obtained last time in the data packet;

[0017] After obtaining the data after the position in the cached data packet and the next data packet, determine the data to be parsed that meets the data preparation condition;

[0018] After sending the data to be parsed to the target parser, delete the data preparation condition.

[0019] In a possible implementation manner, the method further includes: sending, by the target parser, the data preparation condition corresponding to the current protocol parsing logic to the data preparation unit according to the order of the at least one parsing logic; when the data preparation unit determines that the data preparation condition is not cached currently, receive the data preparation condition from the target parser and cache it.

[0020] In a possible implementation manner, the preprocessing includes stream recombination processing, packet out-of-order and packet duplication processing. Determining the corresponding target parser according to the protocol type of the data stream parsed by the preprocessing includes:

[0021] Determine the protocol type of the data stream according to the characteristic information of multiple data packets in the data stream;

[0022] Based on the protocol type, determine the target parser for parsing the protocol type.

[0023] In a possible implementation manner, the data preparation condition includes at least one of the following types of data preparation conditions:

[0024] Data length condition, data matching condition, data length and matching condition, traffic content condition, and parsing end condition;

[0025] Among them, the data length condition is used to indicate obtaining the data to be parsed with a set length, the data matching condition is used to indicate obtaining the data to be parsed that meets the set condition, the traffic content condition is used to indicate obtaining all the data in the data packet, or obtaining all the data after the data packet in the data stream, and the parsing end condition is used to indicate ending the acquisition of the data to be parsed from the data stream.

[0026] In a possible implementation manner, before determining the data preparation conditions respectively corresponding to at least one corresponding protocol parsing logic executed by the target parser, the method further includes:

[0027] Determine that the type of the data preparation condition is not the parsing end condition.

[0028] In a second aspect, an embodiment of the present application provides a streaming network protocol parsing device, including:

[0029] A determination module, configured to preprocess a data stream through a preprocessing unit, and determine a corresponding target parser based on the protocol type of the data stream parsed by the preprocessing unit according to the preprocessing, where the data stream includes a plurality of data packets;

[0030] A data preparation unit determines data preparation conditions corresponding to at least one corresponding protocol parsing logic executed by the target parser based on the target parser corresponding to the current data stream, determines to-be-parsed data that meets the data preparation conditions from the data packets obtained from the preprocessing unit, and sends the to-be-parsed data to the target parser;

[0031] A parsing module, configured to receive the to-be-parsed data through the target parser and parse the to-be-parsed data.

[0032] In a possible implementation manner, the to-be-parsed data includes at least one complete data packet obtained from the preprocessing unit, or at least one of partial data in the data packets obtained from the preprocessing unit.

[0033] In a possible implementation manner, when the determination module determines the to-be-parsed data that meets the data preparation conditions from the data packets obtained from the preprocessing unit, it is specifically configured to:

[0034] When the to-be-parsed data that meets the data preparation conditions determined from the data packets obtained from the preprocessing unit last time includes partial data in the data packets, determine the position of the last byte in the to-be-parsed data obtained last time in the data packet;

[0035] Determine the to-be-parsed data that meets the data preparation conditions from the data after the position in the data packets of the preprocessing unit, and obtain the to-be-parsed data that currently meets the data preparation conditions.

[0036] In a possible implementation manner, when the determination module determines the to-be-parsed data that meets the data preparation conditions from the data packets obtained from the preprocessing unit, it is specifically configured to:

[0037] The data preparation unit receives the data packets preprocessed by the preprocessing unit in real time, and obtains the data preparation conditions corresponding to the protocol parsing logic currently executed from the target parser;

[0038] When the data packets obtained in real time do not meet the data preparation conditions, cache the data packets obtained in real time and the data preparation conditions;

[0039] When it is determined that the to-be-parsed data that met the data preparation conditions last time is partial data in the data packet, determine the position of the last byte in the to-be-parsed data obtained last time in the data packet;

[0040] After obtaining the data after the obtained position from the cached data packet and the next data packet, determine the data to be parsed that meets the data preparation condition;

[0041] Delete the data preparation condition after sending the data to be parsed to the target parser.

[0042] In a possible implementation, the device further includes a receiving module, and the target parser sends the data preparation condition corresponding to the current protocol parsing logic to the data preparation unit according to the order of the at least one parsing logic;

[0043] The receiving module, when the data preparation unit determines that the data preparation condition is not cached currently, receives the data preparation condition from the target parser and caches it.

[0044] In a possible implementation, the preprocessing includes stream reorganization processing, packet out-of-order and packet duplication processing. When the determining module determines the corresponding target parser according to the protocol type of the data stream parsed by the preprocessing, it specifically is used for:

[0045] Determine the protocol type of the data stream according to the feature information of multiple data packets in the data stream;

[0046] Based on the protocol type, determine the target parser for parsing the protocol type.

[0047] In a possible implementation, the data preparation condition includes at least one of the following types of data preparation conditions: data length condition, data matching condition, data length and matching condition, traffic content condition, and parsing end condition;

[0048] Among them, the data length condition is used to indicate obtaining the data to be parsed with a set length, the data matching condition is used to indicate obtaining the data to be parsed that meets the set condition, the traffic content condition is used to indicate obtaining all the data in the data packet or obtaining all the data after the data packet in the data stream, and the parsing end condition is used to indicate ending the acquisition of the data to be parsed from the data stream.

[0049] In a possible implementation, before determining the data preparation conditions corresponding to at least one corresponding protocol parsing logic executed by the target parser, the determining module is further used for:

[0050] Determine that the type of the data preparation condition is not the parsing end condition.

[0051] In a third aspect, an embodiment of the present application provides an execution device, including:

[0052] A memory for storing program instructions;

[0053] A processor, configured to obtain program instructions stored in the memory and execute the methods described in the first aspect and different implementation manners of the first aspect according to the obtained program instructions.

[0054] In a fourth aspect, the present application provides a computer-readable storage medium storing computer instructions, which, when running on a computer, cause the computer to execute the methods described in the first aspect and different implementation manners of the first aspect.

[0055] For the technical effects brought by any one of the implementation manners in the second aspect to the fourth aspect, reference may be made to the technical effects brought by the first aspect and different implementation manners of the first aspect, which will not be elaborated herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0057] Figure 1 It is a schematic diagram of the architecture of a service system provided by an embodiment of the present application;

[0058] Figure 2 It is a flowchart of a method for parsing a streaming network protocol provided by an embodiment of the present application;

[0059] Figure 3 It is an overall flowchart of parsing a streaming network protocol provided by an embodiment of the present application;

[0060] Figure 4 It is a schematic diagram of a device for parsing a streaming network protocol provided by an embodiment of the present application;

[0061] Figure 5 It is a schematic diagram of an execution device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some, but not all, embodiments of the present application. Usually, the components of the embodiments of the present application described and illustrated in the drawings here can be arranged and designed in various different configurations.

[0063] Accordingly, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but merely represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application.

[0064] It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the presence of additional identical elements in the process, method, article or device including the said element.

[0065] In the field of network communication, protocol parsing function is a very important technology. Protocol parsing is the core input function of network security detection capabilities. As the communication scenarios that require security protection become increasingly rich, including but not limited to the Internet, industrial Internet, vehicle Internet, Internet of Things, and 5G, etc., the demand for protocol parsing is also increasing. Streaming network protocol parsing means that in a complex network environment, cross-packet situations may occur, resulting in the content to be parsed not being in the same packet. Therefore, in order to accurately parse the protocol, it is necessary to parse based on the data stream. Currently, in common intrusion detection and prevention systems (IDPS) such as Snort and Suricata, the streaming parsing of protocols highly depends on the TCP stream reconstruction function. The state jumps and data caching in the streaming protocol parsing part are both maintained by the parser itself. Protocol parsing software such as Wireshark needs to cache all data packets for parsing and display, consuming a large amount of resources and not being applicable to the scenario of real-time traffic parsing.

[0066] The embodiments of the present application provide a method and apparatus for parsing a streaming network protocol. A data preparation unit is added during the parsing of the streaming network protocol to isolate the data stream from the parser. When the data preparation unit receives the data stream and the data preparation conditions sent by the parser, it determines the data to be parsed from the data stream according to the data preparation conditions, and then sends the data to be parsed to the parser. The parser obtains the data to be parsed from the data preparation unit and performs parsing. This method only needs to focus on the implementation logic of the protocol itself, greatly reducing the difficulty of writing and maintaining the parser. This method can enhance its own parsing ability by dynamically expanding the data acquisition ability of the data preparation unit.

[0067] Figure 1 Exemplarily, a schematic diagram of the architecture of a service system provided by the embodiments of the present application is shown. The service system includes a preprocessing unit S1, a parser S2, and a data preparation unit S3.

[0068] Preprocessing unit S1: It is used to receive the data stream and preprocess the data stream. Among them, the preprocessing includes stream recombination processing, packet out-of-order and packet duplication processing, and determines the protocol type of the data stream. After determining the protocol type of the data stream, it determines the target parser corresponding to the protocol type.

[0069] Target parser S2: It is used to send the data preparation conditions corresponding to the protocol parsing logic to the data preparation unit S3, and receive the data to be parsed sent by the data preparation unit S3, so as to parse the data to be parsed according to the protocol parsing logic.

[0070] Data preparation unit S3: It is an intermediate module between the preprocessing unit S1 and the target parser S2. It is used to receive the data packets sent by the preprocessing unit S1 and the data preparation conditions sent by the target parser S2, determine the data to be parsed that meets the data preparation conditions from the data packets, and send it to the target parser S2.

[0071] After the preprocessing unit S1 receives the data stream, it preprocesses the data stream and determines the target parser S2 corresponding to the protocol type of the data stream. In this service system, there are multiple parsers, and different parsers are used to parse different protocol types, and the parsing logic corresponding to each parser is fixed. Therefore, after determining the target parser S2, the parsing logic corresponding to the protocol type of the target parser S2 can be determined. Then, the data preparation conditions corresponding to the parsing logic are sent to the data preparation unit S3 one by one according to the parsing logic. The data preparation unit S3 can obtain the processed data packets from the preprocessing unit S1, determine the data to be parsed that meets the data preparation conditions from the data packets according to the data preparation conditions, and send the data to be parsed to the target parser S2. Further, the target parser S2 parses the received data to be parsed.

[0072] An embodiment of the present application provides a method for parsing a streaming network protocol. Refer to Figure 2 As shown, this method can be executed by a service system such as Figure 1 As shown. The specific process is as follows:

[0073] 201. Preprocess the data stream through a preprocessing unit, and determine a corresponding target parser based on the protocol type of the preprocessed data stream by the preprocessing unit.

[0074] In some embodiments, the preprocessing unit receives the data stream and preprocesses the data stream. Among them, the data stream includes multiple data packets. The preprocessing may include stream recombination processing, packet out-of-order and packet duplication processing. After preprocessing the data stream, a target parser corresponding to the protocol type of the data stream can be determined according to the data stream. In some scenarios, determining a corresponding target parser according to the protocol type of the preprocessed data stream includes: determining the protocol type of the data stream according to the characteristic information of multiple data packets in the data stream; determining a target parser for parsing the protocol type based on the protocol type.

[0075] 202. The data preparation unit determines data preparation conditions corresponding to at least one corresponding protocol parsing logic executed by the target parser based on the target parser corresponding to the current data stream, determines the data to be parsed that meets the data preparation conditions from the data packets obtained from the preprocessing unit, and sends it to the target parser.

[0076] In some embodiments, the target parser determines corresponding data preparation conditions according to the data used when executing the parsing logic. For example, if the data used when executing the current parsing logic is 13 bytes after the data corresponding to the previous parsing logic, the data preparation condition is to obtain 13 bytes. Further, the target parser can send the data preparation conditions corresponding to the current protocol parsing logic to the data preparation unit according to the order of at least one parsing logic. For example, if the protocol parsing logics of the target parser include A1, A2, and A3 in sequence, the target parser will first send the data preparation conditions corresponding to the protocol parsing logic A1 to the data preparation unit. Further, when the data preparation unit determines that the data preparation conditions are not cached currently, it receives the data preparation conditions from the target parser and caches them.

[0077] In some embodiments, the data preparation unit can obtain the preprocessed data packets from the preprocessing unit. After determining that the data preparation unit obtains the data packets and the data preparation conditions, the data to be parsed that meets the data preparation conditions can be determined from the data packets.

[0078] In some scenarios, the data to be parsed includes at least one complete data packet obtained from the preprocessing unit, or at least one of the partial data in the data packets obtained from the preprocessing unit. For example, the data to be parsed includes 5 bytes in a data packet, or the data content of 2 data packets, or the partial bytes of 2 data packets and the 3rd data packet.

[0079] In some embodiments, the data to be parsed that meets the data preparation condition is determined from the data packets obtained from the preprocessing unit, and can be specifically implemented in the following manner: when the data to be parsed that meets the data preparation condition determined from the data packet obtained from the preprocessing unit last time includes partial data in the data packet, determine the position of the last byte in the data to be parsed obtained last time in the data packet. Further, the data to be parsed that meets the data preparation condition can be determined from the data after the position in the data packet obtained from the preprocessing unit, and the currently data to be parsed that meets the data preparation condition is obtained.

[0080] As an example, the data packet from the preprocessing unit obtained by the data preparation unit is Packet1. When the data to be parsed data1 that meets the data preparation condition C1 determined from Packet1 last time includes partial data in Packet1, determine the position of the last byte in the data to be parsed data1 in Packet1. Further, the data to be parsed that meets the data preparation condition C2 can be determined from the data after the position of the last byte of the data to be parsed data1 determined last time in the data packet Packet1. Among them, the order of the protocol parsing logic corresponding to the data preparation condition C2 is later than the protocol parsing logic corresponding to the data preparation condition C1. For example, when the position of the last byte in the data to be parsed data1 in Packet1 is 7. Then, the data to be parsed that meets the data preparation condition C2 is determined after the 8th byte in the data packet Packet1.

[0081] In some other embodiments, the to-be-parsed data that meets the data preparation condition can be determined from the data packets obtained from the preprocessing unit in the following manner: The data preparation unit receives in real time the data packets preprocessed by the preprocessing unit, and obtains the data preparation condition corresponding to the currently executed protocol parsing logic from the target parser. As an example, the data packet obtained from the preprocessing unit is Packet1, and the data preparation condition obtained from the target parser is C1. When the data packet obtained in real time does not meet the data preparation condition, the data packet obtained in real time and the data preparation condition are cached. That is, when the data in the data packet Packet1 does not meet the data preparation condition C1, the data packet Packet1 and the data preparation condition C1 are cached. Further, when it is determined that the to-be-parsed data that met the data preparation condition last time is part of the data in the data packet, the position of the last byte in the to-be-parsed data obtained last time in the data packet is determined. That is, it is determined whether the to-be-parsed data determined last time contains the data in the data packet Packet1. If the to-be-parsed data determined last time contains part of the data in Packet1, the position of the last byte in the to-be-parsed data determined last time in the data packet Packet1 is determined. For example, the position of the last byte of the to-be-parsed data last time in Packet1 is 7.

[0082] Further, the to-be-parsed data that meets the data preparation condition can be determined from the data after the position obtained from the cached data packet and the next data packet. As an example, after it is determined that the data packet Packet1 does not meet the data preparation condition C1, the next data packet Packet2 is obtained from the preprocessing unit, and the to-be-parsed data that meets the data preparation condition C1 is determined starting from the 8th byte in the data packet Packet1.

[0083] In some embodiments, after the to-be-parsed data that meets the data preparation condition is determined, the to-be-parsed data can be sent to the target parser, and the data preparation condition is deleted. That is, after the to-be-parsed data is obtained, the data preparation condition C1 in the data preparation unit can be deleted.

[0084] In some embodiments, the data preparation condition includes at least one of the following types of data preparation conditions: data length condition, data matching condition, data length and matching condition, traffic content condition, and parsing end condition.

[0085] Among them, the data length condition is used to indicate to obtain to-be-parsed data of a set length. For example, a length of 20 bytes is required, which can be expressed as NEED(20). Then the data preparation condition is that the parser needs the data 20 bytes offset from the current position of the current data packet, and the data preparation unit returns the corresponding data content to the target parser after determining the to-be-parsed data.

[0086] The data matching condition is used to indicate the acquisition of data to be parsed that meets the set conditions. For example, if it is necessary to return the data before "\r\n", it can be expressed as NEED("\r\n"). Then the data preparation condition is that the parser needs the data between the current position of the current data packet and until "\r\n" is encountered, and then returns the determined data content to the target parser.

[0087] The traffic content condition is used to indicate the acquisition of all data in the data packet, or the acquisition of all data after the data packet in the data stream. For example, to obtain all the content of the current packet, it can be expressed as NEED(packet). Then the data preparation condition is that the parser needs the content from the current position of the current stream to the end of the current data packet after offset. For example, if it is necessary to obtain all the content of the current data stream, it can be expressed as NEED(stream). Then the data preparation condition is that the parser needs the data content from the current position of the current data stream until the end of the current data stream.

[0088] The parsing end condition is used to indicate the end of the acquisition of data to be parsed from the data stream. For example, if it is necessary to end the current decoding, it can be expressed as NEED(exit). Then the operation of acquiring data to be parsed from the data stream will end.

[0089] In some embodiments, the attributes of the data preparation condition can be changed by adding options. For example, if the data preparation condition is NEED(20, ignore), then the data preparation condition is that it is necessary to skip the content of 20 bytes after the current position of the current data packet and does not need to return.

[0090] In some embodiments, before determining the data preparation conditions corresponding to at least one corresponding protocol parsing logic executed by the target parser, it is also necessary to determine that the type of the data preparation condition is not the parsing end condition.

[0091] 203, receive the data to be parsed through the target parser and parse the data to be parsed.

[0092] Through the above solution, a data preparation unit is added in the process of parsing the streaming network protocol to isolate the data stream from the parser. When the data preparation unit receives the data stream and the data preparation condition sent by the parser, it will determine the data to be parsed from the data stream according to the data preparation condition, and then send the data to be parsed to the parser. This method of protocol parsing only needs to focus on the implementation logic of the protocol itself, greatly reducing the difficulty of writing and maintaining the parser. This method can enhance its own parsing ability by dynamically expanding the data acquisition ability of the data preparation unit.

[0093] As an example, the overall process of the streaming network protocol parsing method provided by the embodiments of the present application is as Figure 3 shown as follows:

[0094] Step 1 (S1.1): Initialize the parser and load the protocol parser. The parser can include parsers for different languages, such as C language parser, Rust language parser, Lua language parser, and other language parsers, etc.

[0095] Step 2 (S1.2): Read the externally input data stream and preprocess the data stream (stream reorganization, handling problems such as packet disorder and packet duplication).

[0096] Step 3 (S1.3): Identify the protocol type of the current data stream based on the characteristics of the data stream and determine the target parser corresponding to the protocol type.

[0097] Step 4 (S3.1): Determine whether the current data stream has ended. If the data stream has ended, end this processing. Otherwise, proceed to Step 5.

[0098] Step 5 (S3.2): Obtain the data packet.

[0099] Step 6 (S3.3): Detect whether there is a data preparation condition for the cached data. If so, execute Step 9. If not, execute Step 7.

[0100] Step 7 (S2.1): Execute the protocol parsing logic in the protocol parser to perform protocol parsing on the data to be parsed.

[0101] Step 8 (S2.2): Determine the data preparation condition. In protocol parsing, according to the parsing logic, obtain the original content in the data packet as needed, and this operation is completed through the data preparation condition. After determining the data preparation condition, execute Step 11.

[0102] Step 9 (S3.4): Restore the data preparation condition. Restore the data preparation condition cached in the data preparation unit.

[0103] Step 10 (S3.5): Restore the data packet content. Restore the data packet content cached in the data preparation unit.

[0104] Step 11 (S3.6): Determine whether the data preparation condition is the parsing end condition. If so, directly end; if not, execute Step 12.

[0105] Step 12 (S3.7): Execute the data preparation logic. Determine the corresponding data to be parsed according to the data preparation condition.

[0106] Step 13 (S3.8): Determine whether the data preparation condition is met. Determine whether the data packet meets the data preparation condition. If it meets, return the determined data to be parsed to the parser so that the parser can execute Step 7 based on the data to be parsed. If it does not meet, it means that the content of the current data packet does not meet the data required by the data preparation condition, and then execute Step 14.

[0107] Step 14 (S3.9): Cache the data preparation condition. Since the current data packet cannot meet the current data preparation condition, cache the currently executing data preparation condition.

[0108] Step 15 (S3.10): Cache the content of the data packet. Since the current data packet cannot meet the current data preparation condition, cache the content of the current relevant data packet and execute Step S3.1.

[0109] Among them, the above Steps S1.1 - S1.3 are executed by the preprocessing unit, Steps S2.1 - S2.2 are executed by the target parser, and Steps S3.1 - 3.10 are executed by the data preparation unit.

[0110] This application proposes a streaming parsing framework. Different from the general protocol parsing framework, it decouples the state maintenance of the data stream from the protocol parsing module and adds a layer of data stream state maintenance layer, that is, the data preparation unit. Determine the data preparation conditions of the parser through S3.6, S3.7, and S3.8. Cache the data preparation conditions and the content of the data packet through S3.9, S3.10, and S3.1 - S3.5 to achieve streaming decoding. The data preparation unit can provide a general data acquisition interface and can dynamically expand the data acquisition ability. The parser only needs to obtain the required data to be parsed from the data preparation unit according to the parsing logic. The managers of the protocol parser can complete the protocol parsing only through S2.1 and S2.2. By providing the data stream management ability through the framework, that is, adding the data preparation unit, the difficulty of writing and maintaining the parser is greatly reduced. The parser only needs to care about the protocol parsing logic, that is, the steps involved in the parser will be greatly reduced. Due to the general ability provided by the framework, it is also easy to integrate parsers developed in various languages as long as they can express the corresponding parsing logic, greatly increasing the scalability of the framework.

[0111] Based on the same technical concept, an embodiment of this application provides a streaming network protocol parsing device 400, as Figure 4 shown. The device 400 can execute any step in the above-mentioned streaming network protocol parsing method. To avoid repetition, it will not be elaborated here. The device 400 includes a determination module 401, a parsing module 402, and a receiving module 403.

[0112] A determination module 401 is configured to preprocess a data stream through a preprocessing unit, and determine a corresponding target parser based on the protocol type of the data stream parsed by the preprocessing unit according to preprocessing. The data stream includes a plurality of data packets.

[0113] A data preparation unit determines data preparation conditions corresponding to at least one corresponding protocol parsing logic executed by the target parser based on the target parser corresponding to the current data stream, determines data to be parsed that meets the data preparation conditions from the data packets obtained from the preprocessing unit, and sends the data to the target parser.

[0114] A parsing module 402 is configured to receive the data to be parsed through the target parser and parse the data to be parsed.

[0115] In some embodiments, the data to be parsed includes at least one complete data packet obtained from the preprocessing unit, or at least one of partial data in the data packets obtained from the preprocessing unit.

[0116] In some embodiments, when the determination module 401 determines the data to be parsed that meets the data preparation conditions from the data packets obtained from the preprocessing unit, it is specifically configured to: when the data to be parsed that meets the data preparation conditions determined from the data packets obtained from the preprocessing unit last time includes partial data in the data packets, determine the position of the last byte in the data to be parsed obtained last time in the data packets; determine the data to be parsed that meets the data preparation conditions from the data after the position in the data packets of the preprocessing unit, and obtain the data to be parsed that currently meets the data preparation conditions.

[0117] In some embodiments, when the determination module 401 determines the data to be parsed that meets the data preparation conditions from the data packets obtained from the preprocessing unit, it is specifically configured to: receive in real time the data packets preprocessed by the preprocessing unit through the data preparation unit, and obtain the data preparation conditions corresponding to the protocol parsing logic currently executed from the target parser; when the data packets obtained in real time do not meet the data preparation conditions, cache the data packets obtained in real time and the data preparation conditions; when it is determined that the data to be parsed that met the data preparation conditions last time is partial data in the data packets, determine the position of the last byte in the data to be parsed obtained last time in the data packets; determine the data to be parsed that meets the data preparation conditions from the data after the obtained position in the cached data packets and the next data packet; delete the data preparation conditions after sending the data to be parsed to the target parser.

[0118] In some embodiments, the device further includes a receiving module 403, and the target parser sends the data preparation conditions corresponding to the current protocol parsing logic to the data preparation unit according to the order of the at least one parsing logic.

[0119] When the data preparation unit determines that the current data preparation conditions are not cached, the receiving module 403 receives the data preparation conditions from the target parser and caches them.

[0120] In some embodiments, the preprocessing includes stream reorganization processing, packet out-of-order and packet duplication processing. When determining the corresponding target parser according to the protocol type of the data stream parsed by the preprocessing, the determining module 401 is specifically configured to: determine the protocol type of the data stream according to the characteristic information of multiple data packets in the data stream; determine the target parser for parsing the protocol type based on the protocol type.

[0121] In some embodiments, the data preparation conditions include at least one of the following types of data preparation conditions: data length condition, data matching condition, data length and matching condition, traffic content condition, and parsing end condition; wherein, the data length condition is used to indicate obtaining the to-be-parsed data of a set length, the data matching condition is used to indicate obtaining the to-be-parsed data that meets the set conditions, the traffic content condition is used to indicate obtaining all the data in the data packet, or obtaining all the data after the data packet in the data stream, and the parsing end condition is used to indicate ending the acquisition of the to-be-parsed data from the data stream.

[0122] In some embodiments, before determining the data preparation conditions corresponding to at least one corresponding protocol parsing logic executed by the target parser, the determining module 401 is further configured to: determine that the type of the data preparation conditions is not the parsing end condition.

[0123] Based on the same technical concept, an embodiment of the present application provides an execution device 500, which can implement any step of the streaming network protocol parsing method described above. Please refer to Figure 5 . The device includes a memory 501 and a processor 502.

[0124] The memory 501 is used to store program instructions;

[0125] The processor 502 is configured to call the program instructions stored in the memory and execute the above-mentioned streaming network protocol parsing method according to the obtained program.

[0126] In the embodiments of the present application, the processor 502 may be a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and may implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0127] As a non-volatile computer-readable storage medium, the memory 501 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory 501 may include at least one type of storage medium. For example, it may include flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (RAM), a static random access memory (SRAM), a programmable read-only memory (PROM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic memory, a magnetic disk, an optical disk, and so on. The memory 501 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 501 in the embodiments of the present application may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and / or data.

[0128] Based on the same technical concept, the embodiments of the present application provide a computer-readable storage medium, including: computer program code, which, when running on a computer, causes the computer to execute the streaming network protocol parsing method as described above. Since the principle of solving problems by the above computer-readable storage medium is similar to that of the streaming network protocol parsing method, the implementation of the above computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be described again.

[0129] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0130] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0131] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufacture including instruction means that implement the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks or combinations of blocks.

[0133] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A method for parsing a streaming network protocol, characterized in that Including: Preprocessing a data stream by a preprocessing unit, and determining a corresponding target parser based on the protocol type of the preprocessed data stream by the preprocessing unit, where the data stream includes a plurality of data packets; Based on the target parser corresponding to the current data stream, a data preparation unit determines data preparation conditions corresponding to at least one corresponding protocol parsing logic executed by the target parser, determines data to be parsed that meets the data preparation conditions from the data packets obtained from the preprocessing unit, and sends it to the target parser; Receiving the data to be parsed by the target parser and parsing the data to be parsed.

2. The method according to claim 1, wherein The data to be parsed includes at least one complete data packet obtained from the preprocessing unit, or at least one of partial data in the data packets obtained from the preprocessing unit.

3. The method according to claim 2, wherein Determining the data to be parsed that meets the data preparation conditions from the data packets obtained from the preprocessing unit includes: When the data to be parsed that meets the data preparation conditions determined from the data packets obtained from the preprocessing unit last time includes partial data in the data packets, determining the position of the last byte in the data to be parsed obtained last time in the data packets; Determining the data to be parsed that meets the data preparation conditions from the data after the position in the data packets obtained from the preprocessing unit, and obtaining the data to be parsed that currently meets the data preparation conditions.

4. The method according to claim 2, wherein Determining the data to be parsed that meets the data preparation conditions from the data packets obtained from the preprocessing unit further includes: The data preparation unit receives in real time the data packets preprocessed by the preprocessing unit, and obtains the data preparation conditions corresponding to the protocol parsing logic currently executed from the target parser; When the data packets obtained in real time do not meet the data preparation conditions, caching the data packets obtained in real time and the data preparation conditions; When it is determined that the data to be parsed that met the data preparation conditions last time is partial data in the data packets, determining the position of the last byte in the data to be parsed obtained last time in the data packets; Determining the data to be parsed that meets the data preparation conditions from the data after the obtained position in the cached data packets and the next data packet; Deleting the data preparation conditions after sending the data to be parsed to the target parser.

5. The method according to any one of claims 1-4, characterized in that, Further including: The target parser sends the data preparation conditions corresponding to the current protocol parsing logic to the data preparation unit according to the order of the at least one parsing logic; When the data preparation unit determines that there is no cached data preparation condition currently, receiving the data preparation condition from the target parser and caching it.

6. The method according to claim 1, wherein The preprocessing includes stream reorganization processing, packet out-of-order and packet duplication processing. Determining a corresponding target parser according to the protocol type of the data stream parsed by the preprocessing includes: Determining the protocol type of the data stream according to the characteristic information of a plurality of data packets in the data stream; Based on the protocol type, determining a target parser for parsing the protocol type.

7. The method according to any one of claims 1-4 and 6, characterized in that, The data preparation conditions include at least one of the following types of data preparation conditions: Data length condition, data matching condition, data length and matching condition, traffic content condition, and parsing end condition; Among them, the data length condition is used to indicate obtaining to-be-parsed data of a set length, the data matching condition is used to indicate obtaining to-be-parsed data that meets the set conditions, the traffic content condition is used to indicate obtaining all the data in the data packet or obtaining all the data after the data packet in the data stream, and the parsing end condition is used to indicate ending the acquisition of to-be-parsed data from the data stream.

8. The method according to claim 7, wherein Before determining the data preparation conditions respectively corresponding to at least one corresponding protocol parsing logic executed by the target parser, the method further includes: Determining that the type of the data preparation condition is not the parsing end condition.

9. A streaming network protocol parsing device, characterized in that, Including: A determining module, configured to preprocess a data stream through a preprocessing unit and determine a corresponding target parser based on the protocol type of the data stream preprocessed by the preprocessing unit, where the data stream includes multiple data packets; A data preparation unit determines the data preparation conditions respectively corresponding to at least one corresponding protocol parsing logic executed by the target parser based on the target parser corresponding to the current data stream, determines to-be-parsed data that meets the data preparation conditions from the data packets obtained from the preprocessing unit, and sends the to-be-parsed data to the target parser; A parsing module, configured to receive the to-be-parsed data through the target parser and parse the to-be-parsed data.

10. An execution device, characterized in that, Including: A memory, configured to store program instructions; A processor, configured to obtain the program instructions stored in the memory and execute the method according to any one of claims 1-8 according to the obtained program instructions.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and when the computer instructions run on a computer, the computer is caused to execute the method according to any one of claims 1-8.

Images