Method and apparatus for probing payload of private protocol

Abstract

The application provides a private protocol detection payload mining method and device, the method comprises the following steps: randomly selecting a piece of data from a test data set as target test data, and sending the target test data to a target program; wherein the target program is used for protocol analysis on the received data according to a target private protocol; obtaining target protocol analysis state information obtained in each protocol analysis period of the target program analyzing the target test data in sequence; determining the constraint content of the target test data of the target program based on the target protocol analysis state information; based on the constraint content, the target test data is mutated to generate multiple pieces of mutated data; the mutated data is added to the test data set to mine the detection payload of the target private protocol according to the updated test data set. In this way, the data obtained by mutation is closer to the format specification of the target private protocol, the test is more focused on the protocol analysis code, has high universality, and improves the test efficiency.

Description

Technical Field

[0001] This invention relates to the field of fuzz testing technology, and in particular to a method and apparatus for detecting payloads using a proprietary protocol. Background Technology

[0002] Fuzzing techniques are commonly used in software vulnerability discovery, protocol probe payload discovery, and software testing. Based on the "law of large numbers," fuzzing can uncover hidden vulnerabilities by repeating the test a sufficient number of times and ensuring the randomness of the input data. Generally, fuzzing can be categorized into white-box fuzzing, gray-box fuzzing, and black-box fuzzing, depending on the level of understanding of the target program. Gray-box fuzzing typically involves dynamic software instrumentation, seed selection, test data generation, test data input, program anomaly detection, and seed pool updates. The well-known fuzzing tool AFL pioneered coverage-guided fuzzing. In each test, test data that triggers a new coverage level in the target program is added to the seed pool and given higher priority in subsequent tests. This improves the quality of the seeds in the seed pool, ensuring the entire test process comprehensively covers the target program's code logic. Currently, coverage-guided fuzzing has become the mainstream strategy, used by tools such as VUzzer and Angora.

[0003] In the scenario of probing payloads using proprietary protocols, repeated fuzzing is used to continuously mutate the input test data to conform to the protocol format specifications of the proprietary protocol, thereby obtaining the target program's response data through the protocol format verification process. The iterative testing process primarily focuses on the communication module within the target program's code. However, using coverage-guided testing in this case presents several problems: the communication module may only exist as a small part of the target software; a high coverage rate triggered by test data may not necessarily originate in the communication module but could also be triggered in other code unrelated to input data parsing. In such cases, testing might be redirected to code unrelated to the communication module. Furthermore, the target program may require field-by-field validation of the input data. Regardless of the success or failure of a single field validation, it could trigger a new coverage rate. Coverage-guided testing cannot accurately distinguish whether the test data more closely conforms to the target proprietary protocol's format specifications, significantly reducing the overall efficiency of the testing process. Summary of the Invention

[0004] This invention provides a method and apparatus for mining the probe payload of a proprietary protocol, which enables test data to closely approximate the protocol format specification of the proprietary protocol with fewer mutations in fuzzing scenarios where effective seeds are lacking. This improves fuzzing efficiency and has high universality, as it can be performed without the need for complete control flow information of the target program.

[0005] This invention provides a method for mining probe payloads using a proprietary protocol, comprising:

[0006] A single data point is randomly selected from the test dataset as the target test data, and the target test data is sent to the target program; wherein, the target program is used to perform protocol parsing on the received data according to the target private protocol;

[0007] Sequentially obtain the target protocol parsing status information obtained in each protocol parsing cycle of the target program parsing the target test data;

[0008] Based on the target protocol parsing status information, the constraints of the target program on the target test data are determined;

[0009] Based on the constraints, the target test data is mutated to generate multiple mutated data.

[0010] The mutated data is added to the test data set to mine the probe payload of the target private protocol based on the updated test data set.

[0011] According to a method for mining the probe payload of a proprietary protocol provided by the present invention, the target protocol parsing status information includes: the assembly instruction information of the target program for parsing the target test data, the instruction operand content, the operand taint status, and the offset of the content verified by the instruction in the target test data.

[0012] According to a method for mining probe payloads of a proprietary protocol provided by the present invention, the step of determining the constraint content of the target program on the target test data based on the parsed state information of the target protocol includes:

[0013] For each protocol parsing cycle, the target protocol parsing status information is acquired, and the target protocol parsing status information is compared with the protocol parsing status information in the protocol parsing status information set.

[0014] When the target protocol parsing status information includes a new protocol parsing status, the target protocol parsing status information is saved to the protocol parsing status information set, and the constraint content is obtained based on the new protocol parsing status.

[0015] According to a method for mining the probe payload of a proprietary protocol provided by the present invention, the step of sequentially acquiring the target protocol parsing status information obtained by the target program parsing the target test data in each protocol parsing cycle includes:

[0016] Whenever a protocol parsing cycle ends, at least one protocol parsing state generated within that protocol parsing cycle is obtained;

[0017] The at least one protocol parsing state is determined as the target protocol parsing state information for the protocol parsing cycle.

[0018] According to the method for mining the probe payload of a private protocol provided by the present invention, before obtaining at least one protocol parsing state generated within the protocol parsing cycle whenever the end of a protocol parsing cycle is determined, the method further includes:

[0019] Whenever the target program calls the specified read function, the target read data read by the specified read function is saved to a preset array; wherein, the target read data is the data read by the target program in segments from the target test data;

[0020] Obtain the target protocol parsing status obtained by the target program parsing the target read data;

[0021] If the target protocol parsing state satisfies the specified protocol parsing state, the target protocol parsing state is determined as the initial protocol parsing state of the protocol parsing cycle.

[0022] The moment when the target program calls the specified read function to read the target data is determined as the starting point of the protocol parsing cycle;

[0023] If the target protocol parsing state does not satisfy the specified protocol parsing state, and after determining the initial protocol parsing state of the protocol parsing cycle, the target protocol parsing state is determined as another protocol parsing state of the protocol parsing cycle.

[0024] According to a method for mining probe payloads of a proprietary protocol provided by the present invention, determining the end of a protocol parsing cycle includes:

[0025] When an initial protocol parsing state is obtained again or the target program closes the socket, the end point of the protocol parsing cycle is determined.

[0026] According to a method for mining probe payloads using a proprietary protocol provided by the present invention, before randomly selecting a data point from the test data set as the target test data and sending the target test data to the target program, the method includes:

[0027] Instrument multiple I / O functions of the target program;

[0028] Send a first random data to the target program;

[0029] Based on the first I / O function called by the target program to read the first random data collected by the stake probe, the first I / O function is determined to be the specified reading function.

[0030] According to the method for probing payloads of a proprietary protocol provided by the present invention, the IO functions include: IO functions of a blocking I / O model, IO functions of a non-blocking I / O model, IO functions of a message-based WSAAsyncSelect model, IO functions of an event-based WSAEventSelect model, IO functions of an overlapped I / O model, and IO functions of a completion port model.

[0031] The method for mining probe payloads using a proprietary protocol according to the present invention further includes:

[0032] Based on the data collected by the insertion probe, the length of the first random data read by the first I / O function during the first call of the target program is the expected data length;

[0033] Send a second random data with a length equal to the desired data length to the target program;

[0034] The target program sequentially obtains multiple protocol parsing states obtained by parsing the second random data, and determines the first obtained protocol parsing state as the specified protocol parsing state.

[0035] The present invention also provides a proprietary protocol-based probe payload excavation device, comprising:

[0036] The sending module is used to randomly select a piece of data from the test data set as the target test data and send the target test data to the target program; wherein, the target program is used to perform protocol parsing on the received data according to the target private protocol;

[0037] The acquisition module is used to sequentially acquire the target protocol parsing status information obtained by the target program in each protocol parsing cycle of parsing the target test data;

[0038] The determination module is used to determine the constraint content of the target program on the target test data based on the target protocol parsing status information;

[0039] The generation module is used to mutate the target test data based on the constraint content to generate multiple mutated data;

[0040] The processing module is used to add the mutation data to the test data set in order to mine the probe payload of the target private protocol based on the updated test data set.

[0041] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a probe payload mining method according to any of the proprietary protocols described above.

[0042] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a probe payload mining method according to any of the proprietary protocols described above.

[0043] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a probe payload mining method according to any of the proprietary protocols described above.

[0044] The method and apparatus for mining probe payloads of a private protocol provided by this invention utilize test data to perform fuzz testing on a target program. After receiving the test data, the target program parses the test data based on the target private protocol. During the parsing process, each protocol parsing cycle generates corresponding protocol parsing status information. The protocol parsing status information focuses on the target program's parsing and verification process of the test data. From the protocol parsing status information, the constraints imposed by the target program on the test data can be determined. Then, the test data is mutated using the constraints to obtain multiple mutated data. The mutated data obtained in this way is closer to the format specification of the target private protocol and is more focused on the protocol parsing code, thus having a higher probability of passing the target program's parsing and verification. The target program is then tested again using the mutated data. By using this mutated data to mine the probe payloads of the target private protocol, the coverage rate can be improved, and it is ensured that the high coverage rate triggered is related to the parsing and verification of the target private protocol. Compared with the prior art, this method has high universality and improves testing efficiency. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figure 1 This is one of the flowcharts illustrating the proprietary protocol-based payload mining method provided by this invention;

[0047] Figure 2This is the second flowchart of the proprietary protocol-based payload mining method provided by the present invention;

[0048] Figure 3 This is a schematic diagram of the structure of the proprietary protocol-based detection payload excavation device provided by the present invention;

[0049] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0051] It should be noted that, in the description of the embodiments of the present invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. Those skilled in the art can understand the specific meaning of the above terms in the present invention according to the specific circumstances.

[0052] The terms "first," "second," etc., used in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0053] The following is combined Figures 1-2 This invention describes a proprietary protocol-based method for detecting payloads.

[0054] like Figure 1 As shown, the present invention provides a method for mining probe payloads using a proprietary protocol, which can be applied to testing tools, including:

[0055] Step 110: Randomly select one piece of data from the test data set as the target test data, and send the target test data to the target program; wherein, the target program is used to perform protocol parsing on the received data according to the target private protocol.

[0056] Specifically, the test dataset is used to store the data to be tested. When the testing tool starts testing, it randomly selects a data point from the test dataset as the target test data and sends it to the target program, which can then receive the target test data.

[0057] Step 120: Sequentially obtain the target protocol parsing status information obtained by the target program in each protocol parsing cycle of parsing the target test data.

[0058] Specifically, proprietary protocols typically specify the data length required to parse a complete session. However, at the initial stage of fuzz testing of a proprietary protocol, the data length required for parsing a complete session is unknown. Generally, test data is set to a relatively long length to prevent the proprietary protocol from being unable to parse a complete session. One protocol parsing cycle is the time required to parse a complete session. Therefore, the target program parses the received target test data in units of complete session data, thereby obtaining the target protocol parsing status information for each protocol parsing cycle.

[0059] In one embodiment, the target protocol parsing state refers to the state of the target program when parsing data. Specifically, the target protocol parsing state information may include: assembly instruction information of the target program parsing the target test data, instruction operand content, operand taint status, and the offset of the content verified by the instruction within the target test data. The assembly instruction information may include assembly instructions, instruction addresses, etc.

[0060] Step 130: Determine the constraints imposed by the target program on the target test data based on the target protocol parsing status information.

[0061] Specifically, constraint content refers to the constraint condition that exists in the data at the constraint position of the target test data by the target program. The constraint content, including the constraint condition and the constraint position, can be determined from the target protocol parsing state information.

[0062] For example, suppose there exists a private protocol p, consisting of fields f1 (4 bytes) and f2 (4 bytes). The protocol specification requires that field f1 contain the protocol identifier string "test", and f2 be an integer value less than 0x700. Suppose there exists a target program test.exe that implements the parsing process of the private protocol p. At address 0x40000, there exists instruction 1: cmp[eax], 0x74736574, which verifies whether field f1 conforms to the specification; and at address 0x4000B, there exists instruction 2: cmp[eax], 0x700, which verifies whether field f2 conforms to the protocol specification. When the target program executes these two instructions to parse the data, it will generate two different "protocol parsing states". The instruction address for the "protocol parsing status" of instruction 1 is 0x40000, the assembly instruction is cmp[eax], 0x700, the operands include operand [eax] and comparison value 0x700, and the offset is 0x0-0x4. Therefore, instruction 1 imposes a constraint of 0x700 on the data content at positions 0-4 of the target test data. Instruction 2 corresponds to a parsing status address of 0x4000B, the assembly instruction is cmp[eax] 0x74736574, the operands are [eax] and comparison value 0x74736574, and the offset is 0x4-0x8. Instruction 2 imposes a constraint of 0x74736574 on the data content at positions 4-8 of the target test data.

[0063] Step 140: Based on the constraints, mutate the target test data to generate multiple mutated data.

[0064] Taking instruction 1 as an example, instruction 1 is cmp[eax],0x700, which compares the operand [eax] with 0x700. The comparison result may be greater than, less than or equal to, so the possible solution set is {0x6ff,0x700,0x701}. Based on the possible solution set, three new mutated data can be generated, that is, {0x6ff,0x700,0x701} is converted into a byte stream of length 4 according to the corresponding byte order, and the data in 0x0-0x4 in the original test data is replaced, thereby generating new test data.

[0065] Step 150: Add the mutation data to the test data set to mine the probe payload of the target private protocol based on the updated test data set.

[0066] Specifically, the mutated data is used as test data to perform fuzz testing on the target program. This iterative testing can quickly pass the target program's verification process for the input data and rapidly generate test data that conforms to the protocol specifications of the target's private protocol, enabling the test data to pass the target program's parsing and verification.

[0067] The method for mining probe payloads of a private protocol provided by this invention uses test data to perform fuzz testing on a target program. After receiving the test data, the target program parses the test data based on the target private protocol. During the parsing process, each protocol parsing cycle generates corresponding protocol parsing status information. The protocol parsing status information focuses on the target program's parsing and verification process of the test data. From the protocol parsing status information, the constraints imposed by the target program on the test data can be determined. Then, the test data is mutated using the constraints to obtain multiple mutated data. The mutated data obtained in this way is closer to the format specification of the target private protocol and has a higher probability of passing the target program's parsing and verification. The target program is then tested again using the mutated data. Mining probe payloads of the target private protocol using this mutated data can improve coverage and ensure that the high coverage triggered is related to the parsing and verification of the target private protocol. Compared with the prior art, this method has high universality and improves testing efficiency.

[0068] It should be noted that in fuzz testing scenarios for proprietary protocols, there is typically no prior knowledge of the target proprietary protocol's format specifications or available protocol traffic sample data initially; in other words, effective seed data is lacking. Therefore, before acquiring mutation data, a single data point can be randomly generated as the initial target test data and added to the test data set. In one embodiment, random streaming data can be generated and sent to the target program.

[0069] Before performing fuzz testing on the target program, it is necessary to preprocess the target program and obtain relevant information.

[0070] like Figure 2 As shown, in one embodiment, the specific preprocessing process includes the following steps:

[0071] Step 210: Instrument multiple I / O functions of the target program.

[0072] Specifically, after the target program runs, the testing tool can attach to the target process of the target program using the DBI (Dynamic Binary Instrumentation) tool to listen for events during fuzz testing and obtain relevant information. First, instrumentation is performed on multiple I / O functions in the target program to determine the read functions used by the target program to read data.

[0073] In one embodiment, the IO function includes: IO functions of a blocking I / O model, IO functions of a non-blocking I / O model, IO functions of a message-based WSAAsyncSelect model, IO functions of an event-based WSAEventSelect model, IO functions of an overlapped I / O model, and IO functions of a completion port model.

[0074] Step 220: Send a first random data to the target program.

[0075] Specifically, the first random data can be data of any length.

[0076] Step 230: Based on the first IO function called by the target program to read the first random data collected by the stake probe, determine that the first IO function is the specified reading function.

[0077] Specifically, after the target program receives the first random data, it will call some of the IO functions that were instrumented in step 210. These IO functions can be collected by the instrumentation probe, and the IO model used by the target program can be identified based on these IO functions. The specific first IO function for reading the first random data can be determined, and thus the first IO function is determined as the designated reading function of the target program for subsequent fuzzing process.

[0078] In one embodiment, the specific preprocessing process further includes the following steps:

[0079] Step 240: Based on the length of the first random data read by the first I / O function when the target program first calls the first IO function, the length of the data is determined to be the expected data length.

[0080] Specifically, based on the information collected by the instrumentation probe when the target program calls the first I / O function, the data length when the target program first calls the first I / O function can be determined.

[0081] Step 250: Send a second random data with a length equal to the desired data length to the target program.

[0082] Step 260: Sequentially obtain multiple protocol parsing states obtained by the target program from parsing the second random data, and determine the first obtained protocol parsing state as the specified protocol parsing state.

[0083] Specifically, when the testing tool detects that the target program calls the first IO function to read the second random data, it can perform taint tracking on the second random data. It will detect that the target program is parsing the second random data and will obtain all the protocol parsing states that appear during the parsing process in sequence. The first protocol parsing state obtained is the initial protocol parsing state generated by the target program when parsing a piece of data. It will be determined as the specified protocol parsing state for use in subsequent fuzzing tests.

[0084] Thus, the preprocessing process concludes, yielding two pieces of information: the specified read function and the specified protocol parsing state. The specified read function determines the specific function used by the target program to read and parse the data, while the specified protocol parsing state determines the initial protocol parsing state generated when the target program begins parsing the data. These two pieces of information will be used to define the start and end points of each protocol parsing cycle when the target program parses the target test data.

[0085] In one embodiment, step 120 may specifically include: whenever a protocol parsing cycle ends, obtaining at least one protocol parsing state generated within the protocol parsing cycle; and determining the at least one protocol parsing state as the target protocol parsing state information for the protocol parsing cycle.

[0086] Specifically, for a complete session data set, the target program parses it field by field, parsing each field sequentially. During the parsing of each field, a corresponding protocol parsing state is generated. When the target program reads a segment of data, the testing tool marks that segment as tainted data for taint tracking, and the target program begins parsing that segment of data field by field. When a field of data is fully parsed, its corresponding protocol parsing state is reflected in the taint tracking results. When the parsing of a complete session data set is complete, the collected protocol parsing states for each field constitute the target protocol parsing state information for one protocol parsing cycle.

[0087] It should be noted that when the target program parses a complete session data field by field, a field may pass validation or fail validation. Specifically, if the field data conforms to the protocol specification (i.e., passes validation), the target program will continue parsing the next field after parsing that field. If the field data does not conform to the protocol specification (i.e., fails validation), the target program will treat the entire session data as non-protocol format data and will not perform any further parsing operations on the remaining fields of the session data. Therefore, the time interval of each protocol parsing cycle may be the same or different; the protocol parsing status information of each protocol parsing cycle may be the same or different.

[0088] Regarding the determination of the start point of a protocol parsing cycle, in one embodiment, before obtaining at least one protocol parsing state generated within the protocol parsing cycle whenever the end of a protocol parsing cycle is determined, the method further includes:

[0089] (1) Whenever the target program calls the specified reading function, the target read data read by the specified reading function is saved to a preset array.

[0090] Specifically, the target program reads the target test data segment by segment, meaning the target program reads the data segment by segment sequentially. The testing tool can save each instance of the target program reading the data to a preset array. In one example macro, the target program saves each instance of reading the data to the end of a defined array variable, Array.

[0091] (2) Obtain the target protocol parsing status obtained by the target program parsing the target read data.

[0092] Specifically, after the target program calls the specified read function, it parses the target read data and generates the target protocol parsing status of the target read data. Since the test tool monitors the target program in real time, it can obtain the target protocol parsing status.

[0093] (3) If the target protocol parsing state satisfies the specified protocol parsing state, the target protocol parsing state is determined as the initial protocol parsing state of the protocol parsing cycle.

[0094] (4) The moment when the target program calls the specified reading function to read the target data is determined as the starting point of the protocol parsing cycle.

[0095] Specifically, it can be determined whether the target protocol parsing state is the initial protocol parsing state by judging whether the assembly instruction information, operand content, operand taint status, and offset of the content verified by the instruction in the target read data are consistent with the specified protocol parsing state.

[0096] In other words, the target reads the first field of a complete session data. Accordingly, the moment when the target program calls the read function to read the target reads the data can be determined as the starting point of a protocol parsing cycle.

[0097] (5) If the target protocol parsing state does not satisfy the specified protocol parsing state, and after determining the initial protocol parsing state of the protocol parsing cycle, the target protocol parsing state is determined as another protocol parsing state of the protocol parsing cycle.

[0098] Specifically, if the assembly instruction information, operand content, operand taint status, and offset of the content verified by the instruction in the target read data of the target protocol parsing state are inconsistent with the specified protocol parsing state, it can be determined that the target protocol parsing state is not the initial protocol parsing state, but the protocol parsing state of other field data in a protocol parsing cycle.

[0099] In one embodiment, determining the end of a protocol parsing cycle includes: determining the end of the protocol parsing cycle when an initial protocol parsing state is obtained again or when the target program closes the socket.

[0100] Specifically, determining the end point of a protocol parsing cycle includes the following two cases:

[0101] (1) Determined by obtaining an initial protocol parsing state again

[0102] If the target private protocol does not close the socket after parsing a complete session of data, in this case, the current protocol parsing cycle has ended and the next protocol parsing cycle has begun by determining whether the initial protocol parsing state of the next protocol parsing cycle has been obtained.

[0103] (2) Determine by monitoring the target program closing the socket.

[0104] If the target private protocol closes the socket after parsing a complete session of data, then closing the socket can be used to determine the end of a protocol parsing cycle.

[0105] Therefore, a protocol parsing cycle can be accurately identified. After determining a protocol parsing cycle, the complete session data packets of that cycle can be obtained. Multiple target read data are extracted from a preset array, and all elements in the preset array are merged sequentially to obtain the complete session data.

[0106] It should be noted that if the end point of a protocol parsing cycle is determined by obtaining an initial protocol parsing state again, the target read data stored in the last element of the preset array is the first field data of the next protocol parsing cycle. Therefore, in this case, the last element needs to be removed and then merged to obtain the complete session data.

[0107] It should be understood that if a field in a protocol parsing cycle does not conform to the protocol specification, the target program will treat the entire session data as non-protocol format data and will not perform parsing operations on the remaining fields of the session data. Instead, it will merge the parsed field data stored in the preset array, which will be the session data packet for that protocol parsing cycle.

[0108] In one embodiment, step 130 may specifically include: for each protocol parsing cycle, obtaining the target protocol parsing status information, comparing the target protocol parsing status information with the protocol parsing status information in the protocol parsing status information set; when the target protocol parsing status information includes a new protocol parsing status, saving the target protocol parsing status information to the protocol parsing status information set, and obtaining the constraint content based on the new protocol parsing status.

[0109] Specifically, before performing fuzz testing on the target program, a protocol parsing state information set can be initialized to store the protocol parsing state information obtained during the testing process.

[0110] When the target program parses the target test data, it can obtain the target protocol parsing status information for each protocol parsing cycle. This target protocol parsing status information is then compared with the protocol parsing status information stored in the current protocol parsing status information set. If the target protocol parsing status information includes a new protocol parsing status, this new status information is saved to the protocol parsing status set. Furthermore, constraints are determined based on the new protocol parsing status, and the target test data is then mutated according to these constraints to obtain the mutated data.

[0111] As mentioned earlier, when the target program receives the target test data, it parses the data field by field according to the target's proprietary protocol specifications. The target's proprietary protocol specifications also define the length of a complete session data. The target program completes the parsing of a complete session data by parsing multiple fields (the aforementioned target read data). Within a protocol parsing cycle, the target program generates a protocol parsing state for each field it parses. If the field data conforms to the target's proprietary protocol format specifications, the target program will proceed to parse the next field data, generating a protocol parsing state. If the next field data does not conform to the target's proprietary protocol format specifications, it will not proceed to parsing the next field data, and no new protocol parsing state will be generated. It can be understood that whether a field data conforms to the target's proprietary protocol format specifications is determined by the target program's parsing of the field data and whether it passes the validation. Whether a field data conforms to the target's proprietary protocol format specifications determines whether to trigger the reading and parsing of the next field data.

[0112] For example, suppose a target private protocol receives a complete session data of 20 bytes and specifies that it parses the data field by field, in three parts: 4 bytes, 8 bytes, and 12 bytes. The target program first calls the specified read function to read and parse the 4-byte field, generating an initial protocol parsing state (hereinafter referred to as state 1). This field passes the validation. The target program then calls the specified read function again to read and parse the 8-byte field, generating a protocol parsing state (hereinafter referred to as state 2). This field fails the validation. Therefore, the target program considers the entire session data to be non-protocol format data and will not parse the next 12-byte field.

[0113] Thus, a protocol parsing cycle ends. The target protocol parsing state information obtained in this cycle includes state 1 and state 2. The target protocol parsing state information is then compared with the current protocol parsing state set.

[0114] If the protocol parsing state information stored in the current protocol parsing state set does not include states 1 and 2, then states 1 and 2 are new protocol parsing states. Based on the constraints of states 1 and 2, the target test data can be mutated to generate mutated data, and this target protocol parsing state information is saved to the protocol parsing state set. If the protocol parsing state information stored in the current protocol parsing state set includes state 1 but not state 2, then state 2 is a new protocol parsing state. Based on the constraints of state 2, the target test data can be mutated to generate mutated data, and this target protocol parsing state information is saved to the protocol parsing state set. If the protocol parsing state information stored in the current protocol parsing state set includes both states 1 and 2, then states 1 and 2 are not new protocol parsing states, and no processing is performed.

[0115] The proprietary protocol probe payload mining method of the present invention requires the target program to perform field-by-field verification on the test data. By tracking the field data read each time through taints, the target program generates a corresponding protocol parsing state when parsing and verifying each field data. Based on the protocol parsing state, it can be obtained which data positions the target program has verified for the currently sent test data, as well as the constraints used for verification.

[0116] In the fuzzing scenario of private protocols, there is usually a lack of effective initial test data. Therefore, it is necessary to construct random data for testing. During the cyclic testing process, the new protocol parsing state information generated in each round of testing will be used to mutate the test data. The multiple mutated data generated by the mutation will be used as test data for subsequent cyclic testing processes.

[0117] From the perspective of the target program, the protocol parsing state corresponds to the target program's verification process of the data (field data) read from the target. Each protocol parsing state corresponds to the verification of a single field. The target program verifies the test data field by field. If the target program passes the current field verification, it will proceed to the next field verification, and a new protocol parsing state will be generated accordingly; if the current field verification fails, no new protocol parsing state will be generated.

[0118] Determining whether new protocol parsing states have been generated during the current testing process allows us to determine whether the mutated test data can pass the target program's field validation. At the beginning of the test, randomly generated data often does not conform to the target's private protocol format specifications; therefore, only a limited amount of protocol parsing state information can be obtained in the initial testing phase. Later, test data generated through mutation may pass the target program's field validation, thus generating more protocol parsing state information, which guides the entire fuzzing process.

[0119] Based on the above logic, iterative testing of mutated data allows the fuzzing process to focus on the protocol parsing logic code of the target program, quickly generating data that can pass the target program's field validation logic. During iterative testing, when the test data sent by the testing tool passes all field validations of the target program, a response will be received from the target program. The data read and parsed by the target program in this protocol parsing cycle is then considered the "probe payload."

[0120] The following describes the proprietary protocol detection payload mining device provided by the present invention. The proprietary protocol detection payload mining device described below and the proprietary protocol detection payload mining method described above can be referred to in correspondence.

[0121] like Figure 3 As shown, the present invention provides a proprietary protocol-based probe payload mining device, comprising:

[0122] The sending module 310 is used to randomly select a piece of data from the test data set as the target test data and send the target test data to the target program; wherein, the target program is used to perform protocol parsing on the received data according to the target private protocol;

[0123] The acquisition module 320 is used to sequentially acquire the target protocol parsing status information obtained by the target program in each protocol parsing cycle of parsing the target test data;

[0124] The determining module 330 is used to determine the constraint content of the target program on the target test data based on the target protocol parsing status information;

[0125] The generation module 340 is used to mutate the target test data based on the constraint content to generate multiple mutated data;

[0126] The processing module 350 is used to add the mutation data to the test data set in order to mine the probe payload of the target private protocol based on the updated test data set.

[0127] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a method for mining probe payloads of a private protocol. This method includes: randomly selecting a data point from a test data set as target test data and sending the target test data to a target program; wherein the target program is used to perform protocol parsing on the received data according to a target private protocol; sequentially acquiring target protocol parsing status information obtained by the target program in each protocol parsing cycle of the target test data; determining the constraint content of the target program on the target test data based on the target protocol parsing status information; mutating the target test data based on the constraint content to generate multiple mutated data points; and adding the mutated data to the test data set to mine the probe payload of the target private protocol based on the updated test data set.

[0128] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0129] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the probe payload mining method for the private protocol provided by the above methods. The method includes: randomly selecting a piece of data from a test data set as target test data, and sending the target test data to a target program; wherein the target program is used to perform protocol parsing on the received data according to a target private protocol; sequentially obtaining target protocol parsing status information obtained by the target program in each protocol parsing cycle of parsing the target test data; determining the constraint content of the target program on the target test data based on the target protocol parsing status information; mutating the target test data based on the constraint content to generate multiple mutated data; and adding the mutated data to the test data set to mine the probe payload of the target private protocol based on the updated test data set.

[0130] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for mining probe payloads of the private protocols provided by the methods described above. This method includes: randomly selecting a data point from a test data set as target test data; sending the target test data to a target program; wherein the target program is used to perform protocol parsing on the received data according to a target private protocol; sequentially acquiring target protocol parsing status information obtained by the target program in each protocol parsing cycle of the target test data; determining the constraint content of the target program on the target test data based on the target protocol parsing status information; mutating the target test data based on the constraint content to generate multiple mutated data points; and adding the mutated data to the test data set to mine the probe payloads of the target private protocol based on the updated test data set.

[0131] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0132] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for mining probe payloads using a proprietary protocol, characterized in that, include: A single data point is randomly selected from the test dataset as the target test data, and the target test data is sent to the target program; wherein, the target program is used to perform protocol parsing on the received data according to the target private protocol; Sequentially obtain the target protocol parsing status information obtained in each protocol parsing cycle of the target program parsing the target test data; Determining the constraint content of the target program on the target test data based on the target protocol parsing status information includes: acquiring the target protocol parsing status information for each protocol parsing cycle, comparing the target protocol parsing status information with the protocol parsing status information in the protocol parsing status information set; when the target protocol parsing status information includes a new protocol parsing status, saving the target protocol parsing status information to the protocol parsing status information set, and acquiring the constraint content based on the new protocol parsing status; Based on the constraints, the target test data is mutated to generate multiple mutated data. The mutation data is added to the test data set to mine the probe payload of the target private protocol based on the updated test data set; The target protocol parsing status information includes: the assembly instruction information of the target program for parsing the target test data, the instruction operand content, the operand taint status, and the offset of the content verified by the instruction in the target test data; the constraint content includes constraint conditions and constraint positions.

2. The method for mining probe payloads using a proprietary protocol according to claim 1, characterized in that, The step of sequentially obtaining the target protocol parsing status information obtained in each protocol parsing cycle of the target program parsing the target test data includes: Whenever a protocol parsing cycle ends, at least one protocol parsing state generated within that protocol parsing cycle is obtained; The at least one protocol parsing state is determined as the target protocol parsing state information for the protocol parsing cycle.

3. The method for mining probe payloads using a proprietary protocol according to claim 2, characterized in that, Before obtaining at least one protocol resolution state generated within a protocol resolution cycle whenever the end of a protocol resolution cycle is determined, the method further includes: Whenever the target program calls the specified read function, the target read data read by the specified read function is saved to a preset array; wherein, the target read data is the data read by the target program in segments from the target test data; Obtain the target protocol parsing status obtained by the target program parsing the target read data; If the target protocol parsing state satisfies the specified protocol parsing state, the target protocol parsing state is determined as the initial protocol parsing state of the protocol parsing cycle. The moment when the target program calls the specified read function to read the target data is determined as the starting point of the protocol parsing cycle; If the target protocol parsing state does not satisfy the specified protocol parsing state, and after determining the initial protocol parsing state of the protocol parsing cycle, the target protocol parsing state is determined as another protocol parsing state of the protocol parsing cycle.

4. The method for mining probe payloads using a proprietary protocol according to claim 2 or 3, characterized in that, The process of determining the end of a protocol parsing cycle includes: When an initial protocol parsing state is obtained again or the target program closes the socket, the end point of the protocol parsing cycle is determined.

5. The method for mining probe payloads using a proprietary protocol according to claim 3, characterized in that, Before randomly selecting a data point from the test data set as the target test data and sending the target test data to the target program, the process includes: Instrument multiple I / O functions of the target program; Send a first random data to the target program; Based on the first I / O function called by the target program to read the first random data collected by the stake probe, the first I / O function is determined to be the specified reading function.

6. The method for mining probe payloads using a proprietary protocol according to claim 5, characterized in that, The IO functions include: IO functions for blocking I / O models, IO functions for non-blocking I / O models, IO functions for message-based WSAAsyncSelect models, IO functions for event-based WSAEventSelect models, IO functions for overlapped I / O models, and IO functions for completion port models.

7. The method for mining probe payloads using a proprietary protocol according to claim 5, characterized in that, Also includes: Based on the data collected by the insertion probe, the length of the first random data read by the first I / O function during the first call of the target program is the expected data length; Send a second random data with a length equal to the desired data length to the target program; The target program sequentially obtains multiple protocol parsing states obtained by parsing the second random data, and determines the first obtained protocol parsing state as the specified protocol parsing state.

8. A proprietary protocol-based detection payload excavation device, characterized in that, include: The sending module is used to randomly select a piece of data from the test data set as the target test data and send the target test data to the target program; wherein, the target program is used to perform protocol parsing on the received data according to the target private protocol; The acquisition module is used to sequentially acquire the target protocol parsing status information obtained by the target program in each protocol parsing cycle of parsing the target test data; The determination module is used to determine the constraint content of the target program on the target test data based on the target protocol parsing status information, including: for each protocol parsing cycle, the target protocol parsing status information is acquired, and the target protocol parsing status information is compared with the protocol parsing status information in the protocol parsing status information set; when the target protocol parsing status information includes a new protocol parsing status, the target protocol parsing status information is saved to the protocol parsing status information set, and the constraint content is obtained based on the new protocol parsing status; The generation module is used to mutate the target test data based on the constraint content to generate multiple mutated data; The processing module is used to add the mutation data to the test data set in order to mine the probe payload of the target private protocol based on the updated test data set. The target protocol parsing status information includes: the assembly instruction information of the target program for parsing the target test data, the instruction operand content, the operand taint status, and the offset of the content verified by the instruction in the target test data; the constraint content includes constraint conditions and constraint positions.

Images