A method and apparatus for fuzzing seed screening

Abstract

The application discloses a fuzzy test seed screening method and device. The method is based on multiple sets of fuzzy test seeds. Seed messages corresponding to the fuzzy test seeds are sent to a test object in sequence, and the change degree of each state feature of the test object when processing the seed messages is monitored. The change degree of the state of the test object when processing the messages can determine the complexity of the device under test when processing the messages or commands. The more complex the processing logic is, the more likely bugs exist, and the test should be focused on. Finally, the fuzzy test seeds are sorted according to the state influence degree of the seed messages on the test object, so as to determine the screening priority of each fuzzy test seed, so that the fuzzy test seeds with higher priority are screened preferentially to generate test cases, thereby improving the test efficiency.

Description

Technical Field

[0001] This application relates to the field of software testing technology, and in particular to a fuzz test seed selection method and apparatus. Background Technology

[0002] Fuzz testing is a popular method for discovering software vulnerabilities. It involves setting up specific test cases and performing tests according to those test cases.

[0003] Regarding the generation and creation of test cases, there are currently two common approaches: "generation-based" and "mutation-based." "Generation-based" involves modeling the protocol or file format under test and creating test cases accordingly. "Mutation-based" involves creating test cases by generating mutations in existing data samples.

[0004] "Generation-based" test case generation requires the design documents or technical specifications of the protocol under test to understand its syntax, semantics, and timing in detail. "Mutation-based" test cases, on the other hand, only require data samples of the protocol under test and can create a large number of test cases through mutation strategies such as bit flipping, overwriting, insertion, deletion, and concatenation. Since industrial control equipment widely uses proprietary and non-standard industrial control protocols, manufacturers typically do not publicly disclose protocol design documents for commercial and security reasons. In this scenario, the "mutation-based" test case generation method is more suitable.

[0005] The testing efficiency of "mutation-based" test case generation methods depends on how well data samples are selected, i.e., fuzz test seeds. However, existing "mutation-based" test case generation methods do not consider the state changes of the device under test when selecting fuzz test seeds. In other words, they treat all test seeds indiscriminately, resulting in the production of a large number of useless test cases, a low hit rate of abnormal results, and technical problems of low testing efficiency. Summary of the Invention

[0006] This application provides a fuzz test seed selection method and apparatus to solve the technical problem of low testing efficiency in existing test case generation methods.

[0007] To address the aforementioned technical problems, the first aspect of this application provides a fuzzy testing seed selection method, comprising:

[0008] Obtain multiple sets of candidate fuzz test seeds;

[0009] The seed messages corresponding to each of the fuzzy test seeds are sent to the test object in sequence, and the state characteristics of the test object before and after receiving each of the seed messages are monitored to obtain state characteristic change information.

[0010] Based on the state feature change information, determine the influence data of each fuzzy test seed on the test object;

[0011] Based on the ranking results of the impact data, the screening priority of each fuzzy test seed is determined.

[0012] Preferably, the test object is a host computer or a lower-level device that is communicatively connected to the host computer.

[0013] Preferably, the method for obtaining the candidate fuzz test seed includes:

[0014] The communication messages between the host computer and the lower device are monitored by means of communication message monitoring. The communication messages specifically include: downlink messages sent by the host computer to the lower device, and uplink messages sent by the lower device to the host computer.

[0015] The downlink messages are deduplicated to obtain a downlink fuzzy test seed set;

[0016] The uplink messages are deduplicated to obtain the uplink fuzzy test seed set.

[0017] Preferably, the status characteristics specifically include: network performance characteristics, device electrical characteristics, system resource characteristics, and video image characteristics.

[0018] Preferably, determining the screening priority of each fuzzy test seed based on the ranking results of the impact data specifically includes:

[0019] Based on the aforementioned impact data, the data are sorted according to each of the aforementioned fuzzy test seeds;

[0020] Based on the sorting results, the screening priority of each fuzzy test seed is determined in descending order of influence.

[0021] In addition, a second aspect of this application provides a fuzz test seed screening device, comprising:

[0022] The test seed acquisition unit is used to acquire multiple sets of fuzzy test seeds to be selected;

[0023] The state feature monitoring unit is used to sequentially send the seed messages corresponding to each of the fuzzy test seeds to the test object, and monitor the state features of the test object before and after receiving each of the seed messages, so as to obtain state feature change information.

[0024] The seed influence determination unit is used to determine the influence data of each fuzzy test seed on the test object based on the state feature change information;

[0025] The seed screening priority determination unit is used to determine the screening priority of each of the fuzzy test seeds based on the sorting results of the influence data.

[0026] Preferably, the test object is a host computer or a lower-level device that is communicatively connected to the host computer.

[0027] Preferably, it further includes:

[0028] The communication message monitoring unit is used to monitor the communication messages between the host computer and the lower device through a communication message monitoring method. The communication messages specifically include: downlink messages sent by the host computer to the lower device, and uplink messages sent by the lower device to the host computer.

[0029] The test seed generation unit is used to deduplicate the downlink messages to obtain a downlink fuzzy test seed set, and to deduplicate the uplink messages to obtain an uplink fuzzy test seed set.

[0030] Preferably, the status characteristics specifically include: network performance characteristics, device electrical characteristics, system resource characteristics, and video image characteristics.

[0031] Preferably, the seed screening priority determination unit is specifically used for:

[0032] Based on the aforementioned impact data, the data are sorted according to each of the aforementioned fuzzy test seeds;

[0033] Based on the sorting results, the screening priority of each fuzzy test seed is determined in descending order of influence.

[0034] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0035] This application is based on multiple sets of fuzzy test seeds. By sequentially sending seed messages corresponding to each fuzzy test seed to the test object and monitoring the degree of change in various states of the test object when processing the seed message, the complexity of the device under test (DUT) in processing the message or command can be determined by the strength of the state changes during message processing. The more complex the processing logic, the more likely it is to contain bugs and the greater the impact; therefore, it should be tested more extensively. Finally, these fuzzy test seeds are sorted according to the impact of the message on the state of the test object, thus determining the screening priority of each fuzzy test seed. This allows for the selection of higher-priority fuzzy test seeds for generating test cases, thereby improving testing efficiency. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart illustrating a fuzzy testing seed selection method provided in this application.

[0038] Figure 2 This is a schematic flowchart of another embodiment of a fuzzy testing seed selection method provided in this application.

[0039] Figure 3 This is a schematic diagram of a fuzzy test seed screening device provided in this application. Detailed Implementation

[0040] This application provides a fuzzy testing seed selection method and apparatus to solve the technical problem of low testing efficiency in existing test case generation methods.

[0041] To make the inventive objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] Please see Figure 1 The first embodiment of this application provides a fuzz test seed selection method, including:

[0043] Step 101: Obtain multiple sets of fuzz test seeds to be selected.

[0044] Step 102: Send the seed messages corresponding to each fuzzy test seed to the test object in sequence, and monitor the state characteristics of the test object before and after receiving each seed message to obtain information on changes in state characteristics.

[0045] Step 103: Based on the state feature change information, determine the impact data of each fuzzy test seed on the test object.

[0046] Step 104: Based on the ranking results of the impact data, determine the screening priority of each fuzzy test seed.

[0047] It should be noted that the fuzz test seed selection method provided in this application can be understood as follows: First, prepare multiple sets of candidate fuzz test seeds. Then, send the seed messages corresponding to these fuzz test seeds to the test object in sequence. By monitoring the changes in the state characteristics of the test object before and after receiving each seed message, and then determining the impact data of each fuzz test seed on the test object based on the state characteristic change information obtained from the above monitoring process, the selection priority of each fuzz test seed is determined. Finally, based on the ranking result of the impact data, the selection priority of each fuzz test seed is determined. After obtaining the selection priority of each fuzz test seed, the tester can selectively select more important fuzz test seeds and eliminate some less important fuzz test seeds according to the obtained selection priority information, thereby controlling the number of fuzz test seeds used for mutation and improving testing efficiency.

[0048] Understandably, since the seed messages are sent sequentially, it is only necessary to compare the sending time of each seed message to distinguish the state feature changes corresponding to different seed messages from the overall state feature change information.

[0049] The above content is a detailed description of the first embodiment of the fuzzy testing seed selection method provided in this application. This embodiment is based on multiple sets of fuzzy testing seeds. By sequentially sending seed messages corresponding to each fuzzy testing seed to the test object and monitoring the degree of change in various states of the test object when processing the seed message, the complexity of the device under test (DUT) processing the message or command can be determined by the strength of the state changes when the test object processes the message. The more complex the processing logic, the more likely it is to contain bugs, and therefore it should be tested more extensively. Finally, the fuzzy testing seeds corresponding to these seed messages are sorted according to the impact of the message on the state of the test object. This determines the selection priority of each fuzzy testing seed, allowing for the priority selection of higher-priority fuzzy testing seeds for generating test cases, thereby improving testing efficiency.

[0050] Based on the above-described basic embodiments, this application also provides a fuzzy test seed selection method, as follows:

[0051] Please see Figure 2 Based on the above embodiments, this embodiment provides a fuzzy testing seed selection method.

[0052] Furthermore, the test object is either the host computer or a lower-level device that communicates with the host computer.

[0053] Furthermore, the methods for obtaining candidate fuzz test seeds include:

[0054] Step 1001: Monitor the communication messages between the host computer and the lower device through the communication message monitoring method. The communication messages specifically include: downlink messages sent by the host computer to the lower device, and uplink messages sent by the lower device to the host computer.

[0055] Step 1002: Deduplicat downlink messages to obtain a downlink fuzzy test seed set; deduplicat uplink messages to obtain an uplink fuzzy test seed set.

[0056] Furthermore, the status characteristics specifically include: network performance characteristics, device electrical characteristics, system resource characteristics, and video image characteristics.

[0057] Furthermore, step 104 specifically includes:

[0058] Step 1041: Based on the impact data, sort the data according to each fuzzy test seed;

[0059] Step 1042: Based on the sorting results, determine the screening priority of each fuzzy test seed in descending order of influence.

[0060] It should be noted that, according to the user manual and operation manual of the host computer and device, all functions are operated sequentially on the host computer and device to trigger the corresponding instruction messages, and the downlink message D1 and uplink message U1 of the host computer and device are collected. The downlink message refers to the message sent by the host computer to the device; the uplink message refers to the message sent by the device to the host computer.

[0061] Deduplication is performed on both downlink and uplink messages to obtain the device test seed set D2 (i.e., the downlink fuzzy test seed set) and the host computer test seed set U2 (i.e., the uplink fuzzy test seed set). If the test object is only one side, only one of the downlink or uplink messages can be collected and deduplicated to obtain one of the downlink fuzzy test seed sets and the uplink fuzzy test seed set.

[0062] At this point, the impact of each test seed is still unknown. If all seeds are mutated directly, the efficiency of subsequent tests will be low.

[0063] Therefore, after obtaining the corresponding fuzzy test seeds, the test seed messages in the original test seed set D2 of the device are sequentially resent to the lower-level device. The changes in various state characteristics F of the lower-level device after receiving the message are monitored, including network performance characteristics (communication delay, communication jitter, etc.), equipment electrical characteristics (current jitter, voltage jitter, etc.), system resource characteristics (CPU utilization, memory utilization, etc.), and video image characteristics (changes in service signal lights, changes in monitoring screens, etc.), to obtain F. D .

[0064] F D ={F d |d∈D2}

[0065] F d ={Δf|Δf=|f'-f| / f,f∈F}

[0066] In the formula, D2 is the downlink fuzzy test seed set, d is the fuzzy test seed message in the downlink fuzzy test seed set, f represents the value of various state features of the tested object before receiving the message, and f' represents the value of various state features of the tested object after receiving the message.

[0067] Similarly, the original seed U2 of the host computer test is replayed sequentially, and the changes in various characteristics of the host computer before and after message transmission are monitored to obtain F. U .

[0068] F U ={F u |u∈U2}

[0069] F u ={Δf|Δf=|f'-f| / f,f∈F}

[0070] In the formula, U2 is the downlink fuzzy test seed set, and u is the fuzzy test seed message in the downlink fuzzy test seed set.

[0071] Next, based on the changes in various characteristics of the host computer and device before and after the original seed message is sent, the impact of the initial seed is calculated.

[0072] For the device test raw seed message D2, the calculation formula is as follows:

[0073]

[0074] The calculation formula for the original seed U2 tested by the host computer is as follows:

[0075] A U ={a u |a u =∑ f∈Fu Δf,u∈U2}

[0076] Based on the magnitude of the influence of various subsystems on the host computer and devices, A D and A U Sort D2 and U2 to obtain device test sorting seed D3 and host computer test sorting seed U3. Subsequent formal tests will mutate the seeds according to the sorting to generate a large number of test cases for testing. Among them, device test sorting seed D3 is mainly used for device testing, and host computer test sorting seed U3 is mainly used for host computer testing.

[0077] The above content is a detailed description of the second embodiment of the fuzzy testing seed screening method provided by this application. The following is a detailed description of an embodiment of the fuzzy testing seed screening device provided by this application.

[0078] Please see Figure 3 The third embodiment of this application provides a fuzz test seed screening device, comprising:

[0079] The test seed acquisition unit 201 is used to acquire multiple sets of candidate fuzzy test seeds;

[0080] The state feature monitoring unit 202 is used to send the seed messages corresponding to each fuzzy test seed to the test object in sequence, and monitor the state features of the test object before and after receiving each seed message, so as to obtain state feature change information.

[0081] The seed influence determination unit 203 is used to determine the influence data of each fuzzy test seed on the test object based on the state feature change information;

[0082] Seed screening priority determination unit 204 is used to determine the screening priority of each fuzzy test seed based on the ranking results of the impact data.

[0083] Furthermore, the test object is either the host computer or a lower-level device that communicates with the host computer.

[0084] Furthermore, it also includes:

[0085] The communication message monitoring unit 2001 is used to monitor the communication messages between the host computer and the lower device through the communication message monitoring method. The communication messages specifically include: downlink messages sent by the host computer to the lower device, and uplink messages sent by the lower device to the host computer.

[0086] The test seed generation unit 2002 is used to deduplicate downlink messages to obtain a downlink fuzzy test seed set, and to deduplicate uplink messages to obtain an uplink fuzzy test seed set.

[0087] Furthermore, the status characteristics specifically include: network performance characteristics, device electrical characteristics, system resource characteristics, and video image characteristics.

[0088] Furthermore, the seed screening priority determination unit 204 is specifically used for:

[0089] Based on the impact data, the data is sorted according to each fuzzy test seed;

[0090] Based on the sorting results, the screening priority of each fuzzy test seed is determined in descending order of influence.

[0091] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the terminals, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0092] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0093] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0094] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0095] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0096] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0097] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A fuzzy testing seed selection method, characterized in that, include: Obtain multiple sets of candidate fuzz test seeds; The seed messages corresponding to each of the fuzzy test seeds are sent to the test object in sequence, and the state characteristics of the test object before and after receiving each of the seed messages are monitored to obtain state characteristic change information. The state characteristic change information is used to reflect the degree of change of the state characteristics of the test object before and after receiving the seed messages. The state characteristics specifically include: network performance characteristics, device electrical characteristics, system resource characteristics, and video image characteristics. Based on the state feature change information, and combined with the correspondence between the degree of state feature change and the impact data, the impact data of each fuzzy test seed on the test object is determined; Based on the ranking results of the impact data, the screening priority of each fuzzy test seed is determined.

2. The fuzzy testing seed selection method according to claim 1, characterized in that, The test object is either a host computer or a lower-level device that is connected to the host computer for communication.

3. The fuzzy test seed selection method according to claim 2, characterized in that, The methods for obtaining the candidate fuzz test seeds include: The communication messages between the host computer and the lower device are monitored by means of communication message monitoring. The communication messages specifically include: downlink messages sent by the host computer to the lower device, and uplink messages sent by the lower device to the host computer. The downlink messages are deduplicated to obtain a downlink fuzzy test seed set; The uplink messages are deduplicated to obtain the uplink fuzzy test seed set.

4. The fuzzy testing seed selection method according to claim 1, characterized in that, The determination of the screening priority of each fuzzy test seed based on the ranking results of the impact data specifically includes: Based on the aforementioned impact data, the data are sorted according to each of the aforementioned fuzzy test seeds; Based on the sorting results, the screening priority of each fuzzy test seed is determined in descending order of influence.

5. A fuzzy test seed selection device, characterized in that, include: The test seed acquisition unit is used to acquire multiple sets of fuzzy test seeds to be selected; The state feature monitoring unit is used to sequentially send the seed messages corresponding to each of the fuzzy test seeds to the test object, and monitor the state features of the test object before and after receiving each of the seed messages, so as to obtain state feature change information. The state features specifically include: network performance features, device electrical features, system resource features and video image features. The seed influence determination unit is used to determine the influence data of each fuzzy test seed on the test object based on the state feature change information and the correspondence between the degree of state feature change and influence data. The seed screening priority determination unit is used to determine the screening priority of each of the fuzzy test seeds based on the sorting results of the influence data.

6. The fuzzy test seed screening device according to claim 5, characterized in that, The test object is either a host computer or a lower-level device that is connected to the host computer for communication.

7. The fuzzy test seed screening device according to claim 6, characterized in that, Also includes: The communication message monitoring unit is used to monitor the communication messages between the host computer and the lower device through the communication message monitoring method. The communication messages specifically include: downlink messages sent by the host computer to the lower device, and uplink messages sent by the lower device to the host computer. The test seed generation unit is used to deduplicate the downlink messages to obtain a downlink fuzzy test seed set, and to deduplicate the uplink messages to obtain an uplink fuzzy test seed set.

8. The fuzzy test seed screening device according to claim 5, characterized in that, The seed screening priority determination unit is specifically used for: Based on the aforementioned impact data, the data are sorted according to each of the aforementioned fuzzy test seeds; Based on the sorting results, the screening priority of each fuzzy test seed is determined in descending order of influence.

Images