Dynamic clustering fuzzy test method and system based on efficient byte exploration

Abstract

The invention belongs to the field of universal software fuzz testing, and discloses a dynamic clustering fuzz testing method and system based on efficient byte exploration. According to the method, fuzzy test seed bytes are subjected to clustering analysis by using a density-based spatial clustering algorithm, the positions of key bytes are identified, and the effective variation times of the key bytes are recorded. Then, corresponding weights are distributed to the bytes according to the clustering result and the number of effective variation times of the clustering result, an energy distribution strategy is dynamically adjusted in combination with the path coverage rate and path novelty information, and the exploration efficiency of a new path is improved. And performing mutation on the high-weight bytes to generate effective seeds of a new path, locking key byte positions of the effective seeds, performing normalization processing on other positions, and continuing mutation operation. The test strategy is continuously optimized through repeated execution of the steps, and the fuzzy test efficiency and the vulnerability discovery capability are effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of general software fuzz testing, and in particular relates to a dynamic clustering fuzz testing method and system based on efficient byte exploration. Background Art

[0002] As software complexity increases, hidden security vulnerabilities become difficult to detect. Traditional manual auditing and static analysis are inefficient and difficult to discover complex vulnerabilities. Fuzz testing can efficiently trigger abnormal behavior of programs under boundary conditions by inputting random or mutated data into the program and continuously observing the program's running results. Therefore, it plays an irreplaceable role in discovering unknown vulnerabilities and improving software security.

[0003] Chinese patent CN115292206A discloses a software vulnerability detection method, device, electronic device and storage medium. Its vulnerability detection method includes symbolic execution or random mutation. The target code is fuzz tested using the vulnerability detection method. The vulnerability detection method can be determined according to the path characteristics of the path, and random mutation and symbolic execution are combined. However, its fuzz testing method relies on random mutation, lacks precise control of the mutation position of key bytes, and is prone to generate a large number of redundant inputs, resulting in low test efficiency and hindering further mining of complex vulnerabilities. Summary of the invention

[0004] In order to overcome the shortcomings of the prior art, the present invention provides a dynamic clustering fuzzy testing method and system based on efficient byte exploration, which can identify the positions of key bytes through a clustering algorithm, dynamically adjust the mutation strategy, and improve software testing efficiency and vulnerability discovery capabilities.

[0005] The above object of the present invention is achieved through the following technical solutions:

[0006] A dynamic clustering fuzz testing method based on efficient byte exploration includes the following steps:

[0007] S1. By analyzing the mutation effect of the fuzzy test seed at different byte positions, combined with the density-based spatial clustering algorithm, preferably the DBSCAN algorithm, similar bytes are clustered and their effective mutation times are recorded;

[0008] S2. assigning corresponding weights according to the clustering information and the number of effective mutations obtained in step S1;

[0009] S3. Mutate the fuzzy test seed according to the weight information obtained in step S2, and use the path coverage and path novelty information to allocate energy to the seed to increase the probability of generating a new path.

[0010] S4. For the valid seeds of the new path generated in step S3, lock its key mutation positions, and normalize the remaining positions after excluding these key mutation positions. Combined with the byte weight information and energy distribution values ​​obtained in steps S2 and S3, continue to mutate other bytes to break through more complex or difficult to trigger code branches.

[0011] S5. When all the seeds in the fuzz test queue have been tested, reset the weight information obtained in step S2 to zero to ensure that the bytes that have been fully explored will not continue to occupy too high a weight; repeat steps S1 to S4, and continuously optimize and adjust the mutation strategy of the fuzz test to achieve more efficient software fuzz testing.

[0012] By clustering byte positions using a density-based spatial clustering algorithm, key byte positions can be identified. These positions have a greater impact on software behavior. Therefore, during the fuzz testing process, these key bytes can be mutated more specifically, reducing the number of invalid tests, thereby significantly improving the test efficiency. This method dynamically adjusts the mutation strategy, combines path coverage and path novelty information, gradually reduces the focus on the explored paths, and prioritizes the exploration of untouched code paths. By continuously iterating and optimizing the mutation strategy, it can gradually adapt to software changes and ensure the continued effectiveness of fuzz testing. As the test progresses, the weight of the bytes that have been fully explored will gradually return to zero, avoiding excessive attention to these bytes, thereby ensuring the reasonable allocation of test resources. By locking the key mutation positions and normalizing the remaining positions, fuzz testing can more effectively explore the code space of the software, especially those complex or difficult-to-trigger code branches, increasing the chances of discovering potential vulnerabilities and improving the security of the software.

[0013] Furthermore, the step 1 is specifically as follows:

[0014] S1.1. In units of bytes, select any unvisited data point as the current point, take the current point as the center, specify the density threshold MinPts, form a neighborhood with a radius of ε, and calculate the number of points in the neighborhood. If the number is greater than or equal to the specified minimum number of points MinPts, the current point is a "core point";

[0015] S1.2. If the current point is a "core point", all points in its neighborhood are classified into the same cluster and marked as visited;

[0016] S1.3. For the unvisited points in the neighborhood, continue to determine the core points and repeat steps S1.1 and S1.2 until all points are processed and assigned to different clusters.

[0017] By taking bytes as units and selecting density threshold MinPts and radius to form neighborhoods, this method can accurately identify which byte positions are critical, that is, those bytes that can significantly affect software behavior or trigger specific code paths, which helps to mutate more targeted in the subsequent fuzz testing process. The core point determination and neighborhood point classification process in steps S1.1 and S1.2 can efficiently cluster similar bytes together, improve the stability and targeting of the mutation process, and avoid position offsets and omissions of some key positions caused by irregular mutations.

[0018] Furthermore, the number of points in the neighborhood is the number of times the fuzz test seed mutates to generate a new path within the length of ε bytes before and after.

[0019] Furthermore, the number of effective mutations described in step S2 is the number of times the fuzzy test seed mutates at the corresponding cluster position to generate a new path.

[0020] Furthermore, the corresponding weight formula is assigned according to the clustering information and its effective mutation times as follows:

[0021]

[0022] Among them, w i ' is the cluster position variation weight; w i is the effective number of mutations for each cluster; ɑ is the smoothing term introduced to ensure that all byte positions are assigned a minimum weight; w j is the variation position weight, including the clustered position variation weight and the non-clustered position variation weight; n is the number of all variation positions.

[0023] Furthermore, the formula for allocating energy to seeds using the path coverage and path novelty information in step S3 is as follows:

[0024] E seed =W cov(t) ×C seed +W nov(t) ×N seed

[0025] Where: E seed Allocate energy to seeds; C seed The path value triggered by the current fuzz test seed; N seed is the novel path value triggered; W cov(t) The weight of the path coverage of the current fuzz test seed; W nov(t) A weight representing the novelty of a path.

[0026] Furthermore, the weight W of the path coverage of the current fuzz test seed is cov(t) The calculation formula is as follows:

[0027] W cov(t) =1-ɑ(t)

[0028] The weight W of the path novelty nov(t) The calculation formula is as follows:

[0029] W nov(t) =ɑ(t)

[0030] Where ɑ(t) represents the path novelty weight factor that changes over time, and it is adjusted using Sigmoid.

[0031] The present application also provides a dynamic clustering fuzzy testing system based on efficient byte exploration, using the method described, including:

[0032] A data processing module, used to receive and process fuzz test seeds, cluster similar bytes and record their effective mutation times;

[0033] A weight allocation module, connected to the data processing module, is used to allocate corresponding weights according to the clustering information and the number of effective mutations provided by the data processing module;

[0034] A mutation and energy allocation module, connected to the weight allocation module, is used to mutate the fuzz test seeds according to the weight information provided by the weight allocation module, and allocate energy to the seeds using the path coverage and path novelty information;

[0035] The key mutation position locking and normalization processing module is connected to the mutation and energy allocation module, and is used to lock the key mutation positions of the effective seeds for generating new paths, and normalize the remaining positions after excluding these key mutation positions, and continue to mutate other bytes according to the weight information and energy allocation values ​​obtained in the weight allocation module and the mutation and energy allocation module;

[0036] The loop control module is connected to the data processing module, the weight allocation module, the mutation and energy allocation module, and the key mutation position locking and normalization processing module, and is used to control the loop execution of the entire test process. That is, when all seeds in the fuzzy test queue have been detected, the weight information obtained in the weight allocation module is reset to zero, and the data processing module is triggered to restart the execution of steps S1 to S4 until the test termination condition is met.

[0037] The present application also provides an electronic device, including a memory, a processor, and a computer program stored and running on the memory, wherein the processor implements the dynamic clustering fuzzy testing method based on efficient byte exploration when executing the program.

[0038] The present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the dynamic clustering fuzzy testing method based on efficient byte exploration is implemented.

[0039] In summary, the beneficial effects of the present invention compared with the prior art are:

[0040] This application dynamically adjusts the mutation strategy through a clustering algorithm, and can focus on key bytes with greater potential to trigger new paths based on the mutation effects of different byte positions of the seed, thereby reducing redundant mutation operations and significantly improving test efficiency. This application dynamically allocates seed energy by combining path coverage and path novelty information, and performs more in-depth mutation operations on seeds with higher exploration value, ensuring that the potential of these seeds can be fully explored during the test process, thereby improving test coverage and increasing the probability of discovering new paths. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Flowchart of the dynamic clustering fuzz testing method based on efficient byte exploration. DETAILED DESCRIPTION

[0042] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0043] Example 1

[0044] A dynamic clustering fuzz testing method based on efficient byte exploration, such as Figure 1 The detailed implementation steps are as follows:

[0045] S1, by analyzing the mutation effect of the fuzz test seed at different byte positions, the byte positions are clustered using the density-based spatial clustering algorithm DBSCAN. Similar byte positions are clustered together, and the number of effective mutations at each position is recorded. This step is the basis for assigning weights to each byte position.

[0046] S2, according to the clustering information in step S1, assign corresponding weights to each byte position according to the number of effective mutations in each cluster. By introducing the smoothing term α, it is ensured that all byte positions can obtain a minimum weight to prevent the weight of some positions from being zero. At the same time, the seeds are mutated based on these weight information to improve the exploration of potential effective paths.

[0047] S3, allocates energy to seeds based on path coverage and path novelty information to increase the probability of discovering new paths.

[0048] S4, for the new path seed found in step S3, lock its key mutation positions. These locked byte positions will no longer participate in further mutation operations, while for the unlocked byte positions, continue to mutate according to the normalized weights to break through more complex or difficult to trigger paths.

[0049] S5, when all the seeds in the fuzz test queue have been tested, the weight information obtained in the S2 stage is reset to zero to ensure that the bytes that have been fully explored will not continue to occupy too high a weight, and then the program returns to S1 and repeats.

[0050] Example 2

[0051] A dynamic clustering fuzzy testing system based on efficient byte exploration, using the method described in Example 1, comprising:

[0052] A data processing module, used to receive and process fuzzy test seeds, and to perform step S1 to analyze the mutation effects of the fuzzy test seeds at different byte positions, and to cluster similar bytes and record their effective mutation times in combination with a density-based spatial clustering algorithm;

[0053] The weight allocation module is connected to the data processing module and is used to allocate corresponding weights according to the clustering information and the number of effective mutations provided by the data processing module. The corresponding weights are allocated according to the following formula:

[0054]

[0055] Where: w i ' is the cluster position variation weight; w i is the effective mutation number for each cluster, ɑ is the introduced smoothing term; w j is the variation position weight, including the clustered position variation weight and the non-clustered position variation weight; n is the number of all variation positions.

[0056] The mutation and energy allocation module is connected to the weight allocation module and is used to mutate the fuzz test seeds according to the weight information provided by the weight allocation module, and allocate energy to the seeds using the path coverage and path novelty information; wherein the energy allocation is performed according to the following formula:

[0057] E seed =W cov(t) ×C seed +W nov(t) ×N seed

[0058] Where: E seed Allocate energy to seeds; Cseed The path value triggered by the current fuzz test seed; N seed is the novel path value triggered; W cov(t) The weight of the path coverage of the current fuzz test seed; W nov(t) A weight representing the novelty of a path.

[0059] The weight of path coverage and the weight of path novelty are determined according to the following calculation formulas:

[0060] W cov(t) =1-ɑ(t)

[0061] W nov(t) =ɑ(t)

[0062] The key mutation position locking and normalization processing module is connected to the mutation and energy allocation module, and is used to lock the key mutation positions of the effective seeds for generating new paths, and normalize the remaining positions after excluding these key mutation positions, and continue to mutate other bytes according to the weight information and energy allocation values ​​obtained in the weight allocation module and the mutation and energy allocation module;

[0063] The loop control module is connected to the data processing module, the weight allocation module, the mutation and energy allocation module, and the key mutation position locking and normalization processing module, and is used to control the loop execution of the entire test process. That is, when all seeds in the fuzzy test queue have been detected, the weight information obtained in the weight allocation module is reset to zero, and the data processing module is triggered to restart the execution of steps S1 to S4 until the test termination condition is met.

[0064] An electronic device is also provided, comprising a memory, a processor and a computer program stored and running on the memory, wherein the processor implements the dynamic clustering fuzzy testing method based on efficient byte exploration when executing the program.

[0065] A computer-readable storage medium is also provided, on which a computer program is stored. When the program is executed by a processor, the dynamic clustering fuzzy testing method based on efficient byte exploration is implemented.

[0066] Since all the technical solutions of all the aforementioned embodiments are adopted when the fuzz test program is executed by the processor, it has at least all the beneficial effects brought by all the technical solutions of all the aforementioned embodiments, which will not be described one by one here.

[0067] The foregoing description of specific exemplary embodiments of the present invention is for the purpose of illustration and demonstration. These descriptions are not intended to limit the present invention to the precise form disclosed, and it is clear that many changes and variations can be made based on the above teachings. The purpose of selecting and describing the exemplary embodiments is to explain the specific principles of the present invention and its practical application, so that those skilled in the art can realize and utilize various different exemplary embodiments of the present invention and various different selections and changes. The scope of the present invention is intended to be limited by the claims and their equivalents.

Claims

1. A dynamic clustering fuzzy testing method based on efficient byte exploration, characterized in that the steps include: S1. By analyzing the mutation effect of the fuzz test seed at different byte positions, similar bytes are clustered and their effective mutation times are recorded in combination with the density-based spatial clustering algorithm; S2. assigning corresponding weights according to the clustering information and the number of effective mutations obtained in step S1; S3. Mutate the fuzzy test seed according to the weight information obtained in step S2, and use the path coverage and path novelty information to allocate energy to the seed to increase the probability of generating a new path; S4. For the effective seeds of the new path generated in step S3, the key mutation positions are locked, and the remaining positions are normalized after excluding these key mutation positions, and the other bytes are mutated according to the weight information and energy distribution values ​​obtained in steps S2 and S3; S5. When all seeds in the fuzzy test queue have been tested, the weight information obtained in step S2 is reset to zero, and steps S1 to S4 are repeated.

2. The dynamic clustering fuzzy testing method based on efficient byte exploration according to claim 1 is characterized in that: The step S1 is specifically as follows: S1.

1. In bytes, select any unvisited data point as the current point, take the current point as the center, specify the density threshold MinPts, form a neighborhood with a radius of ε, and calculate the number of points in the neighborhood. If the number is greater than or equal to the specified minimum number of points MinPts, the current point is a "core point"; S1.

2. If the current point is a "core point", all points in its neighborhood are classified into the same cluster and marked as visited; S1.

3. For the unvisited points in the neighborhood, continue to determine the core points and repeat steps S1.1 and S1.2 until all points are processed and assigned to different clusters.

3. The dynamic clustering fuzzy testing method based on efficient byte exploration according to claim 2 is characterized in that: The number of points in the neighborhood is the number of times the fuzz test seed mutates to generate a new path within the length of ε bytes before and after.

4. The dynamic clustering fuzzy testing method based on efficient byte exploration according to claim 1 or 2 is characterized in that: The effective mutation times described in step S2 is the number of times the fuzzy test seed mutates at the corresponding cluster position to generate a new path.

5. The dynamic clustering fuzzy testing method based on efficient byte exploration according to claim 4 is characterized in that: The corresponding weights are assigned according to the clustering information and its effective mutation times. The specific formula is as follows: Where: w i ' is the cluster position variation weight; w i is the effective mutation number for each cluster, ɑ is the introduced smoothing term; w j is the variation position weight, including the clustered position variation weight and the non-clustered position variation weight; n is the number of all variation positions.

6. The dynamic clustering fuzzy testing method based on efficient byte exploration according to claim 1 is characterized in that: Step S3 uses the path coverage and path novelty information to allocate energy to seeds. The specific formula is as follows: HAVE BEEN seed =W cov(t) ×C seed +W nov(t) ×N seed Where: E seed Allocate energy to seeds; C seed The path value triggered by the current fuzz test seed; N seed is the novel path value triggered; W cov(t) The weight of the path coverage of the current fuzz test seed; W nov(t) A weight representing the novelty of a path.

7. The dynamic clustering fuzzy testing method based on efficient byte exploration according to claim 6 is characterized in that: The weight W of the path coverage of the current fuzz test seed cov(t) The calculation formula is as follows: W cov(t) =1-ɑ(t) The weight W of the path novelty nov(t) The calculation formula is as follows: W nov(t) =ɑ(t) Where ɑ(t) represents the path novelty weight factor that changes over time, and it is adjusted using Sigmoid.

8. A dynamic clustering fuzzy testing system based on efficient byte exploration, using the method according to any one of claims 1 to 7, characterized in that: include: A data processing module, used to receive and process fuzz test seeds, cluster similar bytes and record their effective mutation times; A weight allocation module, connected to the data processing module, is used to allocate corresponding weights according to the clustering information and the number of effective mutations provided by the data processing module; A mutation and energy allocation module, connected to the weight allocation module, is used to mutate the fuzz test seeds according to the weight information provided by the weight allocation module, and allocate energy to the seeds using the path coverage and path novelty information; The key mutation position locking and normalization processing module is connected to the mutation and energy allocation module, and is used to lock the key mutation positions of the effective seeds for generating new paths, and normalize the remaining positions after excluding these key mutation positions, and continue to mutate other bytes according to the weight information and energy allocation values ​​obtained in the weight allocation module and the mutation and energy allocation module; The loop control module is connected to the data processing module, the weight allocation module, the mutation and energy allocation module, and the key mutation position locking and normalization processing module, and is used to control the loop execution of the entire test process. That is, when all seeds in the fuzzy test queue have been detected, the weight information obtained in the weight allocation module is reset to zero, and the data processing module is triggered to restart the execution of steps S1 to S4 until the test termination condition is met.

9. An electronic device comprising a memory, a processor and a computer program stored and running on the memory, characterized in that: When the processor executes the program, the dynamic clustering fuzzy testing method based on efficient byte exploration as described in any one of claims 1-7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, it implements the dynamic clustering fuzzy testing method based on efficient byte exploration as described in any one of claims 1 to 7.

Images