Coverage rate driven parallel fuzzy test task dynamic scheduling method and system

Abstract

The invention discloses a coverage rate driven parallel fuzzy test task dynamic scheduling method and a coverage rate driven parallel fuzzy test task dynamic scheduling system. The method comprises the step of adaptively adjusting the task allocation frequency according to the coverage rate increment monitored in real time. Through preset minimum and maximum period limitation, the period of the system is prolonged when coverage rate increase is lower than a threshold value, and resource consumption is reduced; and when the coverage rate increase exceeds a threshold value, the period is shortened, and new code area exploration is accelerated. The threshold value also supports dynamic adjustment, and real-time fine adjustment of the threshold value is realized through the effect of continuously monitoring coverage rate change and a task allocation period by a feedback loop so as to adapt to coverage difficulty of different stages. The dynamic adjustment strategy can effectively cope with the non-uniform coverage difficulty in the fuzzy test process, and it is ensured that the test process is efficient and continuously advanced. According to the design, the resource utilization efficiency and the test progress are optimized to adapt to coverage difficulty changes in different stages, and the efficiency and the effect of parallel fuzzy testing are also remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of software fuzz testing, and in particular to a coverage-driven parallel fuzz testing task dynamic scheduling method and system. Background Art

[0002] With the rapid development of Internet technology, digitalization has become the cornerstone of modern society. In today's fast-growing software industry, as the scale of systems continues to expand and the frequency of updates continues to rise, ensuring the stability and security of software has become the core goal of the development process. This requires that the efficiency of testing and vulnerability mining must be improved simultaneously to accurately mine potential vulnerabilities and ensure the safe operation of the software. The importance and widespread nature of software security are becoming increasingly prominent, because a vulnerability that is not discovered and repaired in time may be exploited by hackers, resulting in data leakage, system paralysis, or even more serious consequences. These incidents not only deal a huge blow to user trust, but may also trigger a series of legal, financial, and social issues.

[0003] Faced with the challenge of testing large and complex software systems, it is particularly important to explore more efficient testing methods and tools. Researchers have explored a variety of technical means to improve efficiency. Among them, fuzz testing detects potential program crashes and other problems by inputting abnormal, unexpected or random data into the software system. Due to its strong adaptability and low labor cost, fuzz testing has gradually been recognized by industry and academia. However, for large-scale and complex actual software systems, traditional single-machine single-core fuzz testing can no longer meet the growing testing needs. Therefore, parallel fuzz testing is proposed as a natural extension of traditional fuzz testing. With the development of parallel computing technology, fuzz testing methods based on parallelization have begun to gain favor. It improves test efficiency by running multiple fuzz test cases simultaneously. With the increase of computing resources, parallel fuzz testing can more easily expand the test scale to cope with the growing software complexity.

[0004] In the current research field of parallel fuzz testing, most of the work focuses on how to reasonably divide tasks, while relatively less attention is paid to dynamic task scheduling. These studies usually allocate test cases once at the beginning of parallel testing according to the characteristics of different tasks, or update tasks at fixed intervals. However, there is a problem with this approach: although the existing method performs well in task definition and allocation, it cannot adapt to the dynamic changes in the fuzz testing process. As the test deepens, the difficulty of exploration gradually increases, and at the same time, the exploration situation in different programs and different environments will also change. Therefore, further research is needed on how to achieve dynamic task scheduling to improve the efficiency and adaptability of parallel fuzz testing.

[0005] In view of the problem that existing parallel fuzz testing lacks an adaptive scheduling strategy for test tasks, the present invention provides a coverage-driven dynamic scheduling method and system for parallel fuzz testing tasks, which flexibly adjusts the frequency of task allocation based on real-time monitoring of coverage growth. By setting the upper and lower limits of the cycle, when the coverage growth is lower than the preset threshold, the system will extend the cycle to reduce resource consumption; on the contrary, when the coverage growth exceeds the threshold, the system will shorten the cycle to speed up the exploration of new code areas. The threshold itself also has the ability to be dynamically adjusted. Through the feedback mechanism, it continuously tracks the effects of coverage changes and task allocation cycles, and realizes real-time fine-tuning of the threshold to meet the needs of different exploration stages. This dynamic adjustment strategy effectively addresses the problem of uneven coverage difficulty in fuzz testing and ensures the continuity and efficiency of the test. By optimizing resource allocation and testing processes, this strategy not only adapts to the changes in coverage difficulty at different stages of the testing process, but also significantly improves the overall efficiency of parallel fuzz testing. Summary of the invention

[0006] The present invention provides a coverage-driven parallel fuzz testing task dynamic scheduling method and system, and the specific technical scheme is as follows:

[0007] In a first aspect of the present invention, a coverage-driven parallel fuzz testing task dynamic scheduling method is provided, comprising the following steps:

[0008] 1) Determine the initial value, minimum period and maximum period of the task allocation cycle, wherein the initial value is twice the minimum period;

[0009] 2) Based on the initial value and minimum period of the task allocation cycle, set the initial value of the threshold of the coverage increment;

[0010] 3) Allocate tasks to the software program to be tested, and perform parallel fuzz testing on the assigned tasks for a corresponding period of time according to the size of the current task allocation cycle. After the test is completed, record the current code coverage and calculate the coverage increment;

[0011] 4) Compare the coverage increment with the threshold, and adjust the task allocation cycle size in real time according to the comparison result;

[0012] 5) Adjust the threshold value in real time according to the coverage increment;

[0013] 6) Repeat steps 3) to 5) until the preset total test time is reached.

[0014] As a preferred solution of the present invention, before step 2), a fuzzy test running environment of the software program to be tested needs to be constructed, including multiple test engines.

[0015] As a preferred embodiment of the present invention, the step 2) is specifically as follows:

[0016] 2.1) Allocate the software program to be tested to multiple test engines, and the multiple test engines perform parallel fuzz testing on the assigned programs until the test duration reaches the minimum cycle, and record the first code coverage;

[0017] 2.2) The test engine continues to execute parallel fuzz testing until the initial value of the task allocation cycle is reached, ends the test, records the second code coverage, and calculates the coverage increment based on the first code coverage and the second code coverage as the initial value of the threshold.

[0018] As a preferred embodiment of the present invention, the step 4) is specifically as follows:

[0019] Compare the current coverage rate with the threshold. If the current coverage rate increment is lower than or equal to the threshold, extend the task allocation cycle by two times, but not exceeding the maximum cycle. If the current coverage rate increment is higher than the threshold, shorten the task allocation cycle by half, but not less than the minimum cycle.

[0020] As a preferred embodiment of the present invention, the step 5) is specifically as follows:

[0021] If the current coverage increment is lower than or equal to the threshold, the threshold is lowered; if the current coverage increment is higher than the threshold, the threshold is raised.

[0022] In a second aspect of the present invention, a coverage-driven parallel fuzz test task dynamic scheduling system is provided to implement the above-mentioned coverage-driven parallel fuzz test task dynamic scheduling method.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1) The present invention dynamically adjusts the task allocation cycle according to the coverage increment, which can make more efficient use of resources. When the coverage increases slowly, the parallel fuzzy testing system will reduce resource investment to avoid waste on inefficient paths; when the coverage increases rapidly, the system will increase resources to speed up the test, ensuring that resources are always concentrated on the most potential test paths;

[0025] 2) The present invention can adapt to changes in the test process more quickly by dynamically adjusting the threshold and task allocation cycle, significantly improving the efficiency of the test, especially in the case of non-uniform coverage difficulty, ensuring that the test process is both efficient and continuously advanced;

[0026] 3) The present invention can monitor coverage changes in real time and adaptively adjust the test strategy according to these changes. This high adaptability makes it particularly suitable for large-scale software projects that require fast iteration and high coverage guarantee, and can quickly respond to the task requirements of dynamic adjustment of parallel fuzz testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 Schematic diagram of the architecture of the coverage-driven parallel fuzz testing task dynamic scheduling system;

[0028] Figure 2 It is a schematic diagram of the workflow of the dynamic cycle adjustment module;

[0029] Figure 3 Schematic diagram of the workflow of the threshold fine-tuning module. DETAILED DESCRIPTION

[0030] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.

[0031] like Figure 1 As shown, a coverage-driven parallel fuzz test task dynamic scheduling system of the present invention consists of five parts: a real-time monitoring and analysis module, a dynamic cycle adjustment module, a threshold fine-tuning module, a task allocation module, and a parallel fuzz test execution module.

[0032] Among them, the real-time monitoring and analysis module continuously tracks the real-time changes of code coverage during the test process, provides necessary basic data for the adaptive adjustment of task allocation frequency, can accurately collect the growth trend of coverage, and provide accurate reference information for the dynamic adjustment of the system based on these data. The code coverage is the ratio of the number of basic blocks executed to the total number of basic blocks in the software program to be tested. Specifically, after each cycle in the parallel fuzzy testing process, the real-time monitoring and analysis module records the current code coverage and calculates the coverage increment (except the first cycle), and the coverage increment is the difference between the code coverage of the current cycle and the code coverage of the previous cycle. After the first cycle, since there is no previous cycle, the coverage increment is not calculated.

[0033] The dynamic cycle adjustment module ensures the rationality of the task allocation cycle by setting the minimum and maximum limits of the cycle, and adjusts the cycle length according to the real-time growth of the coverage rate. Directly adjust the allocation frequency of the test task to ensure the efficiency of the parallel fuzz testing system and the optimization of resource utilization, while speeding up the exploration of new code areas. In this embodiment, the minimum cycle is set to 10 minutes and the maximum cycle is set to 3 hours.

[0034] The threshold fine-tuning module is responsible for dynamically adjusting the threshold during the test process, constantly monitoring the changes in coverage and the effect of the task allocation cycle. The threshold is fine-tuned in real time based on the collected data, so that the parallel fuzz testing system can flexibly adapt to various test scenarios and ensure the stability and efficiency of the test process.

[0035] The task allocation module effectively allocates test tasks to each test engine based on the current test requirements and available resources. Through reasonable task allocation, it ensures that each test engine can receive appropriate test tasks according to its performance and characteristics, thereby optimizing the overall test efficiency.

[0036] In a specific embodiment of the present invention, the specific task allocation process of the task allocation module is as follows: all basic blocks of the software program to be tested are divided into a predecessor part containing only explored basic blocks and a successor part containing unexplored basic blocks, and the successor part is divided into multiple tasks and allocated to multiple test engines.

[0037] The subsequent part can be divided evenly based on the granularity of basic blocks, or other existing division methods can be used according to the performance and characteristics of each test engine to improve the test efficiency.

[0038] The parallel fuzz test execution module includes multiple test engines and is responsible for starting and managing multiple test engines to ensure that they effectively execute the assigned test tasks. Through the parallel execution mechanism, the test process is accelerated and the efficiency of discovering potential problems is improved.

[0039] In a specific embodiment of the present invention, the parallel execution mechanism of the parallel fuzz test execution module is specifically as follows: in each cycle, multiple test engines execute fuzzy tests of corresponding lengths on the multiple tasks in parallel according to the size of the current task allocation cycle; after each cycle, the number of executions of the subsequent basic blocks in this fuzzy test is counted, and the basic blocks that have been executed at least once are recorded as explored basic blocks.

[0040] In a specific embodiment of the present invention, the dynamic cycle adjustment module performs the following process: Figure 2 As shown in the figure, the dynamic cycle adjustment module ensures the rationality of the task allocation cycle through the preset minimum and maximum cycle limits, and makes intelligent adjustments based on the real-time incremental data of code coverage. When the increment is lower than or equal to the set threshold, it indicates that the current test speed is slow, and the system will automatically extend the task allocation cycle to twice the existing cycle, but not more than the maximum cycle value, so as to invest more resources in exploring new paths. If twice the existing cycle exceeds the maximum cycle value, the cycle is adjusted to the maximum cycle value. On the contrary, if the increment is higher than the threshold, it means that the test is quickly covering the new code area, and the system will shorten the cycle to half of the existing value, but not less than the minimum cycle value, to update the task to adapt to the new test requirements to speed up the overall test process of the system. If half of the existing value is lower than the minimum cycle value, the cycle is adjusted to the minimum cycle value. This adaptive adjustment mechanism directly acts on the allocation of test tasks, ensuring that the test can be carried out efficiently and stably under different circumstances.

[0041] In a specific embodiment of the present invention, the threshold fine-tuning module performs the following process: Figure 3 As shown in the figure, the threshold fine-tuning module realizes dynamic adjustment of the threshold by continuously monitoring the coverage change and the effect of the task allocation cycle. When the calculated coverage increment is higher than the set threshold, the module adjusts the threshold to 110% of the existing threshold, reducing resource consumption and avoiding frequent adjustment of the task allocation cycle. On the contrary, if the coverage increment does not exceed the threshold, the module adjusts the threshold to 90% of the existing threshold, making the system more sensitive to coverage changes, thereby triggering the adjustment of the task allocation cycle earlier to explore more potential test paths. This adaptive adjustment mechanism enables the system to flexibly respond to various test situations and maintain the stability and efficiency of the test.

[0042] Based on the above introduction to the execution process of the two main modules, each main module is composed of several sub-modules, and the dynamic period adjustment module includes:

[0043] The cycle limit maintenance component implements and monitors the preset minimum and maximum cycle limits to ensure that all task allocation cycles fall within this safe range. By maintaining these cycle boundaries, the component prevents cycle changes that are too short or too long, thereby protecting the system from unstable cycle changes. This helps maintain the consistency and reliability of the test process and enables the system to operate stably in a dynamic environment;

[0044] Coverage growth analysis component, used to analyze coverage growth data to identify growth trends. These analysis results provide a solid basis for dynamic cycle adjustment, enabling the system to respond more accurately to changes in test requirements. This component ensures the flexibility and adaptability of the test process, which can be adjusted according to actual test conditions;

[0045] The cycle length adjustment component dynamically adjusts the length of the task allocation cycle based on the data provided by the coverage growth analysis component. If the analysis results show that the coverage is growing slowly, the component will extend the cycle length to reduce resource consumption and optimize efficiency; on the contrary, if the coverage is growing rapidly, it will shorten the cycle length to accelerate the exploration of new code areas. This flexible adjustment mechanism optimizes resource usage and improves test efficiency, ensuring that parallel fuzz testing can proceed efficiently.

[0046] The threshold fine-tuning module includes:

[0047] The threshold trend analysis component is responsible for receiving and deeply analyzing coverage data to identify growth trends and patterns. Through accurate analysis of these data, this component provides a solid basis for dynamic adjustment of thresholds, ensuring that the adjustment process is both accurate and efficient;

[0048] The threshold adjustment component is responsible for dynamically adjusting the threshold based on the analysis results provided by the threshold trend analysis component. If the coverage increment is higher than the set threshold, it will adjust the threshold upward by a specific proportion to reduce the frequency of periodic updates and maintain a stable test speed; on the contrary, if the coverage increment is lower than or equal to the threshold, it will adjust the threshold downward by a specific proportion to increase the system's sensitivity to coverage changes. This flexible adjustment mechanism enables the system to respond more accurately to actual test needs and ensure test efficiency.

[0049] The system embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Those of ordinary skill in the art may understand and implement the scheme without creative work.

[0050] The system embodiment of the present invention can be applied to any device with data processing capability, and the device with data processing capability can be a device or apparatus such as a computer. The system embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the corresponding computer program instructions in the non-volatile memory are read into the memory by the processor of any device with data processing capability and run.

[0051] Based on the same inventive concept, a coverage-driven parallel fuzz testing task dynamic scheduling method is proposed, and the main steps are:

[0052] 1) Construct a fuzzy test running environment for the software program to be tested, including multiple test engines; determine the minimum and maximum limits of the task allocation cycle to define the adjustment range. Set the initial cycle to twice the minimum value so that it can be adjusted later according to actual conditions; the test engine is a fuzzy test tool, and the test engine used in this embodiment is AFL++.

[0053] 2) Considering the differences in test efficiency under different programs and environments, it is not appropriate to use a fixed initial threshold. The initial threshold of coverage increment is set through preliminary testing as a benchmark for measuring test efficiency and providing guidance for subsequent task allocation cycle adjustments. Parallel fuzz testing is carried out according to the initial cycle, and the current coverage is recorded at the end of the minimum cycle, and recorded again after the end of the initial cycle. The increment of these two coverages is calculated as the threshold.

[0054] 3) All basic blocks of the software program to be tested are divided into a predecessor part containing only explored basic blocks and a successor part containing unexplored basic blocks. The successor part is divided into multiple tasks and assigned to multiple test engines. The multiple test engines perform fuzzy tests of the multiple tasks in parallel for a corresponding period of time according to the size of the current task allocation cycle; the number of executions of the basic blocks of the successor part in this fuzzy test is counted, and the basic blocks that have been executed at least once are recorded as explored basic blocks. At the end of each cycle, the coverage rate and coverage rate increment are updated.

[0055] The unexplored basic block refers to a basic block that has not been executed in the test of the current cycle and the historical cycle. The predecessor part is located before the execution order of the successor part, and the predecessor part and the successor part are multiple basic blocks that are executed continuously. The division method of dividing the successor part into multiple tasks and allocating them to multiple test engines is to divide evenly with basic blocks as the granularity.

[0056] 4) Based on the calculation results of the coverage increment, the task cycle is adaptively adjusted. When the increment is lower than the threshold, the cycle is extended to trigger more test path exploration; when the increment exceeds the threshold, the cycle is shortened to speed up the coverage of new code areas. The specific adjustment strategy is: if the increment is lower than the threshold, the cycle is adjusted to twice the current length, but not exceeding the set maximum limit; if the increment exceeds the threshold, the cycle is halved, but not lower than the minimum set value.

[0057] 5) Monitor the changes in coverage and the effects of the task cycle, and dynamically adjust the threshold accordingly. When the coverage increment is higher than the set threshold, it means that the current system test efficiency is better than expected, the exploration difficulty of the current exploration area is low, and the coverage rate is growing faster. The system will raise the threshold to 110% of the current threshold to reduce the sensitivity to coverage changes, thereby delaying the time point of cycle adjustment and avoiding frequent adjustment of the cycle; on the contrary, if the increment is lower than the threshold, it means that the current system coverage rate is growing slowly, the exploration difficulty of the current test area is high, and a slight increase in coverage may have a significant impact on the overall program testing and vulnerability mining. The threshold will be lowered to 90% of the existing threshold to increase the sensitivity to coverage changes and start cycle adjustment earlier.

[0058] 6) Repeat steps 3) to 5), and continue to drive the optimization task cycle as the coverage changes to ensure effective resource utilization and in-depth exploration of the test path until the total fuzz testing time reaches the expected time.

[0059] Experiments show that compared with other parallel fuzz testing tools, the parallel fuzz tester after applying the coverage-driven parallel fuzz testing task dynamic scheduling method and system proposed in the present invention has higher testing efficiency. Under the same experimental settings, compared with the parallel mode of the fuzz testing tool AFL++, the present invention has achieved significant improvements in common fuzz testing performance indicators such as edge coverage and the number of vulnerabilities discovered. Among them, compared with the most widely used tool AFL++, for nine commonly used programs of various types, the present invention has improved the program edge coverage index by an average of 18.45%, and the number of vulnerabilities discovered has increased by 18.75%.

[0060] The above-mentioned embodiments only express several implementation methods of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. For ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A coverage-driven parallel fuzz testing task dynamic scheduling method, characterized in that: The following steps are involved: 1) Determine the initial value, minimum period and maximum period of the task allocation cycle, wherein the initial value is twice the minimum period; 2) Based on the initial value and minimum period of the task allocation cycle, set the initial value of the threshold of the coverage increment; 3) Assign tasks to the software program to be tested, and perform parallel fuzz testing of the assigned tasks for a corresponding period of time according to the size of the current task allocation cycle. After the test, record the current code coverage and calculate the coverage increment; 4) Compare the coverage increment with the threshold, and adjust the task allocation cycle size in real time according to the comparison result; 5) Adjust the threshold value in real time according to the coverage increment; 6) Repeat steps 3) to 5) until the preset total test time is reached.

2. The coverage-driven parallel fuzz testing task dynamic scheduling method according to claim 1 is characterized in that: The code coverage is the ratio of the number of executed basic blocks to the number of all basic blocks in the software program to be tested.

3. The coverage-driven parallel fuzz testing task dynamic scheduling method according to claim 1 is characterized in that: Before step 2), it is also necessary to build a fuzzy test running environment for the software program to be tested, including multiple test engines.

4. The coverage-driven parallel fuzz testing task dynamic scheduling method according to claim 3 is characterized in that: The step 2) is specifically as follows: 2.1) Allocate the software program to be tested to multiple test engines, and the multiple test engines perform parallel fuzz testing on the assigned programs until the test duration reaches the minimum cycle, and record the first code coverage; 2.2) The test engine continues to execute parallel fuzz testing until the initial value of the task allocation cycle is reached, ends the test, records the second code coverage, and calculates the coverage increment based on the first code coverage and the second code coverage as the initial value of the threshold.

5. The coverage-driven parallel fuzz testing task dynamic scheduling method according to claim 3 is characterized in that: In step 3), tasks are assigned to the software program to be tested, and parallel fuzz testing of a corresponding length of time is performed on the assigned tasks according to the size of the current task allocation cycle. Specifically, all basic blocks of the software program to be tested are divided into a predecessor part containing only explored basic blocks and a successor part containing unexplored basic blocks, the successor part is divided into multiple tasks and assigned to multiple test engines, and the multiple test engines perform fuzzy testing of a corresponding length of time on the multiple tasks in parallel according to the size of the current task allocation cycle; the number of executions of the basic blocks of the successor part in this fuzzy test is counted, and the basic blocks that have been executed at least once are recorded as explored basic blocks.

6. The coverage-driven parallel fuzz testing task dynamic scheduling method according to claim 5 is characterized in that: The division method of dividing the subsequent part into multiple tasks and allocating them to multiple test engines is: evenly dividing with basic blocks as the granularity.

7. The coverage-driven parallel fuzz testing task dynamic scheduling method according to claim 1 is characterized in that: The coverage increment in step 3) is the difference between the code coverage of the current cycle and the code coverage of the previous cycle.

8. The coverage-driven parallel fuzz testing task dynamic scheduling method according to claim 1, characterized in that: The step 4) is specifically as follows: Compare the current coverage with the threshold, if the current coverage increment is lower than or equal to the threshold, extend the task allocation cycle to twice, but not exceeding the maximum cycle; If the current coverage increment is higher than the threshold, the task allocation period is shortened to half, but not less than the minimum period.

9. The coverage-driven parallel fuzz testing task dynamic scheduling method according to claim 1, characterized in that: The step 5) is specifically as follows: If the current coverage increment is lower than or equal to the threshold, the threshold is lowered; if the current coverage increment is higher than the threshold, the threshold is raised.

10. A scheduling system for implementing the coverage-driven parallel fuzz testing task dynamic scheduling method according to claim 1, characterized in that the system include: Dynamic cycle adjustment module, used to set the minimum cycle, maximum cycle and initial cycle, and also used to continuously adjust the task allocation cycle according to the current coverage increment; Real-time monitoring and analysis module, used to record code coverage during parallel fuzz testing and calculate coverage increment; The threshold fine-tuning module is used to set the initial value of the threshold and also to continuously adjust the threshold size according to the current coverage increment; A task allocation module, used for allocating tasks to the software program to be tested; The parallel fuzz testing execution module is used to carry out parallel fuzz testing according to the assigned tasks.

Images