A fuzzy test driver generation system and method based on unit testing

Abstract

The present invention relates to the field of electronic digital data processing, and discloses a fuzzy test driver generation system and method based on unit testing. The system includes a parsing module, an initial driver generation module, and an optimization module; the parsing module includes a target construction unit and a unit test parsing unit, which are used to automatically build a target Java project and extract a unit test API sequence; the initial driver generation module includes a driver generation unit and an execution unit, which are used to generate and execute an initial fuzzy test driver using a large language model; the optimization module includes an iterative optimization unit and an evaluation unit, which are used to iteratively optimize the fuzzy test driver according to coverage information. The system and method disclosed by the present invention can improve the efficiency and coverage of software testing, provide a new solution for quality assurance in the software development process, and help improve the overall quality and reliability of the software.

Description

Technical Field

[0001] The present invention relates to the field of electronic digital data processing, and in particular to a fuzzy test drive generation system and method based on unit testing. Background Art

[0002] Fuzz testing is one of the most popular software testing technologies used to discover security and reliability issues in software. The core idea is to pass a large amount of random data to a given application and monitor whether abnormal behavior occurs. Since its introduction, fuzz testing technology has discovered countless vulnerabilities and made great contributions to improving application security. OSS-Fuzz is an open source fuzz testing service that currently manages fuzz testing infrastructure for more than 1,000 widely used open source projects and has reported tens of thousands of security and reliability vulnerabilities in these projects.

[0003] When fuzz testing API functions, testers need to analyze the functions and call dependencies of the API to be tested, use the correct API sequence, and manually write a fuzz test driver that can be directly executed to pass test data to the API to fully explore the program status. However, due to the diversity of APIs, this process takes a lot of time, and the quality of the written fuzz driver depends largely on the experience of the tester, which has a high labor cost.

[0004] To alleviate this burden, existing solutions extract API dependencies used in consumer code (i.e., actual use of the API), summarize and infer valid API sequences, and expect to automatically generate fuzzy drivers. Although this method can achieve the goal of reducing manual input, it is still difficult to generate valid API call parameters and high-quality API sequences, and does not support Java programs.

[0005] A high-quality fuzz testing driver first needs to properly allocate test data according to the semantic relationship of API parameters. However, traditional static analysis methods are difficult to accurately reflect the relationship between API parameters, resulting in false crashes and ultimately reducing the test effect. Secondly, the current automatic generation of fuzzy drivers mainly relies on consumer code. When encountering a large number of consumer codes with API calls scattered in complex control flows, the extracted API dependencies may become bloated, causing the driver to call dozens or even hundreds of APIs. The bloated input space will also affect the fuzz testing effect. In addition, for niche APIs to be tested, there may not be enough consumer code for analysis. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a fuzzy test driver generation system and method based on unit testing, so as to achieve the purpose of generating a semantically correct fuzzy test driver program more efficiently.

[0007] To achieve the above object, the technical solution of the present invention is as follows:

[0008] A fuzzy test driver generation system based on unit testing, including a parsing module, an initial driver generation module and an optimization module;

[0009] The parsing module includes a target construction unit and a unit test parsing unit, which are used to automatically build a target Java project and extract a unit test API sequence;

[0010] The initial driver generation module includes a driver generation unit and an execution unit, which are used to generate an initial fuzz test driver using a large language model and execute it;

[0011] The optimization module includes an iterative optimization unit and an evaluation unit, which are used to iteratively optimize the fuzzy test driver according to coverage information.

[0012] In the above scheme, the target construction unit automatically analyzes the Java project to support subsequent fuzz test driver generation, and automatically detects the build system used by the project by traversing the file structure of the target project repository. When configuration files of multiple build systems are detected, the target construction unit selects the most suitable build system according to the preset priority order.

[0013] In the above scheme, the unit test parsing unit performs static analysis on the Java project through the Soot framework to identify all test methods annotated with @Test; during the analysis process, a context-sensitive call graph is constructed, the k-CFA algorithm is adopted, and a balance is achieved between analysis accuracy and computing performance by adjusting the k value.

[0014] In the above scheme, the unit test parsing unit performs a comprehensive API call sequence analysis on each screened test method, and analyzes in detail the parameter type, return value type and related exception handling strategy of each API call.

[0015] In the above scheme, the driver generation unit provides the processed analysis results as input to the large language model, which is used for the large language model to generate an initial fuzzy test driver.

[0016] In the above scheme, the execution unit uses Airflow as a scheduling and monitoring platform through a parallel executor, and uses a fork mechanism to quickly create and manage the fuzz testing process.

[0017] In the above solution, the iterative optimization unit optimizes the generated result, specifically including:

[0018] Driver classification: Based on the coverage data, the improved k-means++ algorithm is used to classify the fuzz test drivers;

[0019] Merge drivers of the same region: For each type of driver, merge drivers covering the same or similar regions;

[0020] Merge drivers of different regions: Based on the merging of drivers of the same region, merge and optimize drivers across categories to improve the overall test coverage.

[0021] In the above scheme, the evaluation unit uses an iterative optimization process to determine whether the coverage has converged, so as to avoid terminating the iteration process too early or too late due to short-term fluctuations; the large model optimizes the test drive according to the coverage data of each round of execution and the abnormal conditions found; the optimization focuses on three aspects: increasing the testing of low coverage areas, deepening the exploration of abnormal triggering paths, and optimizing parameter generation strategies.

[0022] A fuzzy test driver generation method based on unit testing, using the fuzzy test driver generation system based on unit testing as described above, comprises the following steps:

[0023] Step 1, the target construction unit in the parsing module automatically analyzes the Java project to support the subsequent fuzz test driver generation, and automatically detects the construction system used by the project by traversing the file structure of the target project warehouse. When the configuration files of multiple construction systems are detected, the target construction unit selects the most suitable construction system according to the preset priority order;

[0024] Step 2, the unit test parsing unit in the parsing module performs static analysis on the Java project through the Soot framework to identify all test methods annotated with @Test; during the analysis process, a context-sensitive call graph is constructed, the k-CFA algorithm is used, and the k value is adjusted to achieve a balance between analysis accuracy and computing performance;

[0025] Step 3, the driver generation unit in the initial driver generation module provides the processed analysis result as input to the large language model, so that the large language model generates an initial fuzzy test driver;

[0026] Step 4: The execution unit in the initial driver generation module uses Airflow as a scheduling and monitoring platform, and uses a fork mechanism to quickly create and manage the fuzz testing process;

[0027] Step 5, the iterative optimization unit in the optimization module optimizes the generated result;

[0028] Step 6: The evaluation unit in the optimization module uses an iterative optimization process to determine whether the coverage has converged to avoid premature or late termination of the iteration process due to short-term fluctuations; the large model optimizes the test drive based on the coverage data of each round of execution and the abnormal conditions found.

[0029] In the above scheme, the algorithm for determining coverage convergence is as follows:

[0030] (1) Define the sliding window size w and 5 iterations;

[0031] (2) Calculate the average coverage growth rate of the last w iterations:

[0032] Growth rate = (current coverage rate - coverage rate before round w) / coverage rate before round w;

[0033] (3) Set the convergence threshold ε to 0.001 and the minimum number of iterations N to 30 rounds;

[0034] (4) If the growth rate is less than ε for three consecutive rounds and the total number of iterations is greater than or equal to N, convergence is achieved.

[0035] Through the above technical solution, the fuzzy test drive generation system and method based on unit testing provided by the present invention have the following beneficial effects:

[0036] 1. Improve test efficiency and coverage: This invention combines unit test mode and large language model to generate more comprehensive fuzzy test drivers, especially improving the identification of edge cases and abnormal paths. The execution unit and optimization module are used to improve execution efficiency, and the iterative optimization process continuously improves coverage and effectively explores potential software problems;

[0037] 2. Enhanced adaptability and compatibility: The present invention implements adaptive JDK version selection and multi-build system support, introduces a method filtering mechanism, and adapts to Java projects of different scales and complexities. This flexibility improves the versatility and practicality of the test method;

[0038] 3. Improve the degree of intelligence and automation: The present invention makes full use of large language models and introduces intelligent elements in the process of driver generation, optimization and merging. Automated parallel execution, result analysis and iterative optimization reduce manual intervention and provide more reliable and efficient testing solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0040] Figure 1A schematic diagram of a fuzzy test driver generation system based on unit testing disclosed in an embodiment of the present invention.

[0041] Figure 2 A schematic flow chart of a fuzzy test driver generation method based on unit testing disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0043] The method of the present invention utilizes the API usage patterns and test logic in existing unit tests, combined with the capabilities of large language models, to generate a more comprehensive fuzz test driver. This method aims to improve the efficiency and coverage of software testing, especially in identifying boundary conditions and abnormal paths. By introducing parallel processing technology, specific prompt templates and feedback mechanisms, this method optimizes the efficiency and effectiveness of fuzz testing. The application of these technologies enables the generated test cases to better adapt to project characteristics and requirements. The main goal of this method is to improve the quality and efficiency of the software testing process through automation and intelligent means, and to provide developers with a new tool to explore potential problems and vulnerabilities in software in a more systematic and comprehensive way.

[0044] The present invention provides a fuzzy test drive generation system based on unit testing, such as Figure 1 As shown, it includes a parsing module, an initial drive generation module and an optimization module.

[0045] 1. Parsing module

[0046] The parsing module includes a target building unit and a unit test parsing unit, which are used to automatically build the target Java project and extract the unit test API sequence.

[0047] 1. Target building unit

[0048] The target build unit automatically analyzes Java projects to support subsequent fuzz test-driven generation. It can adapt to a variety of build systems and Java development environments and has high flexibility and compatibility. The unit supports a variety of mainstream Java project build methods, including Maven, Gradle, and Ant. By traversing the file structure of the target project repository, the build system used by the project is automatically detected. Specifically, the target build unit searches for specific build configuration files, such as Maven's pom.xml, Gradle's build.gradle or build.gradle.kts, and Ant's build.xml. When multiple build system configuration files are detected, the target build unit selects the most suitable build system according to the preset priority order (Maven > Gradle > Ant).

[0049] Secondly, the target build unit also implements an adaptive JDK version selection mechanism. This mechanism can automatically try different versions of OpenJDK (including but not limited to OpenJDK-15, OpenJDK-17, OpenJDK-11 and OpenJDK-8) until a version that can successfully build the project is found. This method greatly improves the compatibility of the present invention with different Java projects. In addition, in order to ensure that the fuzzy test driver generated subsequently can correctly access all necessary classes, it is necessary to traverse the project folder after the build, find the class files generated by the build, and package these class files into a separate jar file.

[0050] 2. Unit test parsing unit

[0051] The unit test parsing unit uses the Soot framework to perform static analysis on Java projects and identify all test methods annotated with @Test. In order to efficiently capture the API call sequence, the unit uses Soot's Shimple intermediate representation, which is a high-level representation method based on the static single assignment (SSA) form. Shimple technology simplifies the data flow analysis process, making the definition and usage relationship of variables clearer, so that data dependencies between API calls can be accurately tracked. During the analysis process, this unit constructs an accurate context-sensitive call graph. By adopting the k-CFA (k-Control Flow Analysis) algorithm and adjusting the k value, a balance is achieved between analysis accuracy and computational performance. This method can more accurately capture the context information of API calls and help understand the usage pattern of APIs.

[0052] In order to improve the efficiency of analysis and the effectiveness of testing, a screening mechanism is needed to reduce the candidate set of target methods and focus on those methods that have a significant impact on system behavior and are of moderate complexity, so as to improve the efficiency and accuracy of subsequent analysis and testing. Specifically, it includes:

[0053] 1. Method call frequency filtering: By analyzing the method calls of the entire project, those methods with extremely low usage frequencies are ignored because they are unlikely to have a significant impact on the normal operation of the system. Use Soot's CallGraph API to build the method call graph of the entire project, then traverse the CallGraph and calculate the in-degree of each SootMethod, that is, the number of times it has been called. The filtering logic is as follows: First, calculate the threshold, which is based on the overall method call distribution of the project and is usually set to 10% of the average call frequency. Then traverse all methods in the scene, and for each method, if the number of times it has been called is less than the threshold, remove the method from the candidate method list.

[0054] 2. Method complexity filtering: Evaluate the complexity of methods and remove those methods that are too complex or contain logic that may introduce errors (such as complex loops, deep recursion) to improve test efficiency and accuracy. The complexity analysis logic is as follows: Define a complexity analyzer class that inherits from BodyTransformer. In its internal transformation method, first create a unit graph of the method body, and then use the complexity calculator to calculate the complexity of the method. If the calculated complexity exceeds the preset complexity threshold, the method is removed from the list of candidate methods. This process also takes into account factors such as loop nesting depth and recursive calls.

[0055] For each screened test method, the unit test parsing unit performs a comprehensive API call sequence analysis. In addition to extracting the API call sequence, the parameter type, return value type and related exception handling strategy of each API call are analyzed in detail. This unit pays special attention to assertions (assert statements) and verification points (such as JUnit's assertEquals method), which are important for accurately understanding the expected results and verification logic of the test. In addition, the special processing requirements of different types of methods need to be considered. For static methods, directly record their calling methods; for instance methods, analyze the creation and initialization process of the objects to which they belong. For constructors, pay special attention to their parameters and possible nested calls. It is also necessary to identify methods that may throw exceptions and analyze how these exceptions are handled to provide a basis for the subsequent generation of robust fuzz test drivers.

[0056] The unit test parsing unit introduces taint analysis technology to track the propagation path of test input data in the API call sequence, which helps to identify API calls that directly or indirectly depend on the test input. In addition, during the analysis process, special attention should be paid to API calls related to resource management. By identifying the opening and closing modes of resources such as file operations and network connections, it is ensured that the complete resource life cycle information is retained in the extracted API sequence, which is very helpful for generating fuzz test drivers that will not cause resource leakage. Finally, this unit organizes the extracted API call sequence into a directed graph structure. In this structure, the nodes represent each API call, and the edges represent the order relationship and data dependency between the calls. This structured representation method not only retains the order information of the API calls, but also captures the complex relationship between them. The analysis results are output in a structured JSON format, containing key data such as detailed API call sequence information, parameter and return value types, exception handling strategies, assertions and verification points.

[0057] 2. Initial driver generation module

[0058] The initial driver generation module includes a driver generation unit and an execution unit, which are used to generate and execute the initial fuzz test driver using a large language model.

[0059] 1. Drive generation unit

[0060] The driver generation unit provides the processed analysis results as input to large language models (LLMs). In order to make more effective use of large models, the initial driver generation module contains 11 heuristic methods for generating Java fuzz test drivers, combined with the features of the Jazzer framework:

[0061] (1) Static method call: directly call the static target method, using the @FuzzTest annotation and Jazzer's built-in fuzzer;

[0062] (2) Instance method call: After creating an object, the instance method is called, including constructors, static factory methods, and instance factory methods, and a custom fuzzer and @Fuzz annotation are applied;

[0063] (3) Pre-method call: Call the setup method before the target method, using Jazzer's Hook mechanism;

[0064] (4) Result consistency check: execute the method multiple times and compare the results, using the guideTowardsEquality() method;

[0065] (5) Enumeration type processing: randomly select an enumeration value and apply the consumeEnum() method;

[0066] (6) Fixed option parameter processing: randomly select parameter values ​​within the valid range and use the consumeInt() method;

[0067] (7) Type parameter processing: Generate valid object instances using custom fuzzers and @Fuzz annotations;

[0068] (8) Constructor testing: Test the constructors of non-exceptional classes and non-test classes, using coverage guidance functions;

[0069] (9) Exception handling test: Design use cases to trigger and verify exception handling logic, and use Jazzer's exception handling mechanism;

[0070] (10) Resource management test: Check the acquisition, use and release of resources, and apply Jazzer's resource management function;

[0071] (11) Boundary value testing: Explore the boundary values ​​and extreme cases of parameters using the reportFeedback() method.

[0072] In addition, the driver generation unit contains a specific prompt template for generating the initial fuzz test driver for the large model. The prompt template requires the generation of a Jazzer-based Java fuzz test driver based on a given API call sequence and related information. The prompt template lists the information that needs to be provided, including the API call sequence, parameter type and value range, return value type and expected result, exception handling strategy, and assertion and verification points. The template contains nine key points to consider when generating a fuzz test driver:

[0073] (1) Use Jazzer's @FuzzTest annotation to mark the test method;

[0074] (2) Generate random input using Jazzer’s built-in fuzzer;

[0075] (3) Use Jazzer's coverage guidance function to optimize testing;

[0076] (4) Implementing custom fuzzers to handle complex data types;

[0077] (5) Use Jazzer’s Hook mechanism to explore edge cases and abnormal paths;

[0078] (6) Include appropriate assertions and validations, using Jazzer’s built-in checks;

[0079] (7) Use Jazzer's resource management function;

[0080] (8) Implement a custom feedback mechanism to guide the fuzz testing process;

[0081] (9) Set appropriate timeouts and memory limits.

[0082] Finally, the prompt template emphasizes that the generated code should comply with Jazzer's best practices and effectively use Jazzer's features to improve test efficiency and coverage.

[0083] This hint template is designed to guide the large language model to generate comprehensive and efficient fuzz test drivers, making full use of the various functions and features of the Jazzer framework. The large model uses this information to generate initial fuzz test driver code, which can creatively explore new test scenarios and generate boundary conditions and abnormal paths that are not covered in the original unit test.

[0084] 2. Execution Unit

[0085] The execution unit uses an efficient parallel executor, Airflow and fork mechanism to optimize the performance of fuzz testing. The core idea of ​​this unit is to use Airflow as a scheduling and monitoring platform, and use the fork mechanism to quickly create and manage fuzz testing processes. The details are as follows:

[0086] 1. Airflow scheduler: As the hub of the entire system, it is responsible for task scheduling and workflow management, and assigns tasks to each work node;

[0087] 2. Worker nodes: Each worker node is responsible for initializing and running a fuzz test instance. The worker node receives configuration information from the Airflow scheduler, including the number of loops, the number of slave processes, the duration of the fuzz test, and the termination condition (such as the number of crashes found);

[0088] 3. Fork server: Each working node starts a fork server to quickly create a clone of the target program. This method significantly reduces the overhead of process creation and improves testing efficiency;

[0089] 4. Result collector: responsible for collecting running results from each working node, including possible running failure information, discovered uncaught exceptions, etc. In addition, it obtains the specific code lines, branches, and method information covered by each test driver, monitors the code execution path in real time through bytecode instrumentation, and generates an XML report containing line, branch, and method level coverage data. After parsing the XML report, a binary feature vector is constructed for each test driver. The vector is represented by a sparse matrix and stored in the CSR (Compressed Sparse Row) format to save memory and speed up subsequent calculations. The feature vector contains the coverage of code lines, branches, and methods, and each element corresponds to a unique code unit;

[0090] 5. Error detection and logging: Implement error detection mechanisms in worker nodes. When driver execution fails, detailed records of failure reasons, stack traces, and related environment information are recorded. These logs will be sent to the result collector.

[0091] 6. Automatic retry mechanism: For certain types of failures (such as syntax errors), they will be input into the big model with error information for correction, and an automatic retry mechanism will be implemented. The maximum number of retries and retry interval will be set to avoid test interruptions caused by temporary problems.

[0092] 7. Isolate failed drivers: When a test driver fails repeatedly, mark it as a "problem driver" and isolate it to prevent the problem driver from affecting the overall test process.

[0093] 3. Optimization Module

[0094] The optimization module includes an iterative optimization unit and an evaluation unit, which are used to iteratively optimize the fuzz test driver according to the coverage information.

[0095] 1. Iterative Optimization Unit

[0096] The iterative optimization unit optimizes the generated results, including:

[0097] (1) Driver classification: Based on the coverage data, the improved k-means++ algorithm is used to classify the fuzzy test drivers. The initial center point selection adopts the D^2 sampling method to improve the clustering quality. The Mini-Batch K-means variant is used to cope with large-scale data. The number of clusters k is automatically adjusted by the silhouette coefficient to ensure the best clustering effect.

[0098] (2) Merge drivers of the same region: For each type of driver, merge drivers that cover the same or similar regions. The merging process uses dynamic programming to identify similar API call sequences, builds a two-dimensional dynamic programming table, and gradually calculates the longest common subsequence length of the two sequences. By backtracking this table, the specific common subsequence can be obtained, thereby identifying similar API call patterns. The identified similar API call sequences and contextual information such as parameter types, return value usage, and exception handling logic are then provided to the large language model, and the merged driver code is generated based on this information, while retaining the unique test logic of each original driver.

[0099] (3) Merge drivers of different regions: Based on the merging of drivers of the same region, merge and optimize drivers across categories to improve the overall test coverage. Provide the drivers of the previous step to the executor for re-execution to obtain coverage and other result information, and also use the large model to merge these drivers.

[0100] 2. Evaluation Unit

[0101] The evaluation unit uses an iterative optimization process to determine the long-term trend of coverage and avoid premature or late termination of the iteration process due to short-term fluctuations. The large model optimizes the test drive based on the coverage data of each round of execution and the anomalies found. The optimization focuses on three aspects: increasing the testing of low coverage areas, deepening the exploration of anomaly triggering paths, and optimizing parameter generation strategies.

[0102] The algorithm for determining coverage convergence is as follows:

[0103] (1) Define the sliding window size w and 5 iterations;

[0104] (2) Calculate the average coverage growth rate of the last w iterations:

[0105] Growth rate = (current coverage rate - coverage rate before round w) / coverage rate before round w;

[0106] (3) Set the convergence threshold ε to 0.001 and the minimum number of iterations N to 30 rounds;

[0107] (4) If the growth rate is less than ε for three consecutive rounds and the total number of iterations is greater than or equal to N, convergence is achieved.

[0108] The present invention also discloses a fuzzy test driver generation method based on unit testing, using the above fuzzy test driver generation system based on unit testing, such as Figure 2 As shown, the following steps are included:

[0109] Step 1: The target build unit in the parsing module automatically analyzes the Java project to support the subsequent fuzz test driver generation. It automatically detects the build system used by the project by traversing the file structure of the target project repository. When multiple build system configuration files are detected, the target build unit selects the most suitable build system according to the preset priority order.

[0110] Step 2: The unit test parsing unit in the parsing module performs static analysis on the Java project through the Soot framework to identify all test methods annotated with @Test. During the analysis, a context-sensitive call graph is constructed, the k-CFA algorithm is used, and the k value is adjusted to achieve a balance between analysis accuracy and computing performance.

[0111] Step 3, the driver generation unit in the initial driver generation module provides the processed analysis result as input to the large language model, so that the large language model generates an initial fuzz test driver;

[0112] Step 4: The execution unit in the initial driver generation module uses Airflow as a scheduling and monitoring platform, and uses the fork mechanism to quickly create and manage the fuzz testing process;

[0113] Step 5, the iterative optimization unit in the optimization module optimizes the generated results;

[0114] Step 6: The evaluation unit in the optimization module uses an iterative optimization process to determine the long-term trend of coverage to avoid premature or late termination of the iteration process due to short-term fluctuations. The large model optimizes the test drive based on the coverage data of each round of execution and the abnormal situations found.

[0115] In summary, the present invention first implements an automated Java project construction and analysis method. The target construction unit supports multiple construction systems such as Maven, Gradle and Ant, and implements an adaptive JDK version selection mechanism. This mechanism can automatically try different versions of OpenJDK until a version that can successfully build the project is found, which greatly improves the compatibility with different Java projects. The unit test parsing unit uses the Soot framework for static analysis and uses the Shimple intermediate representation and k-CFA algorithm to build an accurate context-sensitive call graph. In order to improve the analysis efficiency, method call frequency filtering and method complexity filtering mechanisms are introduced. These mechanisms can screen out methods that have an important impact on system behavior and are of moderate complexity, thereby improving the efficiency and accuracy of subsequent analysis and testing. Taint analysis technology is also introduced in the analysis process to track the propagation path of test input data in the API call sequence, which helps to identify API calls that directly or indirectly depend on the test input.

[0116] The driver generation module uses LLM to generate the initial fuzz test driver, using specific prompt templates and heuristic methods. The present invention summarizes a series of heuristic methods for generating Java fuzz test drivers, including static method calls, instance method calls, pre-method calls, result consistency checks, etc., and applies these methods in combination with the characteristics of Jazzer. In order to improve the operating efficiency, the present invention implements an efficient parallel executor, which uses Airflow and fork mechanisms to optimize the performance of fuzz testing. The executor includes components such as Airflow scheduler, work node, fork server, result collector, etc., and also implements functions such as error detection and logging, automatic retry mechanism, and isolation of failed drivers. The result collector not only collects the running results, but also obtains the specific code lines, branches and method information covered by each test driver, and monitors the code execution path in real time through bytecode stubs.

[0117] Next, the initial driver generation module optimizes the generated results. First, the fuzzy test drivers are classified using the improved k-means++ algorithm, the Mini-Batch K-means variant is used to cope with large-scale data, and the number of clusters k is automatically adjusted through the silhouette coefficient. Then, for each type of driver, drivers covering the same or similar areas are merged. The merging process uses dynamic programming to identify similar API call sequences and generates the merged driver code through a large language model. On the basis of merging drivers in the same area, cross-category driver merging and optimization are also performed to improve the overall test coverage.

[0118] Finally, the present invention uses an iterative optimization process to determine the long-term trend of coverage, avoiding premature or late termination of the iteration process due to short-term fluctuations. The specific algorithm includes steps such as defining the sliding window size, calculating the average coverage growth rate, setting the convergence threshold and the minimum number of iterations. The large model optimizes the test drive based on the coverage data of each round of execution and the anomalies found, focusing on increasing the test of low coverage areas, deepening the exploration of anomaly triggering paths, and optimizing parameter generation strategies.

[0119] The present invention generates a more comprehensive fuzz test driver by combining the API usage pattern, test logic and LLM capabilities in existing unit tests, especially improving the identification of boundary conditions and abnormal paths. By introducing parallel processing technology, specific prompt templates and feedback mechanisms, the efficiency and effectiveness of fuzz testing are optimized, providing developers with a new tool to explore potential problems and vulnerabilities of software in a more systematic and comprehensive way. The present invention aims to improve the efficiency and coverage of software testing, provide a new solution for quality assurance in the software development process, and help improve the overall quality and reliability of the software.

[0120] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fuzzy test driver generation system based on unit testing, characterized in that: Includes parsing module, initial drive generation module and optimization module; The parsing module includes a target construction unit and a unit test parsing unit, which are used to automatically build a target Java project and extract a unit test API sequence; The initial driver generation module includes a driver generation unit and an execution unit, which are used to generate an initial fuzz test driver using a large language model and execute it; The optimization module includes an iterative optimization unit and an evaluation unit, which are used to iteratively optimize the fuzz test driver according to the coverage information; The target construction unit automatically analyzes the Java project to support subsequent fuzz test driver generation, and automatically detects the construction system used by the project by traversing the file structure of the target project warehouse. When multiple construction system configuration files are detected, the target construction unit selects the most suitable construction system according to a preset priority order; The unit test parsing unit performs static analysis on the Java project through the Soot framework to identify all test methods annotated with @Test; During the analysis process, a context-sensitive call graph is constructed, the k-CFA algorithm is used, and the k value is adjusted to achieve a balance between analysis accuracy and computational performance; The unit test parsing unit performs a comprehensive API call sequence analysis on each screened test method, and analyzes in detail the parameter type, return value type and related exception handling strategy of each API call; The driver generation unit provides the processed analysis results as input to the large language model, which is used for the large language model to generate an initial fuzz test driver; the driver generation unit includes a specific prompt template, which is used for the large model to generate an initial fuzz test driver; the prompt template lists the information that needs to be provided, including API call sequence, parameter type and value range, return value type and expected result, exception handling strategy, and assertion and verification points; The execution unit uses a parallel executor, uses Airflow as a scheduling and monitoring platform, and uses a fork mechanism to quickly create and manage fuzz testing processes; The iterative optimization unit optimizes the generated result, specifically including: Driver classification: Based on the coverage data, the improved k-means++ algorithm is used to classify the fuzz test drivers; Merge drivers of the same region: For each type of driver, merge drivers covering the same or similar regions; Merge drivers of different regions: Based on the merging of drivers of the same region, merge and optimize drivers across categories to improve the overall test coverage; The evaluation unit uses an iterative optimization process to determine whether the coverage has converged, avoiding premature or late termination of the iteration process due to short-term fluctuations; the large model optimizes the test drive based on the coverage data of each round of execution and the abnormal conditions found; the optimization focuses on three aspects: increasing the testing of low coverage areas, deepening the exploration of abnormal trigger paths, and optimizing parameter generation strategies; The algorithm for determining coverage convergence is as follows: (1) Define the sliding window size w and 5 iterations; (2) Calculate the average coverage growth rate of the last w iterations: Growth rate = (current coverage rate - coverage rate before round w) / coverage rate before round w; (3) Set the convergence threshold ε to 0.001 and the minimum number of iterations N to 30 rounds; (4) If the growth rate is less than ε for three consecutive rounds and the total number of iterations is greater than or equal to N, convergence is achieved.

2. A fuzzy test driver generation method based on unit testing, using a fuzzy test driver generation system based on unit testing as claimed in claim 1, characterized in that: The steps include: Step 1, the target construction unit in the parsing module automatically analyzes the Java project to support the subsequent fuzz test driver generation, and automatically detects the construction system used by the project by traversing the file structure of the target project warehouse. When the configuration files of multiple construction systems are detected, the target construction unit selects the most suitable construction system according to the preset priority order; Step 2: The unit test parsing unit in the parsing module performs static analysis on the Java project through the Soot framework to identify all test methods annotated with @Test; During the analysis process, a context-sensitive call graph is constructed, the k-CFA algorithm is used, and the k value is adjusted to achieve a balance between analysis accuracy and computational performance; Step 3, the driver generation unit in the initial driver generation module provides the processed analysis result as input to the large language model, so that the large language model generates an initial fuzzy test driver; Step 4: The execution unit in the initial driver generation module uses Airflow as a scheduling and monitoring platform, and uses a fork mechanism to quickly create and manage the fuzz testing process; Step 5, the iterative optimization unit in the optimization module optimizes the generated result; Step 6, the evaluation unit in the optimization module uses an iterative optimization process to determine whether the coverage rate has converged, so as to avoid terminating the iterative process too early or too late due to short-term fluctuations; The big model optimizes the test drive based on the coverage data of each round of execution and the anomalies found.

Images