Coverage-Guided Code Generation and Fuzzing Methods Based on Large Models

Abstract

A coverage-guided code generation and fuzzing method based on a large model is proposed to address the problems of existing large model fuzzers, which not only ignore the guiding role of coverage information in the fuzzing loop but also generate prompts too hastily. This invention comprises four stages: prompt generation, fuzzing loop, mutation operator selection, and vulnerability detection. This method proposes a prompt generation method based on expert templates and a fuzzing loop method guided by unequal interval coverage information. The former lowers the barrier to entry for users of CoverFuzz and improves the quality of initial prompts; the latter, guided by coverage information, better covers the object under test to detect vulnerabilities. This invention significantly improves the effectiveness of programs generated by fuzzing based on large models, solves the problem of existing large model-based fuzzers ignoring coverage information, and also achieves higher code coverage.

Description

Technical Field

[0001] This invention relates to the field of software testing, specifically to a coverage-guided code generation and fuzzing method based on a large model. This method aims to improve the reliability and security of software systems through efficient code generation and vulnerability detection mechanisms. Background Technology

[0002] Fuzzing is an automated testing method that exposes inherent flaws in a system under test by generating inputs that trigger unexpected behavior. Fuzzing plays a crucial role in discovering errors and vulnerabilities in various systems under test, especially low-level foundational components such as programming language compilers, runtime engines, constraint solvers, and software libraries with accessible APIs. The proper functioning of these components determines the proper functioning of higher-level software; therefore, discovering vulnerabilities in these components is of paramount importance.

[0003] Existing fuzzing methods are mainly divided into two types: generation-based and mutation-based. Generation-based fuzzing utilizes predefined syntax and semantic knowledge of the target language to construct complete code segments, while mutation-based fuzzing generates new fuzzy inputs by iteratively transforming a seed program. In recent years, with the development of Large Language Models (LLMs), large model-based fuzzing methods have gradually emerged. These methods leverage the powerful generative capabilities of large models to generate diverse test code, significantly improving the efficiency and coverage of fuzzing.

[0004] However, existing general-purpose fuzzers have the following problems in the fuzzing loop process:

[0005] Ignoring the guiding role of coverage information: Coverage information is an important feedback in fuzzing, guiding the direction of fuzzer mutation and helping the fuzzer understand uncovered paths in the tested system. For example, JFSAMPLER enhances the code coverage feedback signal of the fuzzer by measuring the coverage of the formula solution space, while Fuzz4All fails to effectively utilize coverage information, resulting in low fuzzy loop coverage efficiency.

[0006] Lack of professional guidance in prompt word generation: The quality of prompt words directly affects the quality of the code generated by the large model. Existing general-purpose fuzzers such as Fuzz4All lack professional guidance in prompt word generation, which may result in low-quality prompt words and consequently affect the quality of subsequent fuzzing loops.

[0007] Insufficient validity and diversity in generated code: Existing fuzzing methods based on large models often fail to adequately consider the validity and diversity of the generated code. This results in potentially numerous syntax or logical errors in the generated code, increasing the workload of manual verification and reducing test coverage.

[0008] To address the aforementioned issues, this invention proposes a coverage-guided code generation and fuzzing method based on a large model. The aim is to improve the efficiency and quality of fuzzing by using fuzzy loops guided by non-uniform coverage information and prompt word generation guided by expert templates. Summary of the Invention

[0009] This invention proposes a coverage-guided code generation and fuzzing method based on a large model, aiming to address the problems of existing general-purpose fuzzers neglecting coverage information during the fuzzing loop and lacking professional guidance in prompt word generation. This invention improves the efficiency and quality of fuzzing by using fuzzing loops guided by non-equidistant coverage information and prompt word generation guided by expert templates.

[0010] To achieve the above objectives, the specific technical solution adopted by this invention is a coverage-guided code generation and fuzz testing method based on a large model, comprising the following steps:

[0011] (1) Prompt word generation stage: Obtain the name of the system under test and the corresponding formal language name input by the user, construct an expert template as the initial prompt word template, inject the name of the system under test and the corresponding formal language name into the expert template to obtain the initial prompt word; the initial prompt word is used to prompt the large model to generate code for testing;

[0012] (2) Fuzzy loop stage: Based on the initial prompt words, code is generated through a large model, and in each fuzzy loop, only a portion of the code is calculated for coverage;

[0013] If coverage is calculated, the code and its coverage information are saved to the database, and the code with the highest number of covered lines is selected as the seed code to enter the next loop.

[0014] If coverage is not calculated, the code is executed directly, and the latest generated code is used as the seed code for the next round of the loop. In each round of the fuzzy loop, an operator is randomly selected from the three mutation operators Generate, Mutate, and Semantics to mutate the code, and the operator is scored based on the coverage and accuracy of the code generated by the operator.

[0015] Update the operator with the lowest score;

[0016] (3) Vulnerability detection phase: After the fuzzy loop phase ends, the code generated by the iteration is run in the system under test, the code running information is recorded and saved as a log file, and the log file is scanned periodically to detect whether there are characters related to the vulnerability report. If a vulnerability report is detected, a vulnerability report is sent to the user.

[0017] Preferably, in the prompt word generation stage:

[0018] The specific form of the expert template includes the following aspects: the task is to generate code for the system using the language, ensure the code syntax is correct, and follow the best practices of the language;

[0019] In this template, the two underscores ("_") represent slots for injecting the formal language name and the name of the system under test, respectively. By explicitly specifying the task objective, this template ensures that the code generated from the large model not only conforms to syntactic requirements but also follows best practices in the language, thereby improving the quality of the generated code.

[0020] Preferably, the fuzzy loop stage further includes:

[0021] Configure the main process and child processes;

[0022] In the main process, code generation and execution operations are continuously performed to drive the fuzzing process, without introducing a coverage calculation step, in order to avoid time consumption caused by coverage statistics;

[0023] In the subprocess, coverage statistics are performed on a portion of the test code, and the results are written to the database in real time for unified management and analysis in subsequent stages. This design decouples coverage calculation from fuzzy loops, maintaining overall execution efficiency while providing accurate feedback to the code generation process, thereby effectively improving test coverage and vulnerability discovery capabilities.

[0024] Each time the code with the highest number of covered lines is retrieved from the database, the number of covered lines corresponding to that code is multiplied by the seed selection parameter K and then put back into the database to avoid selecting the same seed every time, which would reduce the fuzzy loop capability.

[0025] Preferably, the fuzzy loop stage further includes:

[0026] In the initial round of fuzzy loop, the Generate operator is used to generate initial code; in each subsequent round of fuzzy loop, an operator is randomly selected from three mutation operators to achieve further code mutation.

[0027] The Mutate operator is used to fine-tune the code generated in the previous round, such as inserting other functions or modifying parameters. It is suitable for making minor modifications to existing code to explore new code paths.

[0028] Semantics operators: perform semantic transformations on code, keeping the code's functionality unchanged but changing its expression. They are suitable for increasing code diversity without altering its functionality.

[0029] Preferably, in the fuzzy loop stage, the operator with the lowest score is updated according to the following update strategy, the update strategy including:

[0030] Obtain the coverage rate and the correctness ratio of the representation coverage ability of the code generated by each mutation operator, and calculate the score of each mutation operator by weighting the coverage rate and the correctness ratio at a ratio of 1:1.

[0031] Set a time interval, compare the scores of the three operators at regular intervals, and replace the operator with the lowest score.

[0032] Preferably, the operator with the lowest score is replaced according to the following replacement strategy, the replacement strategy including:

[0033] Construct update prompts for updating operators, and regenerate the description of the lowest-scoring operator using the large model:

[0034] The update prompts specifically include: making minor modifications to the sentence to make the text slightly different, but still clearly conveying the original meaning;

[0035] The update prompt requires the large model to modify the token composition of the operators while keeping the semantics of the mutation operators unchanged, so that the operator with the lowest capability can have different performance after mutation.

[0036] Preferably, during the vulnerability detection phase:

[0037] The scheduled scan log file specifically contains code execution information from the scheduled scan log file.

[0038] Regular expressions are used to match the text of the runtime information to detect the presence of characters related to the vulnerability report;

[0039] The characters related to the vulnerability report include: error messages;

[0040] Preferably, the prompt word generation stage further includes:

[0041] The system validates the user-inputted name of the system under test and the formal language name. This ensures that the user input conforms to the preset format and requirements, thereby improving the accuracy and reliability of the prompt word generation.

[0042] Preferably, the fuzzy loop stage further includes: performing preliminary verification on the generated code in each round of fuzzy loop; the preliminary verification includes checking the syntax correctness of the code. This allows for the early screening of obviously erroneous code, avoiding wasted resources on subsequent coverage information calculation or execution, and further improving testing efficiency.

[0043] Preferably, the vulnerability detection phase further includes classifying and prioritizing detected vulnerabilities. Based on the severity and impact of the vulnerabilities, more targeted remediation suggestions and priority guidance are provided to users, helping them to handle and remediate vulnerabilities more efficiently. To ensure the efficiency and accuracy of fuzzing, this invention adopts a coverage information-guided strategy and models the prompt word generation problem in the fuzzy loop phase as an expert-guided template filling task, using expert-preset prompt word templates to improve the quality and professionalism of code generation. To further improve the effectiveness of code generation, this invention incorporates a non-uniformly spaced coverage information-guided fuzzy loop mechanism, performing intermittent coverage information calculations on the generated code, thereby reducing performance bottlenecks caused by frequent coverage calculations. This invention significantly improves the quality and coverage of generated code in large-model-based fuzzing, solving the shortcomings of existing general-purpose fuzzers in code generation efficiency and quality, and also has a higher vulnerability discovery capability.

[0044] This invention proposes a coverage-guided code generation and fuzzing method based on a large model, which has the following characteristics and advantages compared to other fuzzing methods:

[0045] 1) This invention employs a non-uniform coverage information guidance method in the fuzzy loop stage. Through multi-threading, it not only ensures the overall performance of the fuzzy loop but also effectively utilizes coverage information to guide subsequent code generation, significantly improving test coverage.

[0046] 2) In the prompt word generation stage, this invention generates high-quality initial prompt words through expert templates, which lowers the user's usage threshold and improves the quality and professionalism of the generated code. Attached Figure Description

[0047] Figure 1 This is the overall flowchart of the present invention; Detailed Implementation

[0048] The present invention will now be described systematically and completely with reference to the accompanying drawings.

[0049] 1. Prompt Keyword Filtering Stage

[0050] The prompt word filtering stage of this invention specifically includes the following steps:

[0051] (1-1) Obtain the test object and test language given by the user based on their own needs.

[0052] (1-2) User-given parameters are injected into the expert template. During the prompt generation stage, the expert template takes the form: "Your current task is to generate code for the _ system using the _ language. Please ensure the code syntax is correct and follows the best practices of this language." Here, _ represents the name of the system under test and the name of the formal language, respectively. This template, by explicitly specifying the task objective, ensures that the code generated by the large model not only meets syntactic requirements but also follows the best practices of the language, thereby improving the quality of the generated code.

[0053] (1-3) The prompt word templates with injected parameters are used to generate prompt words that can be better based on the large model.

[0054] 2. Fuzzy Loop Stage

[0055] In the fuzzy loop stage described in this invention, calculating the coverage information of the tested object after each code execution significantly impacts the performance and efficiency of the fuzzy loop. For example, when fuzzing g++ using C++, calculating the g++ coverage after each code execution takes two to three minutes to obtain coverage information for each code. Furthermore, since we use a large model via API calls, our 24-hour fuzzing test can only generate a few hundred code snippets, resulting in extremely low efficiency. Because the time spent calculating the coverage information of the tested object is tens of times longer than the time required to execute the code (which averaged 3 seconds in the experiment), the fuzzy loop stage is implemented as follows: Specifically, it includes the following steps:

[0056] (2-1) In the main process, this method drives the fuzzing process by continuously generating and executing code, while deliberately avoiding the introduction of coverage calculation steps to avoid the additional performance consumption caused by coverage statistics, thereby ensuring that the fuzzing loop can run with higher throughput and stability, and further improving the quality and coherence of the testing process.

[0057] (2-2) In the subprocess, this method performs coverage statistics on a portion of the test code and writes the results to the database in real time for unified management and analysis in subsequent stages. This design decouples coverage calculation from the fuzzy loop, maintaining overall execution efficiency while providing accurate feedback to the code generation process, thereby effectively improving test coverage and vulnerability discovery capabilities. Furthermore, after retrieving the code with the highest number of covered lines from the database each time, the corresponding number of covered lines is multiplied by the seed selection parameter K and then re-added to the database to avoid selecting the same seed every time, which would reduce the capability of the fuzzy loop. The algorithm for the fuzzy loop is as follows:

[0058]

[0059] Algorithm 1 describes the flow of the fuzzy loop algorithm in CoverFuzz. In `Init_parameter`, we first initialize `LimitTime` to 86400 seconds (1 day), `Time` to 0, and `Loop` to 0. We then assign initial values ​​to the initial prompt word for the fuzzy loop, initialize the code from the previous round and the fuzzy operator prompt word to empty strings, and initialize the lock object (lines 1-2). During the fuzzy loop, we first record the current timestamp, then increment the loop count by 1 (lines 4-5). In each loop, our prompt word consists of the initial prompt word, the code generated in the previous round, and a randomly generated mutation operator. We then use this prompt word to generate code through the large model (lines 6-7). Next, we check if the current lock is locked. If it is not locked, we acquire the lock and execute a function to calculate coverage information through a subprocess. After that, we unlock the lock, save the code and its coverage information to the database, and use the code with the highest number of covered rows in the database as the example code for the next round (lines 8-12). If the lock is already engaged, then we only run the code without calculating coverage information, and the next round's example code is the latest generated code (lines 13-15). Next, we randomly select one of the three mutation operators as the mutation operator for the next round of fuzzy loop, add the time to the total time, and save the code (lines 16-18). Algorithm 1 has a time complexity of O(N). The time consumed by the entire framework is mainly in calling the large model and using traditional tools to calculate coverage information.

[0060] (2-3) The Generate operator is used to prompt the large model to generate code; in the initial round of fuzzy loop, the Generate operator is used to generate initial code; in each subsequent round of fuzzy loop, a random selection is made from three mutation operators to achieve further code mutation.

[0061] (2-4) Mutate operator: used to fine-tune the code generated in the previous round, such as inserting other functions or modifying parameters. It is suitable for making small modifications to existing code to explore new code paths.

[0062] (2-5) Semantics operators: perform semantic transformations on code, keeping the code's functionality unchanged but changing its expression. They are suitable for increasing code diversity without changing its functionality.

[0063] (2-6) Update Strategy: The capability of fuzzy loops is mainly reflected in their coverage capability, and the ability to generate correct code is equally important. Therefore, we collect the coverage and correctness ratio of the code generated by each mutation operator, and allocate them according to the ratio of coverage capability to correct code generation capability in a (1:1) ratio to calculate the score of each operator. :

[0064]

[0065] Every so often, we compare the scores of the three operators. Among them, This represents the score of the i-th operator. This represents the number of row covers contributed by the i-th operator during this time period. This represents the number of effective procedures contributed by the i-th operator during this time period. This indicates the total number of rows covered during this period. This represents the total number of valid procedures added during this period.

[0066] The lowest-scoring mutation operator performed poorly, resulting in lower code coverage and efficiency compared to the other two operators. Therefore, we regenerate the operator's description based on the large model by adding a mutation operator update prompt. The mutation operator update prompt is as follows: "Please make minor modifications to the sentence to make the text slightly different, but still clearly convey the original meaning." The update prompt requires the large model to modify the operator's token composition while maintaining the semantics of the mutation operator, hoping that the operator with the lowest performance will have a different performance after mutation.

[0067] 3. Vulnerability Detection Phase

[0068] (3-1) Periodically scan the log files for code execution information;

[0069] (3-2) Use regular expressions to match the text of the runtime information to determine whether there are characters related to the vulnerability report, such as specific error messages like "internal compiler error", so as to discover potential vulnerabilities in a timely manner.

[0070] The above-described specific embodiments are only used to further illustrate the present invention and are not the only implementation of the present invention, nor should they be construed as limiting the present invention.

[0071] Experimental results:

[0072] Table 1 Comparison of CoverFuzz and state-of-the-art general-purpose fuzzers (where DSv3 represents DeepseekV3)

[0073]

[0074] Experiment 1: Comparative Experiment.

[0075] Table 1 lists the final detailed results of CoverFuzz and Fuzz4All, including the total number of generated programs, the overall program efficiency, the number of rows covered by the tested objects, the coverage rate, the large model used, the number of tokens consumed, and the monetary cost.

[0076] In terms of the number of generated programs, we found that CoverFuzz generated more programs than Fuzz4All. This is due to our use of fuzz testing guided by non-uniform interval coverage information. Fuzz testing guided by non-uniform interval coverage information uses two processes. The main process and Fuzz4All operate at roughly the same speed, while the child process processes code more slowly. This means that every few code snippets, a piece of code is handed over to the child process for execution and coverage calculation. This results in some rounds of CoverFuzz's main process taking less time, thus performing more rounds in the same amount of time, and therefore generating more code.

[0077] From an efficiency perspective, CoverFuzz outperforms Fuzz4All on most test targets and large models. This demonstrates that CoverFuzz not only generates code that effectively covers paths not executed by the tested object, but also largely avoids meaningless syntax errors. Although CoverFuzz's efficiency is slightly lower than Fuzz4All when using chatgpt to generate C++ code for testing on g++ and when generating SMT2 code for testing on CVC5, its coverage exceeds Fuzz4All by 10.3% and 30.93%, respectively. Even though CoverFuzz's generated code is slightly less efficient on these two projects, the higher coverage indicates that CoverFuzz's generated code is of higher quality.

[0078] In terms of token and monetary costs, CoverFuzz is indeed more expensive than Fuzz4All. This is because we naturally generate more programs than Fuzz4All, so the cost of calling the API is naturally higher than that of Fuzz4All.

[0079] Experiment 2: Ablation Experiment

[0080] Table 2 Functions of each component

[0081]

[0082] To investigate the contributions of each key component of CoverFuzz to the overall fuzz testing framework, we conducted ablation experiments based on three key components and one key parameter of CoverFuzz: (a) cue word generation, i.e., cue word generation based on cue word templates; (b) fuzzy loop, i.e., fuzzy loop guided by non-equidistant coverage information; (c) seed selection strategy, i.e., the seed ranking parameter K in the seed selection strategy; and (d) mutation factor variation, i.e., the statistical score of mutation factors. Table 2 shows the number of covered rows and the effectiveness of the studied variants.

[0083] Regarding the prompt generation, the results show that CoverFuzz's prompt generation component, compared to manually setting prompts, generates code that covers more lines of the tested object and is more efficient than methods without this component on most tested objects. Go has stricter syntax constraints during execution, meaning that even slight syntax errors will cause compiler errors (e.g., unused variables). In other words, Go has a higher threshold for valid code, while other languages ​​have lower thresholds. The results show an efficiency difference of 65.33% in Go, while the difference is not significant in other languages. This is because Go's strict syntax means that Go code that executes successfully is of high quality, while code of moderate quality in other languages ​​may also execute successfully. This further demonstrates that our prompt generation method can generate better code.

[0084] For fuzzing, at this stage, we maintain the initial hints, that is, we uniformly use the hints generated during the hint generation stage to conduct ablation experiments on fuzzy loops. 1) CoverFuzz represents fuzzing guided by non-uniform interval coverage information; 2) Single-threaded approach refers to fuzzing that calculates coverage information for each piece of code generated. From the results, we can see that on the four tested objects, CoverFuzz outperformed Fuzz4All in terms of line coverage by 93.73%, 5.31%, 31.84%, and 39.31%, respectively. This is because calculating the coverage information of the tested object is a very time-consuming process in most tested systems. If we use a single-threaded approach to calculate coverage information for each execution, each round of fuzzing loops will be very long, resulting in fewer generated code and a very low line coverage. From the perspective of fuzzing, even though its effectiveness is generally high, its testing efficiency is low, which means its ability to discover vulnerabilities in the tested system is very weak.

[0085] Regarding the parameter K for the seed selection strategy, in this experiment, a smaller K value indicates a greater reduction in the bonus gained by the most interesting code when the overall coverage is low in the early stages. Specifically, when K=1, the score of the seed with the highest number of covered rows remains unchanged after selection; when K=0.1, the score of the seed with the highest number of covered rows is divided by 10 after selection. The experiment shows that the smaller the K value, the greater the row coverage of the tested system by CoverFuzz. This indicates that by frequently updating previously generated codes that provide a large number of row coverage during the fuzzing loop, we can effectively enhance the fuzzer's coverage of unknown paths. However, when K=1 is chosen, although we select the code that provides the historical maximum row coverage, this results in the same code consistently having the highest number of covered rows. Consequently, each time the code with the highest number of covered rows is used to form the prompt word of the large prompt model, the same prompt word is used, resulting in similar generated codes. Furthermore, this code is used in the main process's fuzzing loop, further consolidating the similarity of the generated codes in each interval, leading to a very low final row coverage.

[0086] For mutation factor variants, we mainly set different variants by changing the scoring function of the mutation factor: 1) CoverFuzz uses the full scoring function; 2) w / o correctness capability, that is, the effect of efficiency is not considered when scoring; 3) w / o coverage capability, that is, the effect of row coverage is not considered when scoring; 4) no variation, that is, no variation is made to the representation of the mutation factor.

[0087] Based on the experimental data, we can see that if we do not perform mutations, the row coverage of the four tested objects is the lowest. If we only consider one of the correctness or coverage capabilities, the row coverage is still not comparable to the method that considers both. This precisely illustrates that in the design of the fuzzing framework, we not only need to consider how to improve the coverage index of the tested objects, but also the generation of correct code is very important.

[0088] Experiment 4: Vulnerability Finding Capabilities. Table 3 shows the vulnerabilities we discovered on the four tested systems using CoverFuzz. CoverFuzz discovered a total of 47 vulnerabilities, 18 of which were confirmed by the developers to be previously unknown vulnerabilities.

[0089] Table 3. Vulnerabilities Discovered

[0090]

[0091] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A large model-based coverage-guided code generation and fuzzing method, characterized in that, The method comprises the following steps: (1) prompt word generation stage: obtaining the system name and the corresponding formal language name input by the user, constructing an expert template as an initial prompt word template, injecting the system name and the corresponding formal language name into the expert template to obtain an initial prompt word; The initial prompt word is used to prompt the large model to generate code for testing; (2) fuzzy loop stage: based on the initial prompt word, the code is generated by the large model, and in each fuzzy loop, only part of the code is calculated for coverage; If the coverage is calculated, the code and its coverage information are saved to the database, and the code with the highest coverage line number is selected as the seed code for the next round of loop; If the coverage is not calculated, the code is directly executed, and the newly generated code is used as the seed code for the next round of loop; in each fuzzy loop, one of the three mutation operators, Generate, Mutate and Semantics, is randomly selected to mutate the code, and the operators are scored according to the coverage and accuracy of the generated code; The operator with the lowest score is updated; (3) vulnerability detection stage: after the fuzzy loop stage, the iteratively generated code is run in the system under test, and the code running information is recorded and saved as a log file, and the log file is scanned regularly to detect whether there are characters related to the vulnerability report, and if so, the vulnerability report is sent to the user; The fuzzy loop stage further comprises: Setting a main process and a sub-process; In the main process, the code generation and execution operations are continuously performed to drive the fuzzing process, and the coverage calculation step is not introduced to avoid time consumption caused by coverage statistics; In the sub-process, the coverage of part of the test code is calculated, and the statistical results are written into the database in real time After obtaining the code with the highest coverage line number from the database each time, the code coverage line number corresponding to the code is multiplied by the seed selection parameter K and then reinserted into the database.

2. The large model based coverage guided code generation and fuzz testing method of claim 1, wherein, In the prompt word generation stage: The specific form of the expert template includes the following aspects: the task is to generate code for the system using the language, ensure the syntax of the code, and follow the best practices of the language; Wherein, the two "_" respectively represent the word slot for injecting the formal language name and the system under test name.

3. The large model based coverage guided code generation and fuzz testing method of claim 1, wherein, The fuzzy loop stage further comprises: In the initial fuzzy loop, the Generate operator is used to generate the initial code; in each subsequent fuzzy loop, one of the three mutation operators is randomly selected to achieve further code mutation; Mutate operator: used for fine-tuning the code generated in the previous round, such as inserting other functions or modifying parameters, suitable for making small modifications based on existing code to explore new code paths; Semantics operator: performs semantic transformation on the code, keeping the code function unchanged but changing the expression form of the code, suitable for increasing the diversity of the code without changing the function.

4. The large model based coverage guided code generation and fuzz testing method of claim 1, wherein, In the fuzzy loop stage, the operator with the lowest score is updated according to the following update strategy, which comprises: Obtain the coverage representing the code generation capability of each mutation operator and the correct proportion representing the correct code generation capability, and calculate the score of each mutation operator by weighting the coverage and the correct proportion with a weight of 1:1; Set a time interval, compare the scores of the three operators every certain period of time, and replace the operator with the lowest score.

5. The large model based coverage guided code generation and fuzz testing method of claim 4, wherein, The operator with the lowest score is replaced according to the following replacement strategy, which includes: Construct an update prompt word for updating the operator, and regenerate the description of the operator with the lowest score using a large model: The update prompt word specifically includes: slightly modifying the sentence to make the text different, but still clearly conveying the original meaning.

6. The large model based coverage guided code generation and fuzz testing method of claim 4, wherein, In the vulnerability detection phase: The timing scan log file is specifically the code running information in the timing scan log file; Use regular expressions to match the text of the running information to detect whether there are characters related to vulnerability reports; The characters related to vulnerability reports include: error information.

7. The large model based coverage guided code generation and fuzz testing method of claim 1, wherein, The prompt word generation phase further includes: Verify the user input system name and formal language name.

8. The large model based coverage guided code generation and fuzz testing method of claim 1, wherein, The fuzzy loop phase further includes: in each round of fuzzy loop, the generated code is preliminarily verified; the preliminary verification includes checking the syntax correctness of the code.

9. The large model based coverage guided code generation and fuzz testing method of claim 1, wherein, The vulnerability detection phase further includes classifying and prioritizing the detected vulnerabilities.

Images