Rag-based binary program fuzzing method

Abstract

The application discloses a binary program fuzzy testing method based on RAG, and comprises the following steps: step 1, constructing a seed knowledge base for a target program; step 2, executing a fuzzy testing main loop based on a feedback-driven fuzzy testing tool; step 3, detecting whether the fuzzy testing enters a bottleneck stage in the fuzzy testing main loop; step 4, in the bottleneck stage, sequentially executing multistage mutation processing on non-elite seeds in a seed directory; step 5, writing next-generation seeds obtained through the multistage mutation processing back to the seed directory as input of the next round of the fuzzy testing main loop; and optionally expanding the seed knowledge base based on the execution result. While retaining continuity of the original fuzzy testing main loop, the application can effectively improve the pertinence and executability of the mutation strategy in the bottleneck stage, and enhance the triggering capability of the fuzzy testing on deep paths and program crashes.

Description

Technical Field

[0001] This invention belongs to the field of software vulnerability detection and relates to a binary program fuzzing method, specifically a binary program fuzzing seed mutation method based on a retrieval enhancement generation mechanism. Background Technology

[0002] Fuzzing is a dynamic analysis technique that discovers software defects and vulnerabilities by automatically generating or modifying input data to drive the execution of a target program and monitoring abnormal behavior during runtime. In security analysis of binary programs, fuzzing has become an important means of discovering vulnerabilities under real-world operating conditions.

[0003] Existing coverage-guided gray-box fuzzing tools collect program execution coverage information through lightweight instrumentation and leverage coverage feedback to drive the retention and evolution of the seed queue, enabling continuous exploration of new execution paths. To enhance the ability to overcome binary program input constraints, researchers further introduced techniques such as static analysis and dynamic taint analysis, combined with genetic algorithms to mutate and evolve the seeds, thereby improving the efficiency of exploring deep paths.

[0004] In recent years, researchers have begun to explore incorporating large language models into fuzzing processes to enhance the effectiveness of test case generation and input mutation. Existing methods mainly include: prompt-driven zero-shot or few-shot generation methods, which guide the model to generate test cases through prompts; and fine-tuning or policy learning methods based on fuzzing data, which train the model using historical data to enable it to output more executable and effective mutation operations.

[0005] However, applying the above method to fuzz testing of binary programs still has the following shortcomings:

[0006] First, the input of binary programs is usually not readable, and its internal input parsing logic and path constraints are relatively implicit. Simply relying on coverage feedback to perform byte-level random mutations often makes it difficult to continuously generate effective samples, which makes fuzz testing prone to getting stuck in a bottleneck stage where coverage growth is slow or mutation benefits are declining.

[0007] Second, the high-quality seeds generated during the testing process by traditional fuzz testing methods, which can improve coverage or trigger program crashes, have not been fully utilized, making it difficult to provide structured and reusable reference information for subsequent search processes.

[0008] Third, when relying solely on a large language model to generate mutation strategies, the generated mutation schemes are difficult to satisfy structural constraints due to the lack of specific perception of the current seed structure features and target program input constraints. This can easily lead to problems such as poor executability and low matching degree with the current seed state.

[0009] Currently, no method has been found that combines building a searchable knowledge base based on existing high-quality seeds with a seed mutation strategy based on a search-enhanced generation mechanism for fuzz testing of binary programs. Summary of the Invention

[0010] To address the problems of existing technologies, this invention provides a binary program fuzzing method based on RAG. This method constructs a searchable seed knowledge base based on high-quality seeds generated during fuzzing that can improve program coverage or trigger program crashes. When fuzzing enters a bottleneck stage where coverage growth is slow or mutation benefits decrease, reference seeds related to the current seed to be mutated are retrieved from the seed knowledge base. A multi-level mutation mechanism combining direct concatenation mutation, retrieval-enhanced mutation generation, and rollback mutation is employed to generate a more targeted next-generation seed. This invention, while preserving the continuity of the original fuzzing main loop, effectively improves the targeting and executability of the mutation strategy in the bottleneck stage, and enhances the fuzzing's ability to trigger deep paths and program crashes.

[0011] The objective of this invention is achieved through the following technical solution:

[0012] A binary program fuzzing method based on RAG includes the following steps:

[0013] Step 1: Construct a seed knowledge base for the target program. The seed knowledge base takes existing seeds that can improve program coverage or trigger program crashes as objects. Record metadata, lexical features, structural features and coverage features for each seed, and build grouped indexes according to the target program and file type.

[0014] Step 2: Execute the main loop of fuzzing based on the feedback-driven fuzzing tool. The main loop of fuzzing includes executing the target program on the seeds in the seed queue, evaluating the fitness based on the execution trajectory and coverage feedback, determining whether to trigger a crash or cover a new basic block, and selecting and mutating the seeds based on the fitness.

[0015] Step 3: In the main loop of fuzzing, detect whether the fuzzing test has entered the bottleneck stage. The criteria for determining the bottleneck stage include, but are not limited to: the program coverage has not increased within a preset number of generations, no new crash samples have been found within a preset observation window, or the fitness-based regular mutation has not brought effective benefits in terms of coverage or crashes within a preset number of generations.

[0016] Step 4: During the bottleneck stage, perform multi-level mutation processing sequentially on the non-elite seeds in the seed directory:

[0017] Step 4.1: Determine whether there are any usable reference seeds in the seed knowledge base. If there are, and the triggering conditions are met, then use the reference seeds as input to perform direct splicing and mutation of the reference seeds to obtain the next generation seeds.

[0018] Step 4.2: If step 4.1 is not triggered, determine whether to call the large language model according to the preset probability, and check whether the large language model call budget has not reached the upper limit; if the call conditions are met, construct the prompt word context with the current seed state, the retrieved reference seed information and the input format constraints, output the structured mutation operation description by the large language model, parse the description into byte-level modification behavior, and apply it to the current seed to obtain the next generation seed;

[0019] Step 4.3: If neither step 4.1 nor step 4.2 is triggered or fails during execution, then revert to the original mutation mechanism and perform a rollback mutation on the current seed;

[0020] Step 5: Write the next generation seed obtained after multi-level mutation processing back to the seed directory as input for the main loop of the next round of fuzz testing; and optionally expand the seed knowledge base based on its execution results.

[0021] Compared with the prior art, the present invention has the following advantages:

[0022] 1. This invention constructs a seed knowledge base oriented towards the target program, and uses metadata, lexical features, structural features, and coverage features as core indexing information to structurally organize existing high-quality seeds. With the support of this knowledge base, coverage-enhancing samples and crash-triggered samples accumulated during fuzzing can be used as searchable and reusable resources, providing a structured reference for subsequent seed mutations and alleviating the problem of insufficient utilization of historical high-quality seeds in existing methods.

[0023] 2. This invention designs a multi-level mutation mechanism consisting of direct splicing mutation, retrieval-enhanced mutation, and rollback mutation, and integrates it into the mutation process at the bottleneck stage of fuzz testing. When the growth of fuzz test coverage tends to stagnate, direct splicing is preferentially performed using reference seeds in the knowledge base. If necessary, a structured mutation operation description is generated under the constraints of the retrieval context by combining a large language model. If the above-mentioned enhanced mutation is not triggered or fails to execute, the original mutation mechanism is rolled back, thereby improving the targeting of mutations while maintaining the continuity of the main fuzz test loop.

[0024] 3. This invention introduces a context construction method based on a retrieval-enhanced generation mechanism during the large language model-assisted mutation process. The retrieved reference seed information and input format constraints are injected into the prompt word context. The structural information provided by the retrieval results can, to some extent, compensate for the large language model's insufficient grasp of binary input structural constraints, making the generated mutation strategy more structurally targeted and executable, thereby improving the fuzzing test's ability to trigger crashes on complex target programs. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall process of the RAG-based binary program fuzzing method.

[0026] Figure 2 A schematic diagram illustrating the process of constructing and indexing the seed knowledge base.

[0027] Figure 3 This is a schematic diagram of the RAG-based multi-level seed mutation process during the bottleneck stage. Detailed Implementation

[0028] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0029] This invention provides a binary program fuzzing method based on RAG. First, features are extracted and indexed from existing high-quality seeds that can improve program coverage or trigger program crashes, resulting in a seed knowledge base for the target program containing metadata, lexical features, structural features, and coverage features. Then, a feedback-driven fuzzing tool is used to execute the main fuzzing loop, detecting whether the fuzzing has reached a bottleneck stage. In the bottleneck stage, multi-level mutation processing is performed on the non-elite seeds to be mutated: sequentially determining whether to use direct concatenation mutation, retrieval-enhanced mutation, or rollback mutation. Finally, the next-generation seed obtained after multi-level mutation processing is written back to the seed directory, entering the next round of the main fuzzing loop. Figure 1 As shown, the specific steps include the following:

[0030] Step 1: Construct a seed knowledge base for the target program, such as... Figure 2 As shown, the specific steps are as follows:

[0031] Step 1.1: Collect existing seeds. The sources of existing seeds include, but are not limited to, sets of seeds that can improve coverage and sets of seeds that trigger program exceptions, preserved in the output directory during historical fuzzing.

[0032] Step 1.2: Extract metadata for each seed. The metadata includes, but is not limited to, target program name, seed source category, byte length, and input file type. The seed source category is used to distinguish between coverage boosting samples and crash triggering samples.

[0033] Step 1.3: Extract lexical features for each seed. The lexical features include, but are not limited to, the proportion of printable characters, the proportion of null bytes, and the hexadecimal representation of the header bytes of a preset length.

[0034] Step 1.4: Extract structural features for each seed. The structural features include, but are not limited to, the global structural fingerprint information of the seed and the structural vector constructed based on the structural fingerprint; establish a structural similarity retrieval library for the structural vectors according to the target program and file type, and maintain the corresponding index mapping.

[0035] Step 1.5: Extract coverage features for each seed. Specifically, the coverage acquisition tool, which is paired with the feedback-driven fuzzing tool, is invoked to execute a controlled target program once for a single seed, and the set of execution coverage edges generated during the execution is parsed into an integer set as the coverage features of that seed; the coverage vector index and its mapping are further constructed as needed.

[0036] Step 1.6: Write metadata, lexical features, structural features, and coverage features into the index file, and organize the index file, structural similarity retrieval library, and corresponding mappings into a seed knowledge base file package; when loading the seed knowledge base, restore the grouped index according to the target program and file type; when the structural similarity retrieval library and its mapping exist, enable the retrieval method based on structural similarity retrieval; otherwise, degenerate into random sampling grouped by target program and file type and scoring and filtering based on seed header prefix similarity.

[0037] Step 2: Execute the main fuzzing loop based on a feedback-driven fuzzing tool. The specific steps are as follows:

[0038] Step 2.1: Perform preprocessing on the target binary program to extract heuristic information related to the input parsing process. The heuristic information includes, but is not limited to, the set of constants corresponding to comparison instructions and the basic block weights obtained based on control flow modeling. The heuristic information is used to assist in the selection of seed mutation direction and fitness evaluation in the main loop of fuzzing test.

[0039] Step 2.2: For each seed to be executed in the seed queue, drive the target program to perform dynamic tracking execution, record the running trajectory and return results; if the seed triggers a program crash, save it as a crash sample and mark it as a special seed; if the seed covers a new basic block, mark it as a special seed; otherwise, calculate the fitness of the seed by combining the basic block weights in the heuristic information and the execution trajectory.

[0040] Step 2.3: Sort and select seeds according to fitness to obtain a set of candidate parents for the next generation of seeds, and perform crossover and mutation operations on the candidate parent set according to the preset genetic operators, and write the results back to the seed directory.

[0041] Step 3: In the main loop of fuzzing, detect whether the fuzzing test has entered the bottleneck stage. The criteria for determining the bottleneck stage include, but are not limited to: the program coverage has not increased within a preset number of generations, no new crash samples have been found within a preset observation window, or the fitness-based regular mutation has not brought effective benefits in terms of coverage or crashes within a preset number of generations. When the determination result is that the bottleneck stage has been entered, perform the multi-level mutation processing in step 4 on the non-elite seeds in the seed directory.

[0042] Step 4: During the bottleneck stage, perform multi-level mutation processing sequentially on non-elite seeds in the seed directory, such as... Figure 3 As shown, the specific steps are as follows:

[0043] Step 4.1: Direct splicing and mutation of the reference seed.

[0044] Step 4.1.1: Using the current seed to be mutated as the query, obtain a set of candidate reference seeds from the seed knowledge base according to its corresponding target program and file type. When the structural similarity retrieval library is available, map the current seed to be mutated into a structural vector and then perform a nearest neighbor search in the structural similarity retrieval library; otherwise, randomly select candidate reference seeds by grouping according to target program and file type, and score and filter the selected candidate seeds according to the similarity of the seed header prefix.

[0045] Step 4.1.2: Determine whether there are any usable reference seeds in the seed knowledge base. If the candidate reference seed set is not empty and meets the preset triggering conditions, then perform direct splicing mutation on the current seed to be mutated and at least one reference seed. The direct splicing mutation includes: selecting several byte fragments from the reference seeds and inserting, replacing or appending them to several positions of the current seed to be mutated to obtain the next generation seed; and using the next generation seed as the output of this multi-level mutation process.

[0046] Step 4.2: Auxiliary mutation based on the large language model generated by retrieval enhancement.

[0047] Step 4.2.1: If step 4.1 is not triggered, determine whether to call the large language model according to the preset large language model mutation trigger probability, and verify whether the number of large language model calls accumulated in this round of fuzz testing has not reached the preset call budget limit; if both are satisfied, proceed to step 4.2.2, otherwise proceed to step 4.3.

[0048] Step 4.2.2: Construct the prompt context. The prompt context adopts a combination of system constraint prompts and task context prompts. The system constraint prompts are used to limit the roles, output formats, and available mutation operation sets of the large language model. The task context prompts are used to inject information related to the current seed to be mutated, information related to the reference seed retrieved from the seed knowledge base, and an input format description corresponding to the target program.

[0049] Step 4.2.3: Input the context of the prompt word into the large language model to obtain the structured mutation operation description. The structured mutation operation description is given in a pre-agreed format, including but not limited to mutation operations such as byte insertion, byte deletion, byte replacement, field pruning, and field reuse, and their effect offsets.

[0050] Step 4.2.4: Parse the structured mutation operation description into byte-level modification behavior, apply it to the current seed to be mutated, and obtain the next generation seed; increment the number of large language model calls by 1.

[0051] Step 4.3: Rollback mutation.

[0052] If neither step 4.1 nor step 4.2 is triggered or fails during execution, a rollback mutation is performed on the current seed to be mutated. The rollback mutation includes: calling the basic mutation operator to perform a regular mutation operation, and further modifying the regular mutation result based on the key byte information obtained from dynamic taint analysis to obtain the next generation seed.

[0053] Step 5: Use the next-generation seed set obtained after multi-level mutation processing as the input for the next round of fuzzing main loop. Repeat steps 2 to 4 until the preset fuzzing termination conditions are met. The termination conditions include, but are not limited to, the runtime reaching a preset upper limit and the number of generations of the fuzzing main loop reaching a preset upper limit. During the fuzzing process, features of new seeds that meet the preset quality conditions can be extracted and written into the seed knowledge base as described in step 1, thereby expanding the seed knowledge base.

[0054] Example:

[0055] In this embodiment, the feedback-driven fuzzing tool employs a coverage-guided fuzzer that combines static analysis and dynamic taint analysis; the large language model uses a general-purpose large language model with reasoning capabilities and is invoked via an application programming interface; the structural similarity retrieval library is implemented using a nearest neighbor retrieval library; the bottleneck stage determination condition is set to the program coverage not increasing within several consecutive generations; the large language model mutation trigger probability and the large language model call budget limit are set according to the test duration and computing resources.

[0056] It should be noted that the feedback-driven fuzzy testing tools, large language models, structural similarity search libraries, and specific parameter settings used in the above embodiments are all illustrative and not intended to limit the scope of protection of this invention. Those skilled in the art can adjust or equivalently replace the tools, models, and parameters according to the specific circumstances such as the input format of the target program, computing resources, and the scale of the test task.

Claims

1. A binary program fuzzing method based on RAG, characterized in that... The method includes the following steps: Step 1: Construct a seed knowledge base for the target program. The seed knowledge base takes existing seeds that can improve program coverage or trigger program crashes as objects. Record metadata, lexical features, structural features and coverage features for each seed, and build grouped indexes according to the target program and file type. Step 2: Execute the main loop of fuzzing based on the feedback-driven fuzzing tool. The main loop of fuzzing includes executing the target program on the seeds in the seed queue, evaluating the fitness based on the execution trajectory and coverage feedback, determining whether to trigger a crash or cover a new basic block, and selecting and mutating the seeds based on the fitness. Step 3: In the main loop of fuzzing, detect whether the fuzzing test has entered the bottleneck stage. The criteria for determining the bottleneck stage include, but are not limited to: the program coverage has not increased within a preset number of generations, no new crash samples have been found within a preset observation window, or the fitness-based regular mutation has not brought effective benefits in terms of coverage or crashes within a preset number of generations. Step 4: During the bottleneck stage, perform multi-level mutation processing sequentially on the non-elite seeds in the seed directory: Step 4.1: Determine whether there are any usable reference seeds in the seed knowledge base. If there are, and the triggering conditions are met, then use the reference seeds as input to perform direct splicing and mutation of the reference seeds to obtain the next generation seeds. Step 4.2: If step 4.1 is not triggered, determine whether to call the large language model according to the preset probability, and verify whether the large language model call budget has not reached the upper limit; If the calling conditions are met, the prompt word context is constructed based on the current seed state, the retrieved reference seed information, and the input format constraints. The large language model outputs a structured mutation operation description, which is then parsed into byte-level modification behavior and applied to the current seed to obtain the next generation seed. Step 4.3: If neither step 4.1 nor step 4.2 is triggered or fails during execution, then revert to the original mutation mechanism and perform a rollback mutation on the current seed; Step 5: Write the next generation seed obtained after multi-level mutation processing back to the seed directory as the input for the main loop of the next round of fuzz testing; The seed knowledge base can be optionally expanded based on its execution results.

2. The RAG-based binary program fuzzing method according to claim 1, characterized in that... The specific steps of step 1 are as follows: Step 1.1: Collect existing seeds; Step 1.2: Extract metadata for each seed; Step 1.3: Extract lexical features for each seed; Step 1.4: Extract structural features for each seed; Step 1.5: Extract coverage features for each seed; Step 1.6: Write metadata, lexical features, structural features, and coverage features into the index file, and organize the index file, structural similarity search library, and corresponding mappings into a seed knowledge base file package; when loading the seed knowledge base, restore the grouped index according to the target program and file type; When a structural similarity retrieval library and its mapping exist, the retrieval method based on structural similarity retrieval is enabled; otherwise, it degenerates into a random sampling method grouped by target program and file type and a scoring and filtering method based on seed header prefix similarity.

3. The RAG-based binary program fuzzing method according to claim 1, characterized in that... The specific steps of step 2 are as follows: Step 2.1: Perform preprocessing on the target binary program to extract heuristic information related to the input parsing process; Step 2.2: For each seed to be executed in the seed queue, drive the target program to perform dynamic tracking execution, and record the running trajectory and return results; If a seed triggers a program crash, it is saved as a crash sample and marked as a special seed; if a seed overwrites a new basic block, it is marked as a special seed; otherwise, the fitness of the seed is calculated by combining the basic block weights in the heuristic information with the execution trajectory. Step 2.3: Sort and select seeds according to fitness to obtain a set of candidate parents for the next generation of seeds, and perform crossover and mutation operations on the candidate parent set according to the preset genetic operators, and write the results back to the seed directory.

4. The RAG-based binary program fuzzing method according to claim 1, characterized in that... In step 2.1, heuristic information is used to assist in the selection of seed mutation direction and fitness evaluation in the main loop of fuzzing test, including but not limited to the set of constants corresponding to comparison instructions and the basic block weights obtained based on control flow modeling.

5. The RAG-based binary program fuzzing method according to claim 1, characterized in that... The specific steps of step 4.1 are as follows: Step 4.1.1: Using the current seed to be mutated as the query, obtain a set of candidate reference seeds from the seed knowledge base according to its corresponding target program and file type. When the structural similarity retrieval library is available, map the current seed to be mutated into a structural vector and then perform a nearest neighbor search in the structural similarity retrieval library; otherwise, randomly select candidate reference seeds by grouping them according to the target program and file type, and score and filter the selected candidate seeds according to the similarity of the seed header prefix. Step 4.1.2: Determine whether there are any usable reference seeds in the seed knowledge base. If the candidate reference seed set is not empty and meets the preset triggering conditions, perform direct splicing mutation on the current seed to be mutated and at least one reference seed to obtain the next generation seed. Use the next generation seed as the output of this multi-level mutation process.

6. The RAG-based binary program fuzzing method according to claim 4, characterized in that... In step 4.1.2, the direct splicing mutation includes: selecting several byte fragments from the reference seed and inserting, replacing or appending them to several positions in the current seed to be mutated.

7. The RAG-based binary program fuzzing method according to claim 1, characterized in that... The specific steps of step 4.2 are as follows: Step 4.2.1: If step 4.1 is not triggered, determine whether to call the large language model according to the preset large language model mutation trigger probability, and verify whether the number of large language model calls accumulated in this round of fuzz testing has not reached the preset call budget limit; If all conditions are met, proceed to step 4.2.2; otherwise, proceed to step 4.

3. Step 4.2.2: Construct the context of the prompt words; Step 4.2.3: Input the context of the prompt word into the large language model to obtain the structured mutation operation description; Step 4.2.4: Parse the structured mutation operation description into byte-level modification behavior, apply it to the current seed to be mutated, and obtain the next generation seed; increment the number of large language model calls by 1.

8. The RAG-based binary program fuzzing method according to claim 7, characterized in that... In step 4.2.2, the context of the prompt word adopts a combination of system constraint prompts and task context prompts. The system constraint prompts are used to limit the role, output format and available mutation operation set of the large language model. The task context prompt is used to inject information related to the current seed to be mutated, information related to the reference seed retrieved from the seed knowledge base, and an input format description corresponding to the target program; in step 4.2.3, the structured mutation operation description is given in a pre-agreed format, including but not limited to mutation operations such as byte insertion, byte deletion, byte replacement, field pruning, and field reuse, and their effect offsets.

9. The RAG-based binary program fuzzing method according to claim 1, characterized in that... In step 4.3, the rollback mutation includes: calling the basic mutation operator to perform a regular mutation operation, and further modifying the regular mutation result based on the key byte information obtained from dynamic taint analysis to obtain the next generation seed.

10. The RAG-based binary program fuzzing method according to claim 1, characterized in that... The specific steps of step 5 are as follows: the next generation seed set obtained after multi-level mutation processing is used as the input of the next round of fuzz test main loop, and steps 2 to 4 are repeated until the preset fuzz test termination condition is reached. The termination condition includes, but is not limited to, the runtime reaching the preset upper limit and the number of generations of the fuzz test main loop reaching the preset upper limit. During fuzz testing, features of new seeds that meet preset quality conditions can be extracted and written into the seed knowledge base as described in step 1, thereby expanding the seed knowledge base.

Images