Multi-machine collaborative vulnerability detection system based on vulnerability clustering and distance space partitioning

Abstract

A multi-machine collaborative vulnerability detection system based on vulnerability clustering and distance space partitioning is proposed. The system extracts potential vulnerabilities from the source code of the program under test through a vulnerability extraction and clustering module. An improved clustering algorithm is used to aggregate representative vulnerability points as baseline reference points. A fuzzing module performs fuzzing tests on the program. During the seed mutation phase, based on distance judgment and pre-calculated basic block-level distances, the currently executed seed (i.e., the comprehensive distance between the test input in the fuzzing test and the baseline reference point of that instance) is calculated, and appropriate energy is allocated based on the comprehensive distance. An information synchronization module performs asynchronous and transparent information synchronization among multiple fuzzing instances, collecting seeds from the seed queues of each instance, calculating the affinity of each seed to each instance, and assigning the seed to its actual fuzzing instance. This invention utilizes additional CPU resources, using some sensitive functions in the program as baseline reference points, and divides the program into multiple segments according to the distance space, allocating them to multiple fuzzing instances for multi-machine collaborative fuzzing, thereby improving the performance of fuzzing by utilizing computing resources.

Description

Technical Field

[0001] This invention relates to a technology in the field of information security, specifically a multi-machine collaborative vulnerability detection system based on vulnerability clustering and distance space partitioning. Background Technology

[0002] As a mainstream vulnerability detection method, fuzzing explores different paths within a program by continuously adjusting its input, aiming to discover vulnerabilities that can be exploited by malicious attackers, leading to program crashes, data leaks, and loss of control. However, as program size increases, traditional fuzzing methods struggle to cover all paths, resulting in high time consumption and low coverage. Summary of the Invention

[0003] This invention addresses the limitations of existing single-instance fuzzing techniques in meeting the vulnerability detection needs of large programs. Simply introducing reinforcement learning methods results in low testing efficiency and fails to consider the division of data flow and control flow within the program. Therefore, this invention proposes a multi-machine collaborative vulnerability detection system based on vulnerability clustering and distance space partitioning. Utilizing additional CPU resources, the system uses some sensitive functions in the program as reference points and divides the program into multiple segments based on distance space. These segments are then assigned to multiple fuzzing instances for multi-machine collaborative fuzzing, thereby improving fuzzing performance by leveraging computational resources.

[0004] This invention is achieved through the following technical solution:

[0005] This invention relates to a multi-machine collaborative vulnerability detection system based on vulnerability clustering and distance space partitioning, comprising: a vulnerability extraction and clustering module, a distance-oriented fuzzing module, and an asynchronous transparent information synchronization module. Specifically: the vulnerability extraction and clustering module extracts potential vulnerabilities of candidate vulnerabilities from the source code of the program under test, and aggregates representative vulnerabilities as benchmark reference points through an improved clustering algorithm; the fuzzing module performs fuzzing tests on the program, and during the seed mutation stage, calculates the currently executed seed (i.e., the comprehensive distance between the test input in the fuzzing test and the benchmark reference point of the instance) based on a pre-calculated basic block-level distance, and allocates appropriate energy according to the comprehensive distance; the information synchronization module performs asynchronous transparent information synchronization among multiple fuzzing instances, collects seeds from the seed queues of each instance, calculates the affinity of each seed to each instance, and assigns the seed to the fuzzing instance to which it actually belongs.

[0006] The aforementioned benchmark reference points are established by identifying the locations of all sensitive functions (vulnerabilities) in the source code of the program under test based on existing vulnerability-related knowledge and experience. The functions and basic blocks containing the vulnerabilities are located in the function call graph and control flow graph of the program, respectively. The reachable paths of each vulnerability are extracted, and the similarity between different reachable paths is analyzed. Similar paths are continuously merged until only a predetermined number of paths and vulnerabilities remain, which serve as benchmark reference points for subsequent fuzzing tests.

[0007] The existing vulnerability-related knowledge and experience mentioned above refer to the following: In the memory management design of the C language, users need to explicitly call the malloc and free functions to allocate and release memory, and then obtain the data stored in the memory block through the starting address of the memory block. Since users can access memory data through pointers, memory access-related vulnerabilities can easily occur in the program, such as buffer overflows caused by memory read / write exceeding the boundary, and use-after-free (UAF) vulnerabilities caused by not setting the pointer to null after memory release. The trigger points of these vulnerabilities are often the memory read / write related library functions built into the C language. This invention refers to these functions as sensitive functions.

[0008] The sensitive functions mentioned refer to common library functions that are prone to causing vulnerabilities, based on existing vulnerability research experience and conclusions. Taking the C language as an example, some memory read / write functions do not perform buffer boundary checks, such as memcpy, strcpy, and strncpy. The use of these functions may lead to buffer overflow vulnerabilities, allowing attackers to control the program execution flow and thus gain system control.

[0009] The reachable path refers to the set of all basic blocks traversed by the program from the entry point to the vulnerability during execution, reflecting the range of the vulnerability that can be reached within the program. When the reachable paths of two vulnerabilities are very similar, it means that the corresponding vulnerabilities are located in close program regions and should be merged and run in the same fuzz test instance.

[0010] The allocation refers to: assigning the clustered vulnerability points to each fuzzy test instance as a reference point, and each instance starting the test with the reference point as the center; each seed calculates the affinity between itself and each reference point based on the number and order of basic blocks visited and the distance to the reference point, and passes it to the instance to which the reference point with the highest affinity belongs, ensuring that the test scope of each instance is clearly defined.

[0011] The aforementioned seed mutation involves selecting one seed from the seed queue in each round of fuzz testing and mutating it to obtain multiple new test inputs. When a test input enables the program to access an edge that has not been explored before, the test input is considered valuable and is added to the seed queue as a new seed.

[0012] The vulnerability extraction and clustering module includes: a source code static scanning unit, a program instrumentation unit, a path analysis unit, and a clustering and merging unit. Specifically: the source code static scanning unit scans the program source code based on a pre-collected list of sensitive functions to obtain a list of vulnerabilities, including the sensitive functions and their locations; the program instrumentation unit inserts instrumentation at the intermediate language level based on the vulnerability locations to obtain the program's function call graph and control flow graph; the path analysis unit traverses the program's function call graph and control flow graph based on the basic blocks and functions where the vulnerabilities are located, performing reachable path discovery operations to obtain a set of reachable paths for each vulnerability; and the clustering and merging unit clusters and merges the vulnerability and its corresponding reachable paths based on similarity comparisons between reachable paths. After multiple iterations, a specified number of vulnerability sets are obtained, which are the benchmark reference point sets used in subsequent steps.

[0013] The distance-oriented fuzz testing module includes: an initialization unit, an initial seed selection unit, a seed energy allocation unit, a seed mutation unit, a program execution unit, and a seed scoring unit. Specifically: the initialization unit assigns a reference point to each fuzz test instance based on the vulnerability set output by the vulnerability extraction and clustering module, and instrumentes the program based on the reference point to obtain an instrumented program with basic block distance calculation logic; the initial seed selection unit randomly assigns the test cases included in the program to be tested to each instance according to the number of fuzz test instances, obtaining the initial seed set for each instance; and the seed energy allocation unit calculates the seed energy based on the distance from the seed to the reference point. Based on the overall distance, energy is allocated to the seeds, so that seeds farther from the reference point receive less mutation energy. The seed mutation unit determines the number of mutations based on the energy allocated to the seed by the seed energy allocation unit, and performs mutation operations on the program input corresponding to the seed to obtain a new set of test inputs. The program execution unit puts the new test inputs obtained by mutation into the instrumented program for execution, obtains the edge coverage corresponding to the test input and the overall distance to the reference point, and also records the program crash situation. The seed scoring unit evaluates the value of the test input based on the edge coverage corresponding to the test input and decides whether it can be added to the queue as a new seed.

[0014] The asynchronous and transparent information synchronization module includes a central scheduling node submodule and an auxiliary synchronization agent submodule. The central scheduling node submodule is unique in this system and is responsible for controlling and scheduling the seed synchronization process between all fuzzy test instances. It includes an initialization unit, a network I / O unit, and an affinity calculation unit. The initialization unit collects the calculated basic block distances from each fuzzy test instance and instrumentes the program based on the distance information from each basic block to each instance, resulting in an instrumented program that can calculate the affinity between the seed and different instances during execution. The network I / O unit is responsible for network I / O communication between the central scheduling node and the auxiliary synchronization agent, receiving new seeds to be synchronized and sending the calculated seeds to their respective instances. The affinity calculation unit calculates the instance to which the seed belongs based on the number and order of basic blocks accessed during program execution and the distance to each reference point. The number of auxiliary synchronization agent submodules is the same as the number of fuzz test instances. Each auxiliary synchronization agent corresponds to a fuzz test instance and acts as an agent for the seed synchronization behavior of that instance. This includes a sending unit and a receiving unit. The sending unit reads the seed queue of the fuzz test instance and sends the unsynchronized seeds to the central scheduling node according to the seed number read last time. The receiving unit receives the seeds belonging to this instance from the central scheduling node, places them in the synchronization directory, and waits for the fuzz test instance to execute.

[0015] Technical effect

[0016] This invention identifies potential vulnerabilities in a program based on existing experience, determines reachable paths to these vulnerabilities using the program's function call graph and control flow graph, and selects representative vulnerabilities from a large pool using a clustering algorithm. Multiple fuzzy testing instances are then initiated, with each instance selecting one or more vulnerabilities as baseline reference points. The program space is divided into multiple segments, and each fuzzy testing instance explores around its respective baseline reference point. An asynchronous and transparent information synchronization module facilitates collaborative exploration among multiple instances.

[0017] This invention utilizes additional computing resources to improve the efficiency of vulnerability detection, focusing on task division and resource allocation during multi-instance collaborative testing. Compared with existing technologies, it can achieve higher vulnerability detection performance when using the same number of instances, and can cover a larger program space in the same amount of time. Attached Figure Description

[0018] Figure 1 This is a logic block diagram of the system modules of the present invention;

[0019] Figure 2 A flowchart illustrating the process of vulnerability extraction and clustering;

[0020] Figure 3This is a flowchart illustrating the distance-guided fuzzy testing module.

[0021] Figure 4 A schematic diagram of the structure of an asynchronous transparent information synchronization system;

[0022] Figure 5 A flowchart illustrating the central scheduling node of the information synchronization module;

[0023] Figure 6 A flowchart illustrating the process of using a synchronization agent to assist the information synchronization module. Detailed Implementation

[0024] like Figure 1 As shown in this embodiment, a multi-machine collaborative vulnerability detection system based on vulnerability clustering and distance space partitioning is involved. This system triggers a crash in the program under test during multi-instance collaborative fuzzing. Based on the crash points provided in the fuzzing report and the input content at the time of triggering, the crash scenario is reproduced, and the stack data in memory at the time of the crash is analyzed to achieve vulnerability discovery. Specifically, this includes:

[0025] Step 1) Extract potential vulnerabilities from the source code of the program under test, and use LLVM intermediate language to generate the function call graph and control flow graph of the program. Obtain the reachable basic blocks, reachable functions and reachable paths of the vulnerabilities in the program. Integrate the above information and merge them into a small number of representative vulnerabilities as benchmark reference points through path-based clustering algorithm.

[0026] Step 2) Pre-calculate the distance from each basic block to the reference point, insert these distances into the program by instrumentation, and then select an initial seed to start the fuzzing process. During the actual execution of the program, the comprehensive distance from the seed to the reference point is dynamically calculated based on the distance from the basic block to the reference point, which is used to calculate the energy that the seed can obtain through subsequent mutations. During the fuzzing process, if the program crashes, the test input of the current execution will be recorded to a file.

[0027] Step 3) The central scheduling node collects the basic block distance information of each instance during compilation, receives candidate seeds sent by each instance at runtime, calculates the proximity of the seed to each instance, and forwards the candidate seed to its closest instance. By setting auxiliary synchronization agents for each instance, it sends unsynchronized seeds in the current seed queue to the central scheduling node and receives seeds from other instances sent by the central scheduling node. The entire synchronization process adopts an asynchronous exchange mechanism, which is completely transparent to the fuzz test instance and has strong availability and scalability.

[0028] like Figure 2As shown, step 1 specifically includes: static scanning of sensitive functions in the source code, instrumentation of the program under test to obtain function call graphs and control flow graphs, basic block and function reachability analysis, and vulnerability clustering and merging.

[0029] 1.1) Static scanning of sensitive functions in source code: Based on the pre-collected list of sensitive functions, static scanning of the source code of the program under test is performed to discover the sensitive functions and obtain a set of vulnerabilities.

[0030] 1.2) Instrumentation of the program under test to obtain function call graph and control flow graph: For each vulnerability in the vulnerability set, instrumentation is performed on the program under test to obtain the basic block and function where the vulnerability is located, and to generate the function call graph and control flow graph of the program under test.

[0031] The function call graph illustrates the calling relationships between various functions in the program. When one function calls another function, there will be a directed edge between the corresponding nodes of the two functions.

[0032] The control flow graph represents the flow relationship between basic blocks within a function, where an edge indicates that a preceding basic block can enter a subsequent basic block through a jump instruction.

[0033] 1.3) Basic Block and Function Reachability Analysis: Based on the position of the basic block and function where the vulnerability is located in the function call graph and control flow graph, the calling relationship between functions is determined. Reachability analysis is performed on all basic blocks in the program under test to obtain the basic block set of reachable vulnerability points, which serves as the reachable path of the vulnerability point and also as the feature representation of the vulnerability point in the clustering process.

[0034] The reachability analysis refers to the following: if a basic block can reach the basic block containing the vulnerability under a specific input, then the vulnerability is considered reachable from that basic block. In this embodiment, based on the function call graph and control flow graph, a breadth-first search method can be used, starting from the basic block. When the vulnerability is successfully explored, it indicates that the basic block is a reachable vulnerability.

[0035] 1.4) Vulnerability Clustering and Merging: Based on the reachable path of each vulnerability, perform path-based clustering and merging, continuously merging vulnerability points with similar reachable paths until the number of remaining vulnerability points drops to a predetermined threshold.

[0036] The distance-based clustering algorithm refers to: analyzing the reachable paths of all vulnerability points, merging relatively close vulnerability points according to the reachable basic block set of each vulnerability point, selecting the vulnerability points with more reachable basic blocks and deeper access levels as the retained vulnerability points, and repeating this process until the number of remaining vulnerability points reaches a predetermined threshold.

[0037] like Figure 3 As shown, step 2 includes: a pre-test preparation stage of selecting a reference point, compiling and instrumenting distance calculation logic, selecting the initial seed, and the actual test stage of seed energy allocation, seed mutation, executing the program to obtain distance information, and seed scoring, specifically:

[0038] 2.1) Selection of reference points: Select one or more vulnerability points from the vulnerability set obtained in step 1 as reference points for the current fuzz test instance, which are used to divide the program space belonging to the current instance.

[0039] 2.2) Compilation and Instrumentation Runtime Distance Calculation Logic: Based on the selected benchmark reference point, the program under test is compiled and instrumented. The shortest distance from each basic block to each vulnerability point contained in the benchmark reference point is obtained from the function call graph and control flow graph of the program. The average distance is taken as the distance from the basic block to the benchmark reference point. Then, additional code logic is inserted at the entry point of each basic block to add the distance from the current basic block to the benchmark reference point to a global variable. This global variable records the sum of the distances of the basic blocks accessed during the current program execution. Another global variable is set to record the sum of the number of basic blocks accessed during the current program execution. After the program execution is completed, the comprehensive distance of the basic blocks accessed during the current execution can be calculated based on these two variables, which will affect the subsequent seed energy allocation stage.

[0040] 2.3) Initial Seed Selection: Using the test input provided by the program under test, the initial seeds are randomly assigned to each fuzzing instance and added to the instance's seed queue. Each instance will first perform a basic test on each initial seed to initialize the various values ​​of the fuzzing test and construct the initial test state.

[0041] Generally speaking, the higher the quality of the initial seed in fuzzing, the easier it is to explore more complex and in-depth program paths. A high-quality initial seed can explore relatively deep paths at the very beginning of the test, while a poor and redundant initial seed will consume computing resources and reduce test efficiency.

[0042] Instrumentation refers to inserting additional code snippets into various nodes during program compilation to dynamically monitor program execution. This embodiment uses LLVM intermediate language for instrumentation, inserting logic to calculate the distance from the current basic block to the reference point at the entry point of each basic block. During execution, the distances corresponding to each basic block are accumulated, allowing the comprehensive distance to the reference point to be calculated in a single program execution.

[0043] The seed queue refers to the following: each instance maintains its own queue, which contains high-quality seeds encountered during the current test. All seeds in the queue together form a maximum coverage set that can cover all program paths accessed during the current test phase.

[0044] 2.4) Seed Energy Allocation: When an instance starts a new round of testing, a seed is first selected from the seed queue, and a certain amount of energy is allocated to it based on the seed's previous execution record. To limit the instance's testing scope to the vicinity of the benchmark reference point, if the seed's overall distance to the benchmark reference point was large during its last execution, the allocated energy will be attenuated to ensure that the test focuses more on seeds closer to the benchmark reference point.

[0045] 2.5) Seed Mutation: Based on the allocated energy, the seed is mutated to modify the corresponding test input, resulting in many new test cases. Seed mutation is divided into deterministic mutation and non-deterministic mutation. Deterministic mutation includes highly deterministic operations such as bit flipping, arithmetic operations, special value replacement, and dictionary replacement. Non-deterministic mutation includes highly random operations such as large-scale destruction and splitting and recombination. During testing, one instance is responsible for the deterministic mutation operation, while other instances only need to perform non-deterministic mutations on the seed.

[0046] 2.6) Obtaining Distance Information by Executing the Program: The new test cases obtained from seed mutation are used as input to execute the program under test. Using compile-time instrumentation, the program obtains the combined distance from basic blocks accessed during execution to the baseline reference point, as well as all paths accessed. If the program crashes during execution, it indicates a potential vulnerability in the paths accessed during this execution. The test cases used in this execution are recorded, and a simple program crash report is generated.

[0047] 2.7) Seed Scoring: After the program execution is complete, the value of the test cases used in this execution needs to be evaluated based on the execution results. When new paths are discovered during program execution, it indicates that the test case has value and needs to be added to the seed queue for subsequent mutation. The more new paths discovered in a single execution, the higher the score of the new seed. Newly added seeds also need to be synchronized to other fuzzing test instances. To decouple the instances, no information about other instances is recorded in the instance itself. Each new seed is sent to the central scheduling node by an auxiliary synchronization agent, and the central scheduling node calculates and analyzes whether it needs to be synchronized to other instances.

[0048] 2.8) Repeat steps 2.4 to 2.7 until the set test time is reached or the test is stopped by manual intervention.

[0049] like Figure 4As shown, step 3 involves two sub-modules: a central scheduling node and an auxiliary synchronization agent. The central scheduling node acts as the manager for information synchronization among all fuzzing instances. It establishes bidirectional data connections with all running instances, continuously receives newly generated seeds from the instances, calculates the closest fuzzing instance to the new seed, and sends the new seed to the corresponding instance. The auxiliary synchronization agent is bound to the fuzzing instance and acts as a proxy for communication between the fuzzing instance and the central scheduling node. It is responsible for completing the seed sending and receiving operations for that instance during the information synchronization process.

[0050] like Figure 5 As shown, the central scheduling node in step 3 specifically includes: initialization and seed synchronization loop.

[0051] 3.1) Central scheduling node initialization: Collects basic block distance information of all fuzz test instances, that is, the distance of each basic block in the program to the reference point. After determining the instance that each basic block tends to based on the distance, the program to be tested is compiled and instrumented so that all basic blocks accessed and the instances that each basic block tends to be recorded during program execution. The affinity between the current seed and each instance is calculated based on the execution results.

[0052] The affinity mentioned refers to the degree of closeness to each instance, calculated by weighting and summing the basic blocks accessed along the program execution path and the instances that each basic block favors. The deeper a basic block is in the path, the more representative it is of that path's affiliation, and thus it receives a greater weight. Furthermore, considering that loop statements may repeatedly access a certain basic block many times, making the affinity calculation strongly biased towards such basic blocks, the more times a basic block is accessed, the less weight it receives, ensuring fairness in the affinity calculation.

[0053] 3.2) Central scheduling node seed synchronization loop: The central scheduling node opens a specific port to continuously receive seeds to be synchronized from each instance. It uses the seeds as input to execute the instrumented program to be tested, calculates the affinity between the current seed and each instance, and forwards it to the instance with the highest affinity.

[0054] like Figure 6 As shown, the auxiliary synchronization agent in step 3 specifically includes: binding with the fuzz test instance, seed sending loop, and seed receiving loop.

[0055] 3.3) Binding the auxiliary synchronization agent to the fuzz test instance: Each fuzz test instance generates a unique ID when it starts up. The auxiliary synchronization agent needs to enter the ID of the corresponding instance when it starts up to complete the binding.

[0056] 3.4) Assisted Synchronization Agent Seed Sending Loop: Start the sending unit, continuously monitor the seed queue of the corresponding fuzzy test instance, maintain the maximum number of the currently synchronized seeds, and when a new seed with a number exceeding the current maximum appears in the seed queue, send the seed to the central scheduling node.

[0057] 3.5) Auxiliary Synchronization Agent Seed Reception Loop: Based on the ID of the corresponding fuzz test instance, listen to a specific port, receive seeds from the central scheduling node, add them to the fuzz test synchronization queue, and wait for the fuzz test instance to synchronize next time to execute.

[0058] Preferably, when fuzzing discovers test cases that can trigger program crashes, the existence of a vulnerability is determined through manual review. Specifically, the test cases that trigger program crashes are used as input, and the program under test is dynamically debugged using the gdb tool. The stack data in memory is observed to confirm whether a vulnerability exists at the crash point, and the exploitability and threat level of the vulnerability are analyzed.

[0059] Through specific experiments, libxml2 was used as the test program, and AFL and AFL-P were selected as reference methods. The experimental configuration was as follows: CPU: Intel(R) Xeon(R) CPUE5-2630v3@2.40GHz, memory: 188G, with 4 instances running in parallel for 24 hours. The final experimental data is shown in the table below:

[0060] method line coverage Function coverage Edge Coverage AFL 9505 517 7780 AFL-P 9840 525 8037 This method 11250 586 10009

[0061] The results show that this embodiment achieves a higher coverage rate than AFL and AFL-P in the same amount of time, whether it is row coverage, function coverage or edge coverage, which indicates that this method has higher performance in exploring the program space.

[0062] Compared to existing technologies, this method initially screens vulnerabilities based on a list of sensitive functions and then extracts the most representative set of vulnerabilities from a large number of vulnerabilities using a distance-based clustering algorithm. Based on these vulnerabilities, the program space is divided into multiple regions, each explored by different fuzzing instances. Each instance is tested within the range defined by its baseline reference point, reducing the overlap of seeds between instances and avoiding resource waste caused by duplicate tasks, effectively improving the efficiency of fuzzing. This method also improves testing efficiency by exchanging seeds among instances and calculating the proximity between seeds and instances based on the distance to basic blocks, thus assigning seeds to specific instances and enabling multiple fuzzing instances to work collaboratively. Finally, this method employs an asynchronous and transparent information synchronization mechanism, extracting the seed synchronization task from the fuzzing instances and having it completed by a central scheduling node and an auxiliary synchronization agent bound to each instance. This avoids the impact of network transmission latency on fuzzing performance and completely decouples the instances, resulting in strong scalability for the entire system.

[0063] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A multi-machine collaborative fuzzy testing system based on vulnerability point clustering and distance space partitioning, characterized in that, include: The system comprises three modules: a vulnerability extraction and clustering module, a distance-oriented fuzzing module, and an asynchronous transparent information synchronization module. Specifically: the vulnerability extraction and clustering module extracts potential vulnerabilities from the source code of the program under test, aggregating representative vulnerabilities using an improved clustering algorithm as baseline reference points; the fuzzing module performs fuzzing tests on the program, calculating the currently executed seed (i.e., the comprehensive distance between the test input and the baseline reference point of the instance) based on pre-calculated basic block-level distances during the seed mutation phase, and allocating appropriate energy accordingly; and the information synchronization module performs asynchronous transparent information synchronization among multiple fuzzing instances, collecting seeds from each instance's seed queue, calculating the affinity of each seed to each instance, and assigning the seed to its actual fuzzing instance. The distance-oriented fuzzy testing module includes: an initialization unit, an initial seed selection unit, a seed energy allocation unit, a seed mutation unit, a program execution unit, and a seed scoring unit. Specifically: the initialization unit assigns a reference point to each fuzzy test instance based on the vulnerability set output by the vulnerability extraction and clustering module, and instrumentes the program based on the reference point to obtain an instrumented program with basic block distance calculation logic; the initial seed selection unit randomly assigns the test cases included in the program to be tested to each instance according to the number of fuzzy test instances, obtaining the initial seed set for each instance; the seed energy allocation unit allocates the test cases based on the distance from the seed to the reference point... The seed energy allocation unit determines the number of mutations based on the energy allocated to the seed by the seed energy allocation unit, and performs mutation operations on the program input corresponding to the seed to obtain a new set of test inputs. The program execution unit executes the newly mutated test inputs into the instrumented program, obtains the edge coverage of the test input and the comprehensive distance to the reference point, and records the program crash status. The seed scoring unit evaluates the value of the test input based on the edge coverage and determines whether it can be added to the queue as a new seed.

2. The multi-machine collaborative fuzzy testing system based on vulnerable point clustering and distance space partitioning according to claim 1, characterized in that, The aforementioned benchmark reference points are established by identifying the locations of all sensitive functions (vulnerabilities) in the source code of the program under test based on existing vulnerability-related knowledge and experience. The functions and basic blocks containing the vulnerabilities are located in the function call graph and control flow graph of the program, respectively. The reachable paths of each vulnerability are extracted, and the similarity between different reachable paths is analyzed. Similar paths are continuously merged until only a predetermined number of paths and vulnerabilities remain, which serve as benchmark reference points for subsequent fuzzing tests.

3. The multi-machine collaborative fuzzy testing system based on vulnerable point clustering and distance space partitioning according to claim 1, characterized in that, The vulnerability extraction and clustering module includes: a source code static scanning unit, a program instrumentation unit, a path analysis unit, and a clustering and merging unit. Specifically: the source code static scanning unit scans the program source code based on a pre-collected list of sensitive functions to obtain a list of vulnerabilities; the program instrumentation unit inserts instrumentation at the intermediate language level based on the location of the vulnerabilities to obtain the program's function call graph and control flow graph; the path analysis unit traverses the program's function call graph and control flow graph based on the basic blocks and functions where the vulnerabilities are located, performing reachable path discovery operations to obtain a set of reachable paths for each vulnerability; and the clustering and merging unit clusters and merges the vulnerabilities and their corresponding reachable paths based on similarity comparisons between reachable paths. After multiple iterations, a specified number of benchmark reference points are obtained. The vulnerability list includes sensitive functions and their locations.

4. The multi-machine collaborative fuzzy testing system based on vulnerable point clustering and distance space partitioning according to claim 1, characterized in that, The asynchronous transparent information synchronization module includes: a central scheduling node submodule and an auxiliary synchronization agent submodule, wherein: the central scheduling node submodule controls and schedules the seed synchronization process between all fuzz test instances; the number of auxiliary synchronization agent submodules is the same as the number of fuzz test instances, and the auxiliary synchronization agent corresponds one-to-one with the fuzz test instance, acting as the agent for the seed synchronization behavior of the instance; The central scheduling node submodule includes an initialization unit, a network I / O unit, and a proximity calculation unit. The initialization unit collects the calculated basic block distances from each fuzzy test instance and instrumentes the program based on the distance information from each basic block to each instance, resulting in an instrumented program that calculates the proximity of the seed to different instances during execution. The network I / O unit is used for network I / O communication between the central scheduling node and the auxiliary synchronization agent, receiving new seeds to be synchronized and sending the calculated seeds to their respective instances. The proximity calculation unit calculates the instance to which the seed belongs based on the number and order of basic blocks accessed during program execution and the distance to each reference point. The auxiliary synchronization agent submodule includes a sending unit and a receiving unit, wherein: the sending unit reads the seed queue of the fuzz test instance and sends the unsynchronized seeds to the central scheduling node according to the seed number read last time; the receiving unit receives the seeds belonging to this instance from the central scheduling node, puts them in the synchronization directory, and waits for the fuzz test instance to be executed.

5. The multi-machine collaborative fuzzy testing system based on vulnerable point clustering and distance space partitioning according to any one of claims 1-4, characterized in that, The allocation refers to: assigning the clustered vulnerability points to each fuzzy test instance as a reference point, and each instance starting the test with the reference point as the center; each seed calculates the affinity between the seed and each reference point based on the number and order of the basic blocks visited and the distance to the reference point, and passes it to the instance to which the reference point with the highest affinity belongs, ensuring that the test scope of each instance is clearly defined. The aforementioned seed mutation involves selecting one seed from the seed queue in each round of fuzz testing and mutating it to obtain multiple new test inputs. When a test input enables the program to access an edge that has not been explored before, the test input is considered valuable and is added to the seed queue as a new seed.

6. A multi-machine collaborative fuzzy testing method, characterized in that, Based on the system described in any one of claims 1-5, vulnerability discovery is achieved by triggering a crash in the program under test during multi-instance collaborative fuzzing, reproducing the crash scenario based on the crash point provided in the fuzzing report and the input content at the time of triggering, and analyzing the stack data in memory at the time of the crash. Specifically, this includes: Step 1) Extract potential vulnerabilities from the source code of the program under test, and use LLVM intermediate language to generate the function call graph and control flow graph of the program. Obtain the reachable basic blocks, reachable functions and reachable paths of the vulnerabilities in the program. Integrate the above information and merge them into a small number of representative vulnerabilities as benchmark reference points through path-based clustering algorithm. Step 2) Pre-calculate the distance from each basic block to the reference point, insert these distances into the program by instrumentation, and then select an initial seed to start the fuzzing process. During the actual execution of the program, the comprehensive distance from the seed to the reference point is dynamically calculated based on the distance from the basic block to the reference point, which is used to calculate the energy that the seed can obtain through subsequent mutations. During the fuzzing process, if the program crashes, the test input of the current execution will be recorded to a file. Step 3) The central scheduling node collects the basic block distance information of each instance during compilation, receives candidate seeds sent by each instance at runtime, calculates the proximity of the seed to each instance, and forwards the candidate seed to its closest instance. By setting auxiliary synchronization agents for each instance, it sends unsynchronized seeds in the current seed queue to the central scheduling node and receives seeds from other instances sent by the central scheduling node. The entire synchronization process adopts an asynchronous exchange mechanism, which is completely transparent to the fuzz test instance and has strong availability and scalability.

7. The multi-machine collaborative fuzzy testing method according to claim 6, characterized in that, Step 1 specifically includes: 1.1) Static source code scanning for sensitive functions: Based on a pre-collected list of sensitive functions, static source code scanning is performed on the program under test to discover sensitive functions and obtain a set of vulnerabilities; 1.2) Instrumentation of the program under test to obtain function call graph and control flow graph: For each vulnerability in the vulnerability set, instrumentation is performed on the program under test to obtain the basic block and function where the vulnerability is located, and to generate the function call graph and control flow graph of the program under test; 1.3) Basic block and function reachability analysis: Based on the position of the basic block and function where the vulnerability is located in the function call graph and control flow graph, the calling relationship between functions is determined. Reachability analysis is performed on all basic blocks in the program under test to obtain the basic block set of reachable vulnerability points, which serves as the reachable path of the vulnerability point and also as the feature representation of the vulnerability point in the clustering process; 1.4) Vulnerability Clustering and Merging: Based on the reachable path of each vulnerability, perform path-based clustering and merging, continuously merging vulnerability points with similar reachable paths until the number of remaining vulnerability points drops to a predetermined threshold. The distance-based clustering algorithm refers to: analyzing the reachable paths of all vulnerability points, merging relatively close vulnerability points according to the reachable basic block set of each vulnerability point, selecting the vulnerability points with more reachable basic blocks and deeper access levels as the retained vulnerability points, and repeating this process until the number of remaining vulnerability points reaches a predetermined threshold.

8. The multi-machine collaborative fuzzy testing method according to claim 6, characterized in that, Step 2 includes: 2.1) Reference point selection: Select one or more vulnerability points from the vulnerability point set obtained in step 1 as reference points for the current fuzz test instance, which are used to divide the program space belonging to the current instance; 2.2) Compilation and instrumentation runtime distance calculation logic: Based on the selected benchmark reference point, the program to be tested is compiled and instrumented. The shortest distance from each basic block to each vulnerability point contained in the benchmark reference point is obtained from the function call graph and control flow graph of the program. The average distance is taken as the distance from the basic block to the benchmark reference point. Then, additional code logic is inserted at the entry point of each basic block to add the distance from the current basic block to the benchmark reference point to a global variable. This global variable records the sum of the distances of the basic blocks accessed during the current program execution. Another global variable is set to record the sum of the number of basic blocks accessed during the current program execution. After the program execution is completed, the comprehensive distance of the basic blocks accessed during the current execution is calculated based on these two variables, which will affect the subsequent seed energy allocation stage. 2.3) Initial Seed Selection: The test input provided by the program under test is randomly assigned to each fuzz test instance as the initial seed for fuzz testing and added to the instance's seed queue; each instance will first perform a basic test on each initial seed to initialize the various values ​​of the fuzz test and build the initial test state. 2.4) Seed Energy Allocation: When an instance starts a new round of testing, a seed is first selected from the seed queue, and a certain amount of energy is allocated to it based on the record of the seed's last execution. In order to limit the test scope of the instance to the vicinity of the benchmark reference point, if the seed's comprehensive distance to the benchmark reference point was large during the last execution, the energy allocated this time will be attenuated to a certain extent, so that the test will focus more on the seeds that are closer to the benchmark reference point. 2.5) Seed Mutation: Based on the allocated energy, the seed is mutated to modify the corresponding test input, resulting in many new test cases. Seed mutation is divided into deterministic mutation and non-deterministic mutation. Deterministic mutation includes highly deterministic operations such as bit flipping, arithmetic operations, special value replacement, and dictionary replacement. Non-deterministic mutation includes two highly random operations: large-scale destruction and splitting and recombination. During the testing process, one instance is responsible for the deterministic mutation operation, while other instances only need to perform non-deterministic mutation on the seed. 2.6) Obtain distance information by executing the program: The new test cases obtained from seed mutation are used as program input to execute the program under test. With the help of the code logic of compile-time instrumentation, the comprehensive distance from the basic block accessed by the program to the benchmark reference point and all paths accessed are obtained. If the program crashes during execution, it indicates that there may be a vulnerability in the path accessed during this execution. The test cases used in this execution will be recorded and a simple program crash report will be generated. 2.7) Seed Scoring: After the program execution is complete, the value of the test cases used in this execution needs to be evaluated based on the execution situation. When a new path is discovered during program execution, it indicates that the test case has value and needs to be added to the seed queue for subsequent mutation. The more new paths discovered in one execution, the higher the score of the new seed. The newly added seeds in the queue also need to be synchronized to other fuzz test instances. To decouple between instances, instances do not record information of other instances. Each new seed is sent to the central scheduling node by the auxiliary synchronization agent, and the central scheduling node calculates and analyzes whether it needs to be synchronized to other instances. 2.8) Repeat steps 2.4 to 2.7 until the set test time is reached or the test is stopped by manual intervention.

9. The multi-machine collaborative fuzzy testing method according to claim 6, characterized in that, Step 3 specifically includes: initialization and seed synchronization loop; 3.1) Central scheduling node initialization: Collect the basic block distance information of all fuzz test instances, that is, the distance of each basic block in the program to the reference point. After determining the instance that each basic block tends to based on the distance, the program to be tested will be compiled and instrumented so that all basic blocks accessed and the instances that each basic block tends to are recorded during program execution. The affinity between the current seed and each instance is calculated based on the execution results. The affinity refers to the degree of affinity with each instance obtained by weighted summation based on the basic blocks accessed along the program execution path and the instances that each basic block tends to. The deeper a basic block is on the path, the more it represents the belonging of that path and will receive a greater weight. In addition, considering that loop statements may repeatedly access a certain basic block many times, making such basic blocks strongly biased in the affinity calculation, the more times each basic block is accessed, the less weight it receives, ensuring the fairness of affinity calculation. 3.2) Central scheduling node seed synchronization loop: The central scheduling node opens a specific port to continuously receive seeds to be synchronized from each instance. It uses the seeds as input to execute the instrumented program to be tested, calculates the affinity between the current seed and each instance, and forwards it to the instance with the highest affinity. 3.3) Binding the auxiliary synchronization agent to the fuzz test instance: Each fuzz test instance generates a unique ID upon startup. The auxiliary synchronization agent needs to input the corresponding instance ID upon startup to complete the binding. 3.4) Assisted Synchronization Agent Seed Sending Loop: Start the sending unit, continuously monitor the seed queue of the corresponding fuzzy test instance, maintain the maximum number of the currently synchronized seeds, and when a new seed with a number exceeding the current maximum appears in the seed queue, send the seed to the central scheduling node. 3.5) Auxiliary Synchronization Agent Seed Reception Loop: Based on the ID of the corresponding fuzz test instance, listen to a specific port, receive seeds from the central scheduling node, add them to the fuzz test synchronization queue, and wait for the fuzz test instance to synchronize next time to execute.

Images