Knowledge base poisoning based security evaluation method and system for multi-agent system

Abstract

This invention discloses a knowledge base-based security assessment method and system for multi-agent systems, comprising: mathematical modeling and structural analysis of the target multi-agent system; constructing a poisoning sample and injecting it into an external knowledge base at a low ratio; constructing a joint optimization objective function that integrates uniqueness, compactness, and coherence constraints, and optimizing and generating triggers that can induce retrieval behavior to hit the poisoning sample; inputting a test task integrating the trigger into the target multi-agent system, and performing full-link monitoring and logging of each agent; comparing and quantifying the retrieval offset of the target multi-agent system under normal input and triggered input, tracing and reconstructing the risk propagation path triggered by the offset based on the collaborative topology graph; quantifying the system risk level based on the retrieval offset rate and code risk rate, and finally outputting a security assessment report. This invention, by constructing a proactive testing and quantitative analysis mechanism, can achieve controllable and realistic security assessment.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence system security technology, specifically relating to a knowledge base-based security assessment method and system for multi-agent systems. Background Technology

[0002] With the increasing capabilities of large-scale pre-trained language models, multi-agent systems based on Large Language Models (LLMs) are gradually becoming a key technology in software development processes. These systems typically involve multiple functionally independent but task-related agents collaborating to complete complex software engineering tasks, including requirements planning, code retrieval, code generation, testing and verification, and final result integration. Compared to traditional single-model generation methods, multi-agent systems, through a "division of labor-collaboration-feedback" mechanism, can significantly improve code quality and engineering capabilities, and have been widely applied in coding assistants, intelligent API generation platforms, and enterprise-level automated R&D systems.

[0003] Existing multi-agent code generation systems generally rely on structured knowledge sources, such as code template libraries, sample code libraries, API documentation libraries, project commit history libraries, and RAG (Retrieval-Augmented Generation) knowledge bases based on vector retrieval. These knowledge bases provide agents with semantic context, development specifications, and reference implementations, playing a crucial role in improving the correctness, consistency, and maintainability of generated code. However, because knowledge bases often allow dynamic updates, and some enterprise environments have mechanisms such as external document synchronization, collaborator editing, and automated script writing, their content is potentially at risk of being contaminated.

[0004] Specifically, when a knowledge base is infiltrated with biased examples, code snippets with insecure patterns, weakly secure interface implementations, or example documents with special triggering characteristics, the retrieval agent may preferentially target these anomalous contents under certain conditions. Because LLMs are highly imitative when generating code based on examples, the code generation agent directly inherits these biased patterns, resulting in code with logical errors, permission bypasses, missing input validation, or other potential security risks. In multi-agent workflows, since the test agent and integration agent typically rely on template-based or automatically generated validation logic, anomalies may not be captured in a timely manner, causing risks to amplify along the agent chain and ultimately appear in the output code.

[0005] Current security detection technologies primarily focus on detecting injection attacks in single-model scenarios, quality control of model generation, or red team testing. They are unable to comprehensively detect system-level risks such as "knowledge base contamination → retrieval offset → multi-agent cascade offset → code leakage." Furthermore, existing technologies lack effective methods for controllably simulating contamination, constructing trigger inputs, recording multi-agent execution chains, and analyzing offset propagation paths. This makes it difficult to identify and quantify latent risks, cross-agent diffusion mechanisms, and critical vulnerabilities within the system.

[0006] Therefore, there is an urgent need for a system-level security assessment method that can simulate knowledge base pollution in a controllable manner, construct triggering scenarios, monitor the execution behavior of multiple agents, and analyze the deviation propagation path without affecting the normal function of the system, so as to comprehensively identify potential risks in code-generating multi-agent systems and provide effective defense strategies. Summary of the Invention

[0007] In view of the above, the purpose of this invention is to provide a knowledge base poisoning-based security assessment method and system for multi-agent systems. This method achieves controllable and realistic security assessment by constructing an active testing and quantitative analysis mechanism. Specifically, poisoned samples are injected into the knowledge base at a low ratio, and trigger inputs are constructed using constraint optimization methods to activate anomalous content in natural tasks. Simultaneously, the execution process of the multi-agent system is logged and differentially analyzed to reconstruct the risk propagation path. Furthermore, retrieval offset rate and code risk rate indicators are introduced to quantitatively assess the security level. Finally, standardized assessment results and defense recommendations are output, providing a reliable basis for the auditing and deployment of multi-agent code generation systems.

[0008] To achieve the above-mentioned objectives, the present invention provides the following technical solution: In a first aspect, embodiments of the present invention provide a security assessment method for knowledge-based poisoning of multi-agent systems, comprising the following steps: Mathematical modeling and structural analysis of the target multi-agent system are performed to obtain the set of agents, the collaborative topology graph constructed based on the message passing relationship between agents, and the external knowledge base on which it depends. Construct poisoned samples containing potential vulnerabilities, example description text, and text feature vectors, and inject poisoned samples of less than or equal to a preset proportion into an external knowledge base to form a contaminated knowledge base; A joint optimization objective function integrating uniqueness, compactness, and coherence constraints is constructed, and a gradient-guided search strategy is used for optimization to generate triggers that can induce retrieval behavior to hit the poisoned sample. The test task with integrated triggers is input into the target multi-agent system to drive the multi-agents to cooperate in execution, and the input, output and key behaviors of each agent are monitored and logged in the whole link. Based on log records, the retrieval offset of the target multi-agent system under normal and triggered inputs is compared and quantified. Based on the collaborative topology graph, the risk propagation path triggered by the offset is tracked and reconstructed. The system risk level is quantified based on the retrieval offset rate and code risk rate, and the final output is a security assessment report that includes the risk level, a visualized propagation path, and targeted defense recommendations.

[0009] Preferably, the target multi-agent system includes at least: a planning agent responsible for parsing user intent and breaking down task steps; a retrieval agent responsible for obtaining context information from an external knowledge base; a code generation agent responsible for generating specific code logic based on context information; a testing agent responsible for performing static analysis or unit testing on the generated code; and an integration agent responsible for summarizing the processing results of each agent and generating the final system deliverable; and a collaborative topology graph is constructed by treating each agent as a node and the message passing relationship between agents as directed edges.

[0010] Preferably, the construct includes a poisoning sample comprising a potential vulnerability, example descriptive text, and text feature vectors, comprising: The potential vulnerability is constructed as a code snippet containing a hidden SQL injection vulnerability or command execution vulnerability. Example description text is written for this code snippet to suit its camouflage purpose, and the example description text is encoded into a text feature vector, thus forming a poisoned sample containing a triple structure of potential vulnerability, example description text, and text feature vector.

[0011] Preferably, the external knowledge base upon which the target multi-agent system relies includes at least: a vector retrieval library, an unstructured document library, and a code template library; wherein, a poisoning sample of less than or equal to a preset proportion is injected into the vector retrieval library of the external knowledge base to ensure the lightweight and covert nature of the poisoning behavior.

[0012] Preferably, the construction of a joint optimization objective function integrating constraints of uniqueness, compactness, and coherence, optimized using a gradient-guided search strategy, and generating a trigger that can induce retrieval behavior to hit the poisoned sample, includes: A uniqueness loss is constructed by maximizing the distance between the cluster centers of the trigger input vector and the normal query input vector; A compact loss is constructed by minimizing the distance between the trigger input vector and the poisoning sample vector in the injected knowledge base; The perplexity of the trigger text is calculated by using a pre-trained language model, and a penalty term is constructed based on the perplexity to limit the text generation space, thus constructing a coherence loss. The uniqueness loss, compactness loss, and coherence loss are weighted and fused into a joint optimization objective function. The joint optimization objective function is iteratively solved using gradient-guided beam search or Gumbel-Softmax relaxation techniques until convergence, and the final natural language trigger is output.

[0013] Preferably, the step of comparing and quantifying the retrieval offset of the target multi-agent system under normal and triggered inputs based on log records, and tracing and reconstructing the risk propagation path triggered by the offset according to the cooperative topology graph, includes: Two sets of tasks are executed in parallel: normal prompt word input and test prompt word input of integrated trigger. The overlap rate of the results returned by the retrieval agent in the two sets of tasks is calculated and compared to quantify the retrieval behavior offset. If the retrieval behavior offset exceeds the preset statistical significance threshold, it is determined that the retrieval agent has retrieval offset. The code generation agent is also analyzed to see if it generates code containing the characteristics of the poisoning sample vulnerability under the trigger input. Based on the collaborative topology graph between agents, if a retrieval offset leads to code risk and the test agent fails to detect it, a complete risk propagation path is marked from the retrieval agent through the code generation agent to the test agent.

[0014] Preferably, the quantification of system risk level based on retrieval offset rate and code risk rate includes: The retrieval offset rate is defined as the proportion of the number of tests in which the retrieval results contain the poisoned sample out of the total number of tests during the trigger test. Code risk rate is defined as the proportion of the number of tests in the triggered tests in which the final generated code is detected by static security tools to contain potential vulnerabilities out of the total number of tests. The system security risk score is calculated by combining the retrieval offset rate, code risk rate and risk propagation path length using a weighted function, and the risk level is determined based on a preset threshold range.

[0015] Secondly, embodiments of the present invention also provide a knowledge base poisoning-based security assessment system for multi-agent systems, which is implemented using the above-mentioned knowledge base poisoning-based security assessment method for multi-agent systems, including: a structure parsing module, a sample injection module, a trigger construction module, an execution monitoring module, an offset analysis module, and a risk output module. The structure parsing module is used to perform mathematical modeling and structure parsing of the target multi-agent system, and to obtain the set of agents, the collaborative topology graph constructed based on the message passing relationship between agents, and the external knowledge base on which it depends. The sample injection module is used to construct poisoned samples containing potential vulnerabilities, example description text, and text feature vectors, and inject poisoned samples less than or equal to a preset proportion into an external knowledge base to form a contaminated knowledge base. The trigger construction module is used to construct a joint optimization objective function that integrates uniqueness, compactness and coherence constraints, and optimizes it using a gradient-guided search strategy to generate a trigger that can induce retrieval behavior to hit the poisoned sample. The execution monitoring module is used to input test tasks with integrated triggers into the target multi-agent system, drive the multi-agent system to execute collaboratively, and perform full-link monitoring and log recording of the inputs, outputs and key behaviors of each agent. The offset analysis module is used to compare and quantify the retrieval offset of the target multi-agent system under normal input and triggered input based on log records, and to track and reconstruct the risk propagation path triggered by the offset based on the collaborative topology map. The risk output module is used to quantify the system risk level based on the retrieval offset rate and code risk rate, and finally outputs a security assessment report that includes the risk level, a visualized propagation path and targeted defense suggestions.

[0016] Thirdly, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described knowledge-based poisoning security assessment method for multi-agent systems when executing the computer program.

[0017] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the aforementioned knowledge-based poisoning security assessment method for multi-agent systems.

[0018] Compared with the prior art, the beneficial effects of the present invention include at least the following: (1) This invention constructs a system-level collaborative topology graph by performing mathematical modeling and structural analysis on the target multi-agent system, and implements execution monitoring and log aggregation covering the entire link of retrieval, generation, and testing. It breaks through the limitations of traditional single-model security detection, and can track and visualize the cascading propagation path of risks among agents, thereby achieving accurate identification of deep structural security risks caused by knowledge base pollution.

[0019] (2) This invention constructs a poisoning sample containing potential vulnerabilities, example description text and text feature vectors, and injects it into the knowledge base in a very low proportion. Combined with a gradient-guided optimization algorithm that integrates uniqueness, compactness and coherence constraints, it generates the optimal natural language trigger. Under the premise of ensuring that the normal function of the system is not affected, it can effectively reproduce the real attack chain of "poisoning-triggering", and provides a controllable verification means for evaluating the actual robustness of the system in the face of supply chain attacks.

[0020] (3) This invention defines the core quantitative indicators of retrieval offset rate and code risk rate, performs weighted evaluation of system security level based on behavioral differential analysis results, and finally outputs a standardized report containing risk level, visualized propagation path and targeted defense suggestions. This evaluation system transforms the complex security situation into measurable and operable engineering basis, provides scientific decision support for the security audit, hardening and continuous integration and deployment of multi-agent systems, and improves the engineering reliability and security of the system. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the knowledge base-based security assessment method for multi-agent systems provided in this embodiment of the invention. Figure 2 This is a schematic diagram of the framework of the knowledge base-based security assessment method for multi-agent systems provided in this embodiment of the invention. Figure 3 This is a flowchart illustrating the specific implementation of the knowledge base-based poisoning security assessment method for multi-agent systems provided in this invention. Figure 4 This is a schematic diagram of the structure of a knowledge-based poisoning security assessment system for multi-agent systems provided in an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0024] The inventive concept of this invention is as follows: As multiple agents deeply participate in the software development process through a "division of labor-cooperation-feedback" mechanism, risks may propagate covertly along the retrieval, generation, and testing stages. To overcome the technical limitations of existing code generation multi-agent system security detection, which focuses on a single model and struggles to cover the multi-agent collaboration chain, and to address the challenge of identifying system-level security risks caused by external knowledge base contamination, this invention provides a knowledge base poisoning-based security assessment method and system for multi-agent systems. Specifically targeting code generation multi-agent systems, it constructs a closed-loop detection system of "structure parsing-sample injection-trigger construction-execution monitoring-offset analysis-risk output," achieving accurate identification and quantification of deep structural risks at the system level without disrupting normal system functionality. The technical solution of this invention will be described in detail below, combining mathematical models, algorithm principles, and specific engineering implementation details.

[0025] like Figure 1 and Figure 2 As shown in the embodiment, a knowledge base-based poisoning security assessment method for multi-agent systems is provided, including the following steps: S1 performs mathematical modeling and structural analysis on the target multi-agent system to obtain a set of agents, a collaborative topology graph constructed based on the message passing relationship between agents, and the external knowledge base on which it depends.

[0026] In this embodiment, to achieve observability and analyzability of complex code-generating multi-agent systems, the target system is first mathematically modeled and its structure analyzed. Specifically, the system first reads the configuration file of the target multi-agent system (such as a YAML-formatted topology definition file) or dynamically constructs a panoramic view of the system by listening to the runtime message bus, and sets up the set of agents in the target system. Defined as: , Each intelligent agent It is abstracted into an independent functional unit, whose internal logic corresponds to a task function. , used to represent the input space of the intelligent agent With output space In code generation scenarios, such as Figure 3 As shown, these intelligent agents typically include, but are not limited to: planning agents ( It is responsible for parsing user intent and breaking down task steps, and retrieving intelligent agents ( (Responsible for obtaining context information from external knowledge bases) and code generation intelligence agent ( (Responsible for generating specific code logic based on context information) and testing the intelligent agent ( The system includes an agent responsible for static analysis or unit testing of the generated code, and an integrated agent responsible for summarizing the processing results of each node and generating the final system product. To describe the collaborative flow between agents, this invention models the message passing relationships within the system as a directed graph. : , in, This refers to the set of agent nodes defined above. Let it be a set of directed edges. If, during task execution, the agent... The output data flows to the intelligent agent As its input, there exists a directed edge in the graph. This graph theory modeling not only clearly maps the system's call topology but also provides a topological foundation for subsequent tracking of the cascading propagation of risks among agents. Furthermore, considering the high dependence of code generation systems on external knowledge augmentation (RAG), this invention defines the set of knowledge bases upon which the system depends as follows: , in, This represents a core retrieval library based on vector indexes. Represents an unstructured document library. This represents a code template library. This example focuses on a vector retrieval library. The security of the retrieval process is mathematically modeled as a retrieval function: , That is, for any given query vector The function returns the most similar result from the knowledge base. document fragments To achieve vectorization, the system determined a corresponding deep learning query encoder. (For example, the Dense Retriever based on the BERT architecture) This encoder maps natural language to a high-dimensional real vector space. Through the above analytical steps, this invention establishes a mapping from physical systems to mathematical models.

[0027] S2, construct a poisoned sample containing potential vulnerabilities, example description text, and text feature vectors, and inject poisoned samples of less than or equal to a preset proportion into an external knowledge base to form a contaminated knowledge base.

[0028] In this embodiment, after completing system modeling, the process proceeds to the stage of constructing security test samples and injecting the knowledge base. The core objective of this stage is to construct a "hidden" risk source, that is, to implant specific poisoning samples into the knowledge base, keeping them silent under normal queries but activated under specific triggering conditions. This invention constructs a set of test samples: , Each sample It is a triple structure containing a potentially vulnerable code snippet. Example description text and the corresponding feature vectors In a specific embodiment, to simulate a real supply chain attack, the present invention constructs... This could be a Python function containing a hidden SQL injection vulnerability or command execution vulnerability (for example, disguised as a "high-performance database connection pool implementation" in code comments to evade simple scrutiny), and This is a seemingly normal API function description (e.g., "context-optimized user login interface"). To inject these samples into the system, this invention defines and executes the injection function: , This function will use the sample set Write to the original vector retrieval library In this process, a contaminated knowledge base is formed. During this process, This is an injection ratio control parameter. To ensure that the evaluation process does not disrupt the system's normal availability (i.e., does not affect query results for normal users), this invention strictly controls its implementation. The value is usually set The specific injection logic includes: first, calculating the feature vector describing the sample. Then insert it into the vector index. Because The sample size is extremely small, and the samples are distributed in a specific local region of the vector space. Therefore, for the vast majority of queries from ordinary users that are distributed in the normal semantic space, this is not feasible. Its relationship with the poisoning vector The similarity to the samples was far below the recall threshold, thus ensuring the "lightweight" and "concealed" nature of the poisoning behavior.

[0029] S3 constructs a joint optimization objective function that integrates uniqueness, compactness, and coherence constraints, and optimizes it using a gradient-guided search strategy to generate triggers that can induce retrieval behavior to hit the poisoned sample.

[0030] In this embodiment, to verify the system's robustness against malicious inducement, a trigger condition construction step is further implemented, aiming to solve the difficult problem of "how to ensure that the retrieval agent will necessarily recall the injected samples under specific inputs." This invention abandons the traditional enumeration method and proposes a gradient-guided constraint optimization trigger generation algorithm. This algorithm transforms the natural language generation problem into an optimization problem in a continuous vector space. The trigger task input is defined as... (i.e., a task description containing specific trigger words), the goal of this invention is to find an optimal trigger. This causes the generated query vector to undergo a directional shift in the embedding space. To address this, the present invention constructs a complex joint optimization objective function. : , in, and As weighting parameters, each part of this formula corresponds to a specific physical meaning and optimization constraint.

[0031] first, The uniqueness loss, mathematically defined, aims to maximize the clustering centers of the triggering input vector and the normal query. The distance between them. In implementation, the system pre-samples a large number of normal business queries and calculates their centroids. The optimization objective is then to minimize the cosine similarity. This forces the generated triggers to be far removed from the normal business distribution in the semantic vector space, thereby achieving a "specific offset" of the retrieval behavior and preventing the triggers from being intercepted by conventional deduplication or caching mechanisms.

[0032] The second term in the formula The compactness loss is crucial for the attack's success. Its objective is to minimize the difference between the trigger input vector and the poisoned sample vector injected in step two. The distance between them. In practice, the square of the Euclidean distance is used to measure it: , By minimizing this loss, the optimization algorithm will drive... The vector representation continuously approaches the region where the poisoned sample is located in multidimensional space. When the distance between the two is less than the recall radius of the retrieval algorithm, the retrieval agent... The poisoned sample will have a very high probability of being poisoned. The Top-K results are returned to the downstream system.

[0033] The third term in the formula For coherence constraints, this ensures the naturalness of the generated trigger in terms of human readability. If the trigger is a string of gibberish, it is easily intercepted by the system's pre-processing traffic filtering rules. Therefore, this invention introduces a pre-trained language model (such as GPT-2 or a small Llama model) to compute the trigger text. The perplexity level. If the perplexity level exceeds a preset natural language threshold, then a high-order weighting coefficient (such as...) is used. A strong constraint penalty is imposed on the objective function. This penalty term aims to cause an order-of-magnitude shift in the loss function, thereby forcing the search algorithm to circumvent non-natural language regions and ensuring that the generated triggers conform to human language logic. This constrains the search space, making the final generated triggers... It is a fluent, grammatically correct piece of natural language (e.g., "Please implement the login module using an efficient pattern with specific context optimizations..."), thus possessing extremely strong deceptive and penetrating power.

[0034] Solving the above joint objective function Since text data is discrete, direct gradient backpropagation is not possible. This invention employs a gradient-guided beam search strategy or a Gumbel-Softmax relaxation technique. Specifically, the system calculates the total loss. gradient relative to the input word embedding matrix Candidate replacement words are selected based on the direction of the gradient, and the top-N candidate sequences that cause the loss function to decrease the fastest are retained for the next iteration. After several rounds (e.g., 50-100 rounds) of iterative optimization, the system outputs the convergent optimal trigger. .

[0035] S4 inputs test tasks with integrated triggers into the target multi-agent system, drives the multi-agent system to perform collaborative execution, and monitors and logs the inputs, outputs and key behaviors of each agent throughout the entire process.

[0036] In this embodiment, after obtaining the optimal trigger, the present invention enters the multi-agent execution link monitoring stage. The system will construct a pre-defined multi-agent execution link monitoring system. The test task is input into the system under test, and a full-link log capture mechanism is initiated. To comprehensively record system behavior, this invention constructs a multi-agent execution log set: , Among them, for different types of intelligent agents The focus of monitoring differs. For retrieval agents... The system not only records its input queries It also focuses on recording the set of candidate documents it returns. (Including document ID, similarity score, and source), especially checking whether it contains documents from... Collective poisoning samples For code generation intelligent agents The system records the complete context it receives and the final generated code content. And it uses code fingerprinting technology to analyze whether it reproduces malicious logic. For the test agent Record the coverage and execution result status (Pass / Fail) of the generated test cases. Represents intelligent agents The key metadata set during task execution is mainly used to link scattered logs into a complete call chain in an asynchronous concurrent environment using the unique identifier TraceID, and to record engineering information including execution status, timestamps and model call parameters, providing underlying data support for reconstructing risk propagation paths based on the collaborative topology map.

[0037] To handle the asynchronous concurrency characteristics of multi-agent systems, the logging system uses a unique identifier, TraceID, to link the logs scattered across various agent nodes, forming a complete call chain.

[0038] S5 compares and quantifies the retrieval offset of the target multi-agent system under normal and triggered inputs based on log records, and tracks and reconstructs the risk propagation path triggered by the offset based on the collaborative topology graph.

[0039] In this embodiment, based on the collected detailed logs, the present invention implements behavioral deviation detection and risk propagation path analysis. This step aims to quantify the difference between a "normal state" and an "attacked state." The system executes two sets of tasks in parallel: one set with clean baseline prompts as input, and the other set with triggers. The test prompt words. For each agent in the link. The present invention defines its behavior offset as: , in, and These represent the outputs of the agent in the two scenarios, respectively. The function dynamically selects the metric based on the output data type: for search results, it uses the Jaccard similarity coefficient or the complement of recall overlap; for generated code, it uses the tree edit distance of the abstract syntax tree or the difference metric of the control flow graph. If the calculated... Exceeded the preset statistical significance threshold If so, it is determined that the agent node has experienced a "functional shift".

[0040] More importantly, this invention focuses not only on single-point offsets but also on the chain reaction of offsets. Based on the diagram constructed in step one... The algorithm of this invention automatically tracks the propagation trajectory of the offset in the graph. If the monitoring data indicates that the retrieval agent... A retrieval offset occurred (i.e.) (It hit the poisoned sample), and this offset caused the downstream code-generating agent to... This produced a product containing The code for the features (i.e., generating offsets), and the final test agent. If the vulnerability is not detected (i.e., test escape), the system builds and marks a complete risk propagation path: , The successful construction of this path intuitively and irrefutably proves that the system has deep-seated structural security vulnerabilities: tiny and extremely low in proportion. Knowledge base pollution, under the influence of specific triggers, can break through the triple defenses of retrieval, generation, and testing, ultimately leading to the production of insecure code.

[0041] S6 quantifies the system risk level based on the retrieval offset rate and code risk rate, and finally outputs a security assessment report that includes the risk level, visualized propagation path and targeted defense suggestions.

[0042] In this embodiment, risk level assessment and system output are performed by integrating all analysis results. The system calculates two core quantitative indicators: the retrieval offset rate (ASR-r), defined as the proportion of tests in the triggered tests where the retrieval results contain the poisoned sample out of the total number of tests; and the code risk rate (ASR-t), defined as the proportion of tests in the triggered tests where the final generated code is detected by static security tools to contain potential vulnerabilities (such as SQL injection) out of the total number of tests. This invention uses a weighted function to combine the retrieval offset rate and the code risk rate to calculate the system security risk score. : , in, , and These are the weight parameters, The path length term is introduced to quantify the cascading diffusion capability of risk in an agent network, representing the length of the risk propagation path. A longer path indicates a weaker structural defense mechanism and a higher degree of risk penetration into the overall logic. If the risk threshold is exceeded (e.g., when ASR-r > 80% and ASR-t > 50%), the system automatically generates a red alert. The final assessment report not only includes the risk level conclusion but also provides visual evidence—using the t-SNE algorithm to reduce the dimensionality of the embedding space and plotting a scatter plot, it visually demonstrates how the query vector with the trigger detaches from the clusters of normal queries and "drifts" into the manifold region where the poisoned sample is located. Based on this analysis, the invention also generates targeted defense suggestions, such as adjusting the training objective of the retrieval encoder to enhance robustness against adversarial examples, or adding an abnormal input filtering layer based on language model perplexity at the retrieval interface to cut off... The initial stage.

[0043] In summary, the knowledge-based poisoning security assessment method for multi-agent systems provided in this invention achieves end-to-end system-level coverage security assessment through knowledge-based poisoning, trigger input construction, multi-agent collaborative execution monitoring, and link offset analysis. Without disrupting the normal function of the system, it can effectively identify and characterize the structural security risks caused by knowledge source bias during multi-agent collaboration.

[0044] Based on the same inventive concept, such as Figure 4 As shown, this embodiment of the invention also provides a knowledge base poisoning-based security assessment system 400 for multi-agent systems, including: a structure parsing module 410, a sample injection module 420, a trigger construction module 430, an execution monitoring module 440, an offset analysis module 450, and a risk output module 460.

[0045] The structure analysis module 410 is used to perform mathematical modeling and structure analysis on the target multi-agent system to obtain the set of agents, the collaborative topology graph constructed based on the message passing relationship between agents, and the external knowledge base on which it depends.

[0046] The sample injection module 420 is used to construct poisoned samples containing potential vulnerabilities, example description text, and text feature vectors, and inject poisoned samples less than or equal to a preset proportion into an external knowledge base to form a contaminated knowledge base.

[0047] The trigger construction module 430 is used to construct a joint optimization objective function that integrates uniqueness, compactness and coherence constraints, and optimizes it using a gradient-guided search strategy to generate a trigger that can induce retrieval behavior to hit the poisoned sample.

[0048] The execution monitoring module 440 is used to input the test task with integrated triggers into the target multi-agent system, drive the multi-agent system to perform collaborative execution, and perform full-link monitoring and log recording of the inputs, outputs and key behaviors of each agent.

[0049] The offset analysis module 450 is used to compare and quantify the retrieval offset of the target multi-agent system under normal and triggered inputs based on log records, and to track and reconstruct the risk propagation path triggered by the offset based on the collaborative topology map.

[0050] The risk output module 460 is used to quantify the system risk level based on the search offset rate and code risk rate, and finally outputs a security assessment report that includes the risk level, visualized propagation path and targeted defense suggestions.

[0051] Based on the same inventive concept, embodiments of the present invention also provide an electronic device, including a memory and one or more processors, wherein the memory is used to store a computer program, and the processor is used to implement the above-described knowledge base-based poisoning security assessment method for multi-agent systems when executing the computer program.

[0052] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a computer, implements the aforementioned knowledge-based poisoning security assessment method for multi-agent systems.

[0053] It should be noted that the knowledge base poisoning security assessment system, electronic device, and computer-readable storage medium for multi-agent systems provided in the above embodiments all belong to the same inventive concept as the knowledge base poisoning security assessment method for multi-agent systems. For details of their specific implementation process, please refer to the embodiments of the knowledge base poisoning security assessment method for multi-agent systems, which will not be repeated here.

[0054] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A security assessment method for knowledge-based poisoning in multi-agent systems, characterized in that, Includes the following steps: Mathematical modeling and structural analysis of the target multi-agent system are performed to obtain the set of agents, the collaborative topology graph constructed based on the message passing relationship between agents, and the external knowledge base on which it depends. Construct poisoned samples containing potential vulnerabilities, example description text, and text feature vectors, and inject poisoned samples of less than or equal to a preset proportion into an external knowledge base to form a contaminated knowledge base; A joint optimization objective function integrating uniqueness, compactness, and coherence constraints is constructed, and a gradient-guided search strategy is used for optimization to generate triggers that can induce retrieval behavior to hit the poisoned sample. The test task with integrated triggers is input into the target multi-agent system to drive the multi-agents to cooperate in execution, and the input, output and key behaviors of each agent are monitored and logged in the whole link. Based on log records, the retrieval offset of the target multi-agent system under normal and triggered inputs is compared and quantified. Based on the collaborative topology graph, the risk propagation path triggered by the offset is tracked and reconstructed. The system risk level is quantified based on the retrieval offset rate and code risk rate, and the final output is a security assessment report that includes the risk level, a visualized propagation path, and targeted defense recommendations.

2. The security assessment method for knowledge-based poisoning of multi-agent systems according to claim 1, characterized in that, The target multi-agent system includes at least: a planning agent, responsible for parsing user intent and breaking down task steps; a retrieval agent, responsible for obtaining context information from an external knowledge base; a code generation agent, responsible for generating specific code logic based on context information; a testing agent, responsible for performing static analysis or unit testing on the generated code; and an integration agent, responsible for summarizing the processing results of each agent and generating the final system deliverable. A collaborative topology graph is constructed by treating each agent as a node and the message passing relationships between agents as directed edges.

3. The security assessment method for knowledge-based poisoning of multi-agent systems according to claim 1, characterized in that, The construct includes a poisoning sample comprising potential vulnerabilities, example descriptive text, and text feature vectors, including: The potential vulnerability is constructed as a code snippet containing a hidden SQL injection vulnerability or command execution vulnerability. Example description text is written for this code snippet to suit its camouflage purpose, and the example description text is encoded into a text feature vector, thus forming a poisoned sample containing a triple structure of potential vulnerability, example description text, and text feature vector.

4. The security assessment method for knowledge-based poisoning of multi-agent systems according to claim 1, characterized in that, The external knowledge base upon which the target multi-agent system relies includes at least: a vector retrieval library, an unstructured document library, and a code template library; wherein, a poisoning sample of less than or equal to a preset proportion is injected into the vector retrieval library of the external knowledge base to ensure the lightweight and covert nature of the poisoning behavior.

5. The security assessment method for knowledge-based poisoning of multi-agent systems according to claim 1, characterized in that, The construction of a joint optimization objective function integrating constraints of uniqueness, compactness, and coherence, and the optimization using a gradient-guided search strategy, generates triggers that can induce retrieval behavior to hit the poisoned sample, including: A uniqueness loss is constructed by maximizing the distance between the cluster centers of the trigger input vector and the normal query input vector; A compact loss is constructed by minimizing the distance between the trigger input vector and the poisoning sample vector in the injected knowledge base; The perplexity of the trigger text is calculated by using a pre-trained language model, and a penalty term is constructed based on the perplexity to limit the text generation space, thus constructing a coherence loss. The uniqueness loss, compactness loss, and coherence loss are weighted and fused into a joint optimization objective function. The joint optimization objective function is iteratively solved using gradient-guided beam search or Gumbel-Softmax relaxation techniques until convergence, and the final natural language trigger is output.

6. The security assessment method for knowledge-based poisoning of multi-agent systems according to claim 1, characterized in that, The method of comparing and quantifying the retrieval offset of the target multi-agent system under normal and triggered inputs based on log records, and tracking and reconstructing the risk propagation path triggered by the offset according to the cooperative topology graph, includes: Two sets of tasks are executed in parallel: normal prompt word input and test prompt word input of integrated trigger. The overlap rate of the results returned by the retrieval agent in the two sets of tasks is calculated and compared to quantify the retrieval behavior offset. If the retrieval behavior offset exceeds the preset statistical significance threshold, it is determined that the retrieval agent has retrieval offset. The code generation agent is also analyzed to see if it generates code containing the characteristics of the poisoning sample vulnerability under the trigger input. Based on the collaborative topology graph between agents, if a retrieval offset leads to code risk and the test agent fails to detect it, a complete risk propagation path is marked from the retrieval agent through the code generation agent to the test agent.

7. The security assessment method for knowledge-based poisoning of multi-agent systems according to claim 1, characterized in that, The system risk level quantification based on retrieval offset rate and code risk rate includes: The retrieval offset rate is defined as the proportion of the number of tests in which the retrieval results contain the poisoned sample out of the total number of tests during the trigger test. Code risk rate is defined as the proportion of the number of tests in the triggered tests in which the final generated code is detected by static security tools to contain potential vulnerabilities out of the total number of tests. The system security risk score is calculated by combining the retrieval offset rate, code risk rate and risk propagation path length using a weighted function, and the risk level is determined based on a preset threshold range.

8. A knowledge-based poisoning security assessment system for multi-agent systems, implemented using the knowledge-based poisoning security assessment method for multi-agent systems as described in any one of claims 1 to 7, characterized in that, include: The module includes a structure parsing module, a sample injection module, a trigger construction module, an execution monitoring module, an offset analysis module, and a risk output module. The structure parsing module is used to perform mathematical modeling and structure parsing of the target multi-agent system, and to obtain the set of agents, the collaborative topology graph constructed based on the message passing relationship between agents, and the external knowledge base on which it depends. The sample injection module is used to construct poisoned samples containing potential vulnerabilities, example description text, and text feature vectors, and inject poisoned samples less than or equal to a preset proportion into an external knowledge base to form a contaminated knowledge base. The trigger construction module is used to construct a joint optimization objective function that integrates uniqueness, compactness and coherence constraints, and optimizes it using a gradient-guided search strategy to generate a trigger that can induce retrieval behavior to hit the poisoned sample. The execution monitoring module is used to input test tasks with integrated triggers into the target multi-agent system, drive the multi-agent system to execute collaboratively, and perform full-link monitoring and log recording of the inputs, outputs and key behaviors of each agent. The offset analysis module is used to compare and quantify the retrieval offset of the target multi-agent system under normal input and triggered input based on log records, and to track and reconstruct the risk propagation path triggered by the offset based on the collaborative topology map. The risk output module is used to quantify the system risk level based on the retrieval offset rate and code risk rate, and finally outputs a security assessment report that includes the risk level, a visualized propagation path and targeted defense suggestions.

9. An electronic device comprising a memory and one or more processors, the memory for storing a computer program, characterized in that, The processor is used to implement the knowledge base-based poisoning security assessment method for multi-agent systems as described in any one of claims 1 to 7 when executing a computer program.

10. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by a computer, it implements the knowledge base-based poisoning security assessment method for multi-agent systems as described in any one of claims 1 to 7.

Images