Large language model backdoor detection method and device based on graph chain thought and medium
By constructing a graph-chain-based thinking template database and generating reasoning graphs, the accuracy and applicability issues of backdoor detection in large language models in existing technologies are solved, achieving efficient and interpretable backdoor detection applicable to black-box models.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-15
- Publication Date
- 2026-04-03
AI Technical Summary
Existing backdoor detection methods have limited effectiveness in large language models, especially in generative tasks where they struggle to capture trigger patterns and cannot effectively detect backdoors implanted by modifying model weights. Furthermore, methods that rely on internal model activation values are not suitable for black-box models.
We construct a graph chain-based thinking template database. By generating and comparing the reasoning graphs of the target large language model and the reference large language model, we use the graph chain concept to detect backdoors, reducing the dependence on large-scale labeled data and improving the accuracy and interpretability of the detection results.
It significantly improves the accuracy and interpretability of backdoor detection, reduces the dependence on large-scale labeled data, and is suitable for black-box model detection of various backdoor attacks.
Smart Images
Figure CN121786829A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer security technology, and in particular to a method, device, and medium for detecting backdoors in large language models based on graph chain concepts. Background Technology
[0002] In recent years, Large Language Models (LLMs) have demonstrated exceptional capabilities in various tasks, including chatbots and medical assistants, and have been widely deployed. However, this widespread application has also raised serious credibility issues, such as privacy breaches, model jailbreaking, and backdoor attacks. Backdoor attacks, as a covert malicious attack method, aim to implant triggers into the model. These backdoor-implanted models behave indistinguishably from normal models when processing normal, benign input, but once they receive input containing specific triggers, they execute malicious behaviors pre-set by the attacker, outputting unexpected or even harmful content. Because these triggers are often difficult for users to detect, backdoor attacks pose a serious potential threat to the application of large language models.
[0003] Existing backdoor detection methods, such as those based on trigger generation, attention analysis, trigger inversion, and meta-neural analysis, are primarily designed for classification tasks based on model output probabilities (logits). Due to the diversity and context-dependent nature of large language models' outputs, these methods have limited effectiveness in generative tasks, struggling to capture consistent trigger patterns. More critically, many existing detection methods rely on access to the model's internal activation values, making them unsuitable for black-box large language models accessed via API interfaces. While some research has attempted to leverage the reasoning capabilities of large language models to detect logical inconsistencies caused by backdoor triggers (e.g., Chain-of-Scrutiny, CoS), the effectiveness of these methods is limited by the model's reasoning ability and performs poorly on small to medium-sized models. Furthermore, these methods primarily target cue-injected backdoors; their effectiveness against backdoors implanted through traditional methods such as modifying model weights remains to be verified, and their reliance on logical inconsistency detection may not uncover more subtle backdoor mechanisms. Summary of the Invention
[0004] To address the shortcomings of existing technologies, embodiments of the present invention provide a method, device, and medium for detecting backdoors in large-scale language models based on the graph chain concept.
[0005] In a first aspect, embodiments of the present invention provide a method for detecting backdoors in large language models based on graph chain concepts, the method comprising: Construct a graph-linked thinking template database; based on the current output response of the target large language model, retrieve the top N graph-linked thinking templates with the highest similarity from the graph-linked thinking template database; A model graph generation query is constructed based on the model graph, generated prompts, N graph chain-like thinking templates, and the current output response of the target large language model. The model graph generation query is then input into the target large language model and several reference large language models to generate a reasoning graph. A reasoning graph discrimination query is constructed based on the discrimination prompt words in the reasoning graph, the reasoning graph generated by the target large language model, and the reasoning graph generated by each reference large language model. The reasoning graph discrimination query is input into the judgment large language model, and the differences between the reasoning graph generated by the target large language model and the reasoning graph generated by each reference large language model are analyzed to obtain the backdoor detection results.
[0006] Secondly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-described large language model backdoor detection method based on graph chain concept.
[0007] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the aforementioned method for detecting backdoors in large language models based on graph chain concepts.
[0008] Fourthly, embodiments of the present invention provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the aforementioned method for detecting backdoors in large language models based on graph chain concepts.
[0009] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention provides a backdoor detection method for large language models based on graph chain thinking. By constructing and utilizing graph chain-like thinking templates to guide the large language model to generate a structured reasoning graph, and comparing the differences between the reasoning graphs generated by the target large language model and the reference large language model, the accuracy of backdoor detection results is improved. This invention not only significantly reduces the dependence on large-scale labeled data, but also improves the interpretability of backdoor detection results. Attached Figure Description
[0010] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1 A schematic diagram illustrating a backdoor detection method for large language models based on graph chain concepts provided in an embodiment of the present invention; Figure 2 A schematic diagram of a graph mind chain template provided in an embodiment of the present invention; Figure 3 A schematic diagram of the graph thinking chain structure provided in an embodiment of the present invention; Figure 4 Example diagrams illustrating interpretable analysis results provided in embodiments of the present invention; Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0012] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0013] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.
[0014] A text-attributed graph, denoted as ,in A node is a set of nodes that represent units of text, such as words, phrases, or reasoning steps. It is a set of edges used to capture the relationships or dependencies between these text units; Text attributes are assigned to each node and edge. The concept of this invention is that, within the context of a large language model, such a graph encodes the reasoning process or semantic structure of the response, thereby enabling systematic comparison and analysis of the behavior of large language models. The goal of this invention is to generate a binary decision by comparing the text attribute graphs of a target large language model. With a set of reference text attribute diagrams This is used to determine whether the response of a target large language model has been manipulated by a backdoor.
[0015] like Figure 1 As shown, this embodiment of the invention provides a backdoor detection method for large-scale language models based on graph chain concepts. The method includes the following steps: Step S1: Construct a graph-linked thinking template database; based on the current output response of the target large language model, retrieve the top N graph-linked thinking templates with the highest similarity from the graph-linked thinking template database.
[0016] Specifically, step S1 includes the following sub-steps: Step S101: For different types of natural language tasks, the reasoning process is transformed into a graph chain thinking template (GCoT) consisting of nodes and dependency edges, where nodes represent analysis steps and edges represent information flow; thereby constructing a graph chain thinking template database.
[0017] Furthermore, a series of graph-chain thinking templates are pre-created for various common natural language processing task types (such as question answering, text summarization, sentiment analysis, etc.). These graph-chain thinking templates are designed based on typical human reasoning patterns when solving corresponding tasks. For example, in Figure 2 The example demonstrates a GCoT template for a commonsense QA task. In this example, the reasoning process is broken down into multiple steps, such as "The main question is...", "We need to know...", "For option (A)...", etc. Each such analysis step is formalized as a node in the GCoT template, and the logical order and dependencies between steps are represented by edges. The "Feature" attribute of each edge describes the specific thought process or action taken in that step.
[0018] Step S102: Match the current output response of the target large language model with the natural language task types in the graph chain thinking template database to confirm the natural language task type corresponding to the current output response; based on the similarity between the current output response of the target large language model and the example questions associated with each graph chain thinking template under its corresponding natural language task type, retrieve the top N most similar graph chain thinking templates.
[0019] Furthermore, when a current output response R from the target large language model is obtained, the most suitable graph chain thinking template needs to be selected from the graph chain thinking template database.
[0020] First, category matching is performed, matching the current output response R against a predefined set of natural language task categories C (where C represents the set of all natural language task categories) in the GCoT template database. The most suitable natural language task category c is selected by maximizing the task type alignment between the current output response R and each natural language task category, as shown in the following expression: In the formula, It is a function that measures the degree of fit between the current output response R and the type of natural language task represented by the category.
[0021] Next, template retrieval is performed. Within the selected natural language task category c, further example questions are associated with each graph-linked thinking template T under that natural language task category c, based on the current output response R. The similarity between them is used to retrieve the N most similar graph-chain thinking templates, denoted as T1, T2, ..., T. N This process can be represented as: In the formula, This is the problem of calculating the correspondence between the current output response R of the target large language model and the graph chain thinking template. A function of similarity between them.
[0022] Finally, a threshold judgment and manual design are performed. If the similarity between the graph chain thinking template selected according to the above formula and the current output response R is lower than the preset threshold, the result is considered. (Right now If the graph chain thinking template database does not contain a sufficient number of suitable graph chain thinking templates, then a new GCoT template needs to be manually designed for the current output response R of the target large language model.
[0023] Step S2: Construct a model graph generation query based on the model graph generated prompt words, N graph chain-like thinking templates, and the current output response of the target large language model; input the model graph generation query into the target large language model and several reference large language models respectively to generate a reasoning graph.
[0024] Specifically, step S2 includes the following sub-steps: Step S201: Generate prompt words based on a general model diagram. N Graph Chain Thinking Templates The current output response of the target large language model Construct a model graph to generate queries The expression is as follows: In the formula, This is a general model graph generation prompt. The prompt instructs the LLM on how to construct the graph based on subsequent templates and response content. For example... Figure 2 As shown, an example of such a prompt is provided: "Represent the most recent response as a graph. Nodes indicate each step of a thought or action. Edges (from source to target) indicate which nodes the current node depends on. Responses must strictly follow the following format and the result must be represented as a graph: Source (node id) | Target (node id) | Feature (action or thought)." R represents the N GCoT templates retrieved in step S1. R is the current output response from the target large language model. ⊕ indicates a concatenation operation, representing the combination or concatenation of text.
[0025] Step S202: Generate a query from the model graph obtained in step S201. The inputs are fed into the target large language model and several reference large language models with comparable performance to generate inference graphs.
[0026] Specifically, the constructed model will generate queries. The inputs are fed into the target large language model under review and one or more reference large language models, respectively. Each large language model receives the model-generated query. Then, following the prompts and template guidance, a structured reasoning diagram will be generated: The nodes in the generated inference graph G represent each thought step or specific action of the LLM in forming the final response R, while the edges clearly show the dependencies between these steps and the order of inference.
[0027] For example, such as Figure 3 As shown, one node might represent "the core requirement of the analysis problem," and another node might represent "evaluating the rationality of option A," connected by an edge. To ensure the fairness and validity of the comparison, the capabilities of the selected reference LLM (such as model size, training data, inference level, etc.) should be comparable to the target LLM being detected. If the capabilities of the reference LLM are significantly lower or higher than those of the target LLM, the difference in the model's inherent capabilities may lead to significant differences in the generated inference graphs, introducing confusion and affecting the accuracy of backdoor detection. In particular, if the target LLM generates a poor-quality response due to a backdoor attack, its inference graph itself may differ significantly from that of the high-quality reference model, which could actually aid in detection.
[0028] Step S3: Construct a reasoning graph discrimination query based on the discrimination prompt words in the reasoning graph, the reasoning graph generated by the target large language model, and the reasoning graph generated by each reference large language model; input the reasoning graph discrimination query into the judgment large language model, analyze the differences between the reasoning graph generated by the target large language model and the reasoning graph generated by each reference large language model, and obtain the backdoor detection result.
[0029] Specifically, step S3 includes the following sub-steps: Step S301: Determine the clue words based on the reasoning diagram. Inference graph generated by the target large language model Inference graphs generated by each reference large language model Construct a reasoning graph to discriminate queries .
[0030] Design a reasoning graph to discriminate queries. This is used to guide a powerful discriminative LLM (such as GPT-4) to perform graph comparison tasks. The query is constructed as follows: in It is a general inference graph discrimination prompt, indicating how the discrimination LLM compares the input graph and from which aspects the differences are analyzed. It is the inference graph generated by the target large language model being reviewed in step S2. It is a set of inference graphs generated by one or more reference LLMs in step S2.
[0031] Step S302: Determine and query the reasoning graph from step S301. The input is fed into a discriminative LLM with powerful analytical capabilities. This discriminative LLM comprehensively analyzes the significant differences between the target inference graph and the reference inference graph from three dimensions: structure and process (comparing node attributes, topology, graph metrics, etc.), reasoning process (comparing the steps of the thought chain, logical jumps and abnormal connections), and logical consistency (checking the logical coherence and contradictions between inference nodes). Based on the difference analysis results, it ultimately determines whether the target LLM's response has backdoor behavior and can provide interpretive analysis.
[0032] Perform differential analysis on the discriminant LLM and output the detection results. (The discriminant query will be used.) The input is fed into the discriminative LLM, which will then analyze it. and The differences between them are analyzed, and an explanatory analysis of these differences is provided, ultimately outputting a backdoor detection result: in, This is typically a binary decision (e.g., "a backdoor exists" or "no backdoor detected"), and may be accompanied by explanatory statements. Discriminant LLMs primarily analyze the differences between graphs from the following three aspects: ① Structure and Flow: This aspect focuses on the macroscopic and microscopic structural characteristics of the graph. Determining an LLM involves comparing the target graph with a reference graph in terms of node attributes (such as the semantics and complexity of text content), topological structure (such as node connection methods, graph depth and breadth, critical paths, etc.), and graph metrics (such as graph density, average path length, centrality metrics, etc.). For example, backdoor behavior might lead to abnormally shortened inference paths or illogical node jumps, all of which will be reflected in the graph's structure and flow.
[0033] ② Reasoning Process: This aspect focuses on the internal logic of the thought chain represented by the graph. Since backdoor attacks typically establish an abnormal association of "specific trigger - malicious behavior" outside of the normal "trigger-response" logic, the reasoning steps of models affected by backdoors often differ significantly from those of benign models. Benign models, after being guided by GCoT templates, can usually learn a certain degree of similar, canonical reasoning patterns. By explicitly representing each reasoning step as nodes and edges in the graph, discriminative LLMs can more precisely compare the differences between different models in specific reasoning stages. For example, the reasoning graph of the target model may contain illogical intermediate reasoning nodes unrelated to the task objective, or, under similar inputs, its reasoning path may differ greatly from that of the reference model.
[0034] ③ Logical Consistency: This aspect focuses on whether the transitions between nodes in the target inference graph conform to normal logical coherence. Backdoor behavior often causes the model to deviate from its inherent, normal logical reasoning pattern. By comparing it with the inference graph generated by the reference model, which represents normal logic, logical inconsistencies, contradictions, or jumps in the target graph can be effectively identified. For example, the target model may behave normally in the first few inference steps, but suddenly turn to a conclusion that contradicts the previous analysis after a certain key node, which indicates a logical inconsistency.
[0035] Furthermore, such as Figure 4 As shown, discriminative LLM integrates all the differences observed in the above three aspects to give a final backdoor detection conclusion. For example, if the target graph has abnormal branches in its structure, illogical steps occur in the reasoning process, and there are obvious logical breaks, then discriminative LLM tends to judge that the target model has a backdoor.
[0036] Accordingly, this application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the large language model backdoor detection method based on the graph chain concept described above. Figure 5 The diagram shown illustrates a hardware structure of any device with data processing capabilities for implementing the large-scale language model backdoor detection method based on graph chain concepts, as provided in this embodiment of the invention. (Except for...) Figure 5 In addition to the processor, memory, and network interface shown, any data processing device in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.
[0037] Accordingly, this application also provides a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the large language model backdoor detection method based on the graph chain concept described above. The computer-readable storage medium can be an internal storage unit of any data-processing device as described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data-processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data-processing device, and can also be used to temporarily store data that has been output or will be output.
[0038] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only.
[0039] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.
Claims
1. A method for detecting backdoors in large-scale language models based on graph chain concepts, characterized in that, The method includes: Construct a graph-linked thinking template database; based on the current output response of the target large language model, retrieve the top N graph-linked thinking templates with the highest similarity from the graph-linked thinking template database; A model graph generation query is constructed based on the model graph, generated prompts, N graph chain-like thinking templates, and the current output response of the target large language model. The model graph generation query is then input into the target large language model and several reference large language models to generate a reasoning graph. A reasoning graph discrimination query is constructed based on the discrimination prompt words in the reasoning graph, the reasoning graph generated by the target large language model, and the reasoning graph generated by each reference large language model. The reasoning graph discrimination query is input into the judgment large language model, and the differences between the reasoning graph generated by the target large language model and the reasoning graph generated by each reference large language model are analyzed to obtain the backdoor detection results.
2. The method for detecting backdoors in large language models based on graph chain concepts according to claim 1, characterized in that, The process of constructing a graph-linked thinking template database and retrieving the top N graph-linked thinking templates by similarity based on the current output response of the target large language model includes: For each type of natural language task, several graph-linked thinking templates are constructed to obtain a graph-linked thinking template database. Match the current output response of the target large language model with the natural language task types in the graph chain thinking template database to confirm the natural language task type corresponding to the current output response; Based on the similarity between the current output response of the target large language model and the example questions associated with each graph chain thinking template under its corresponding natural language task type, the top N most similar graph chain thinking templates are retrieved.
3. The method for detecting backdoors in large language models based on graph chain concepts according to claim 1, characterized in that, Based on the current output response of the target large language model, the process of retrieving the top N graph-linked thinking templates by similarity from the graph-linked thinking template database also includes: Set a similarity threshold; If the similarity between the current output response of the target large language model and the example questions associated with each graph chain thinking template under its corresponding natural language task type is less than the similarity threshold, then several graph chain thinking templates are redesigned for the current output response of the target large language model.
4. The method for detecting backdoors in large language models based on graph chain concepts according to claim 1, characterized in that, The model graph generation prompt is as follows: Represent the most recent output response as a graph, nodes indicate each step of thought or action, edges (from source to target) indicate which nodes the current node depends on, and the response must strictly follow the following format and the result is represented as a graph: source (node id) | target (node id) | feature (action or thought).
5. The method for detecting backdoors in large language models based on graph chain concepts according to claim 1, characterized in that, The target large language model is comparable to the reference large language model in terms of model size, training data, and inference performance.
6. The method for detecting backdoors in large language models based on graph chain concepts according to claim 1, characterized in that, Input the inference graph discriminative query into the large language model and analyze the differences between the inference graph generated by the target large language model and the inference graph generated by each reference large language model, including analysis from at least one of the following dimensions: Compare the inference graph generated by the target large language model with the inference graph generated by the reference large language model in terms of node attributes, topology, and graph metrics. Compare the inference graph generated by the target large language model with the inference graph generated by the reference large language model in terms of thought chain steps, logical jumps, and abnormal connections; And / or, Check whether the node transitions in the inference graph generated by the target large language model conform to logical coherence, and identify whether there are logical contradictions or jumps.
7. The method for detecting backdoors in large language models based on graph chain concepts according to claim 1, characterized in that, The backdoor detection result includes: a binary decision result for determining whether the target large language model output by the large language model has a backdoor, and generating an explanatory description of the difference between the inference graph generated by the target large language model and the inference graph generated by the reference large language model.
8. An electronic device comprising a memory and a processor, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the large language model backdoor detection method based on graph chain concept as described in any one of claims 1-7.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the large-scale language model backdoor detection method based on the graph chain concept as described in any one of claims 1-7.
10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instruction is executed by the processor, it implements the large language model backdoor detection method based on the graph chain concept as described in any one of claims 1-7.