Large language model security defense method and system based on dynamic risk reasoning
By employing a defense framework based on three-stream discrimination and secure decoding, risks are dynamically assessed and resources are allocated, overcoming the limitations of large-scale language model defense schemes and achieving effective defense against complex jailbreak attacks and efficient operation of benign tasks.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-14
- Publication Date
- 2026-04-07
AI Technical Summary
Existing security defense solutions for large-scale language models have limitations. They cannot effectively detect complex jailbreak attacks, and the computational overhead during the defense process is high, affecting the effectiveness of beneficial tasks and failing to fully utilize the model's inherent ability to identify harmful content.
A defense framework based on three-stream discrimination and security decoding is adopted. It performs rapid risk assessment through a probabilistic risk classifier, dynamically allocates defense resources, and combines a security-sensitive expert model and distributed guided decoding to handle advanced attacks and uncertainties while maintaining the general utility of the model.
It achieves efficient defense against diverse jailbreak attacks, reduces computational costs, improves the model's security under high-risk and uncertain conditions, and keeps the performance of benign tasks unaffected.
Smart Images
Figure FSA0000302643890000011 
Figure FSA0000302643890000012 
Figure FSA0000302643890000013
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence and natural language processing security technology, and in particular to a dynamic multi-stage security defense method for large language models. Specifically, this method involves dynamic risk assessment based on a probabilistic risk classifier, training of a security-sensitive expert model, and distributed security intervention technology during decoding. This is used to effectively resist jailbreak attacks and suppress the generation of harmful content, while maintaining the model's utility in benign tasks. Background Technology
[0002] Large Language Models (LLMs) have demonstrated exceptional capabilities in various natural language tasks and have been widely applied in real-world scenarios such as dialogue systems and code assistants. However, with the deployment of LLMs in high-risk fields such as education, healthcare, and law, the vulnerability of their security alignment mechanisms is becoming increasingly apparent. Jailbreak attacks can induce models to output prohibited content (such as weapon manufacturing guides, medical misinformation, or privacy-violating content) through adversarial suffixes, prefixes, and semantic rewriting, posing a serious threat to public safety and trust in AI systems.
[0003] Despite significant progress in secure alignment techniques, existing defense solutions have obvious technical flaws:
[0004] (1) Limitations of static single defense: While existing post-processing classifiers or input filters can block obvious attacks, they cannot effectively detect sophisticated jailbreaks that have been disguised. Alignment fine-tuning can enhance rejection consistency, but it often manifests as "shallow security alignment," meaning that the defense is only focused on the initial stage of generation and is easily bypassed by pre-filling techniques that disrupt the initial token generation.
[0005] (2) Rigidity of binary classification decision: Traditional defense mechanisms usually adopt a black-and-white rejection / response strategy, which cannot handle the "gray area" queries where the model's own judgment is ambiguous. This is the key area for the success of many advanced attacks (such as GCG and AutoDAN).
[0006] (3) The cost of defense during decoding: Methods such as SafeDecoding continuously intervene throughout the generation process. Although they provide dynamic protection, they bring significant computational overhead and are not effective when adversarial continuation is similar to benign text, which seriously affects the utility of the model in benign tasks such as mathematics and code.
[0007] (4) Waste of inherent security signals: Existing methods fail to make full use of the harmful content identification capabilities that LLMs already possess. These inherent discrimination signals are a valuable resource for building an effective defense.
[0008] Therefore, there is an urgent need in the field for a new defense framework that can dynamically assess risks, allocate defense resources on demand, and remain vigilant throughout the entire generation trajectory, in order to resolve the aforementioned contradiction between security and efficiency, and practicality. Summary of the Invention
[0009] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a language model output control method based on three-stream discrimination and secure decoding. By constructing an adaptive, multi-stage defense framework, the method can improve the language model's ability to defend against diverse jailbreak attacks, while maximizing the model's general utility while ensuring security.
[0010] To achieve the above objectives, embodiments of the present invention provide a language model output control method based on three-stream discrimination and secure decoding, comprising the following steps:
[0011] Step S1, Probabilistic Risk Classification and Tri-stream Decision: A lightweight probabilistic risk classifier based on an improved neural network architecture is used to quickly assess the risk of the input query and generate a harmful probability P. harm and safety probability P safe Based on this probability, the query is split into three paths:
[0012] (1) High-risk path (immediate rejection): When P harm Significantly higher than P safe (i.e. P) harm >P safe And |P harm -P safe When |≥δ), the standardized rejection mechanism is triggered, and a fixed response template is returned;
[0013] (2) Low-risk path (expert-guided generation): When P safe Significantly higher than P harm (i.e. P) harm <P safe And |P harm >P safe When |≥δ), the query will be routed to the security-sensitive expert model for response generation;
[0014] (3) Uncertain Risk Path (Distribution-Guided Thinking): When P safe With P harm The difference is less than the threshold δ (i.e., |P harm <P safe When |<δ), the query will be elevated to the distributed guided security decoding component for in-depth processing.
[0015] Step S2, Security Sensitive Expert Model Processing: The security sensitive expert model is obtained by augmenting the aligned base model within a fully parameterized fine-tuning framework. This model is trained on a hybrid dataset consisting of harmful recovery data and utility protection data, specifically optimizing its ability to recover secure responses from potentially harmful contexts while maintaining response quality to benign queries.
[0016] Step S3, Distribution-Guided Security Decoding: For uncertain risk queries, the output distributions of the base LLM and the security-sensitive expert model are calculated synchronously at each decoding step. By dynamically evaluating the similarity between the two distributions, a candidate token pool is adaptively constructed, and tokens that meet the security objectives are prioritized, achieving real-time security intervention in the generation process.
[0017] Step S4, Defense Effectiveness Evaluation: Use an independent evaluation model to analyze the attack success rate of the system's final output and quantify the overall defense effectiveness of the framework.
[0018] The present invention has the following advantages:
[0019] (1) Dynamic resource allocation: By using a probabilistic risk classifier for pre-risk reasoning, the computationally expensive deep defense measures are used only for uncertain and high-risk queries that are truly needed, thus achieving the best balance between security and efficiency.
[0020] (2) Eliminate shallow alignment vulnerabilities: By introducing a security-aware twin expert model (SATE) specifically trained for prefix padding attacks, security denial can be restored even if the initial defense is bypassed.
[0021] (3) Effective handling of uncertainty: The innovative introduction of the "uncertainty risk" path and the use of the distribution guided thinking decoding (DDGT) processing model to judge the fuzzy advanced attacks fill the gap in binary classification defense.
[0022] (4) Effective preservation of practicality: The basic model's capabilities are preserved in low-risk paths and cooperative decoding, ensuring that performance on benign tasks such as GSM8K and code generation is not severely affected. Attached Figure Description
[0023] Figure 1 This is a schematic diagram of the probability distribution and risk area division of benign commands, obviously harmful commands, and complex jailbreak attack commands by the query risk inference gateway in this embodiment of the invention; the diagram shows the distribution characteristics of jailbreak attacks mainly concentrated in uncertain risk areas.
[0024] Figure 2This is an overall process architecture diagram of the large language model security defense method based on dynamic risk reasoning provided in the embodiments of the present invention; it shows the entire process from input query to gateway diversion, entering three processing paths of high risk, low risk and uncertain risk respectively, and finally generating a response.
[0025] Figure 3 This is a flowchart illustrating the dynamic intervention logic of the Distributed Guided Thinking Decoding (DDGT) strategy in this embodiment of the invention. It shows the specific steps of dynamically switching between adversarial intervention mode and cooperative decoding mode based on the comparison results of the distribution similarity and threshold between the base model and the expert model during the decoding process. Detailed Implementation
[0026] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of the invention. Figure 2 It can be seen that the method includes the following steps:
[0027] Step S1: Query Risk Inference Gateway Construction and Computation. This step is the entry point for the entire defense framework. This embodiment constructs a lightweight query risk inference gateway. The gateway is built on top of the last hidden layer of a target Base LLM with frozen parameters. Specifically, the gateway is a trained probabilistic head. It uses the last hidden state of the base model after encoding the input query text x as feature input. The training data contains approximately 10,000 benign instructions and 10,000 harmful instructions, trained using binary cross-entropy loss. During the inference phase, for any user input, the gateway outputs two probability values: a harmless probability P. safe And the probability of harm P harm And satisfy P safe +P harm =1.
[0028] Step S2: Risk Level Classification and Dynamic Routing This invention abandons the traditional binary classification and instead classifies queries into three categories based on probability confidence. An uncertainty threshold of δ is set (in this embodiment, through experimental optimization, δ can be set between 0.2 and 0.8, for example, 0.7). The routing logic is as follows: (1) High-Risk Path: When P harm >P safe And |P harm -P safeWhen |>δ, it indicates that the model is very confident that the input is harmful. At this time, the system directly activates the standardized rejection mechanism and returns a preset fixed rejection text (such as: "I apologize, but I cannot assist with this request...") to intercept obvious attacks and prevent information leakage with the highest efficiency. (2) Low-Risk Path: When P harm <P safe And |P harm- P safe When | > δ, it indicates that the model is confident that the input is safe. At this time, the query is routed to the Security-Aware Twin Expert Model (SATE) for generation. (3) Uncertain-Risk Path: When |P harm -P safe When |≤δ, it indicates that the model has doubts about the nature of the input (this usually occurs in complex jailbreak attacks or spoofing attacks). At this point, the query is escalated and enters the Distributed Guided Thinking Decoding (DDGT) process.
[0029] Step S3: Construction and Application of the Safety-Aware Twin Expert Model (SATE) To support low-risk and uncertain paths, this embodiment constructs the SATE model. The SATE model is not trained from scratch, but rather fine-tuned based on the base model using LoRA (Low-Rank Adaptation) technology. Its training dataset D consists of two parts: D = D H ∪ D B Among them, D H The dataset for secure recovery contains triples (x, h, r), representing a malicious instruction, a malicious prefix (JailbrokenResponse Prefix), and a secure rejection response, respectively. During training, the prefix length k follows a specific distribution P. k The model is forced to "turn the tide" and output a rejection response r even after seeing a harmful prefix. B To preserve the utility of the dataset, benign instructions and their high-quality responses are included to maintain the model's generalizability. The composite loss function for training is as follows: L = α*E[(x, h, r) ~ D H ][-log p θ′ (r|x, h≤k)]+(1-α)*E[(x′, y′)~D B ][-log p θ′ [y′|x′)] where α is the balance coefficient (taken as 0.2 in this embodiment), p θ′ This represents the probability distribution of the SATE model. Through this training, SATE can intercept spoofed attacks that are mistakenly identified as "low-risk" by the gateway.
[0030] Step S4: Distributed Guided Thinking Decoding (DDGT) For uncertain risk queries, the DDGT strategy dynamically intervenes at each time step n of the decoding process. (1) Dynamically construct a shared vocabulary: Obtain the candidate token sets of the base model and the SATE model respectively, and expand them until the intersection of the two contains at least k tokens, forming a shared vocabulary U. n (2) Calculate distribution similarity: Calculate the cosine similarity (sim) of the probability distributions of the two models on the shared vocabulary. n :sim n =cos(p θ (·|x_<n), p θ′ (·|x<n))(3) Dual-mode intervention decision-making: Set an intervention threshold τ (e.g., 0.9). Mode 1: Adversarial Intervention. When sim n When the value is less than τ, it indicates that the base model may be generating harmful content (significantly deviating from the distribution of security experts). At this point, the system forcibly discards the output of the base model and directly selects the token with the highest probability from the SATE model as the output y for the current step. n =argmax p θ′ (y). Mode 2: Cooperative Decoding. When sim n When the value is ≥τ, it indicates that the two values are consistent, and the risk is low. In this case, weighted interpolation is used to generate the final probability distribution: P n (·)=(1-λ)*p θ (·)+λ*p θ′ (·) where λ controls the influence of the expert model.
[0031] Step S5: To quantify the defensive capabilities of this method, an independent automated evaluation process is employed.
[0032] In this embodiment, a specially trained DeepSeek classification model is used as the evaluator. This evaluator is trained on a labeled dataset containing various types of harmful content and is capable of accurately identifying security violations in text. The evaluation process is as follows:
[0033] The system's final output for all queries (including harmful and benign) in the test set is submitted to the evaluator, which determines whether the output contains harmful content. The attack success rate is calculated as: ASR = (Number of Successful Jailbreaks) / (Total Number of Harmful Queries). Comprehensive evaluation on benchmark sets including various advanced attacks such as GCG, AutoDAN, PAIR, and Jailbroken demonstrates that this method can significantly reduce ASR. Simultaneously, utility retention is evaluated on a benign task dataset to ensure that the defense measures do not unduly impact normal use.
[0034] In this embodiment, the evaluation also includes uncertainty analysis, which involves statistically analyzing the proportion of queries that are determined to be uncertain and the proportion of these uncertain queries that are ultimately confirmed as attacks, thereby verifying the effectiveness of the uncertainty judgment.
Claims
1. A security defense method and system based on a large language model with dynamic risk reasoning, characterized in that, Includes the following steps: Step S1, Security Classification and Diversion: Receive user input queries, and use the query risk reasoning gateway built on the frozen basic large language model to judge the input, calculate the harmless probability and the harmful probability, and divide the input query into three levels of high risk, low risk or uncertain risk according to the relationship between the difference between the harmless probability and the harmful probability and the preset threshold, and divert the input to the corresponding processing path. Step S2, Risk Branch Decision: When the classification result is high risk, the standardized rejection mechanism is triggered, and a predefined rejection template is directly output; when the classification result is low risk, the security-aware twin expert model is invoked to generate a response; when the classification result is uncertain risk, the distributed-guided thinking decoding branch is entered; Step S3, Distributed-Guided Thinking Decoding: During the decoding process, the basic large language model and the security-aware twin expert model perform synchronous reasoning on the input respectively, dynamically calculate the similarity between the two generated distributions, and dynamically select an adversarial intervention strategy or a cooperative decoding strategy to generate the final response based on the comparison result of the similarity and the intervention threshold.
2. The method according to claim 1, characterized in that, The query risk inference gateway is a probabilistic classification head, and its construction method includes: extracting features from the last hidden layer of the basic large language model as input, training it using a dataset containing benign and harmful instructions, and outputting a harmless probability P that satisfies the normalization condition during inference. safe And the probability of harm P harm .
3. The method according to claim 2, characterized in that, The specific rule for classifying risk levels in step S1 is as follows: An uncertainty threshold of δ is set; when P... harm >P safe And |P harm -P safe When |≥δ, it is judged as high risk; when P harm <P safe And |P harm -P safe When |≥δ, it is judged as low risk; when |P harm -P safe When | < δ, it is judged as an uncertain risk.
4. The method according to claim 1, characterized in that, The method for constructing the security-aware twin expert model includes the following steps: initializing parameters based on a fundamental large language model, using low-rank adaptation (LoRA) or full-parameter fine-tuning strategies; constructing a security recovery dataset containing malicious instruction triples (x, h, r), where x is a jailbreak attack instruction, h is a malicious prefix, and r is a security denial response; constructing a utility protection dataset containing benign instructions and their high-quality responses (x′, y′); and training using a composite loss function, wherein the loss function is: in, The total loss value is given by α, which is the balancing factor. For the secure recovery dataset, Let θ′ be the predicted probability distribution of the security-aware twin expert model, and k be the prefix length distribution P. k The cutoff length of the mid-sample, h ≤k This represents the first k tokens with the harmful prefix h.
5. The method according to claim 1, characterized in that, The specific implementation process of distributed guided thinking decoding in step S3 includes: at each decoding time step n, obtaining the candidate token sets of the basic large language model and the security-aware twin expert model respectively, and expanding the sets until the intersection of the two contains at least k tokens, forming a shared vocabulary U. n ; Calculate the two models in the shared vocabulary U n cosine similarity of probability distributions on sim n .
6. The method according to claim 5, characterized in that, The dynamic selection strategy based on the comparison results of similarity and intervention threshold includes: setting the intervention threshold to τ; when sim < τ, determining that there is high divergence between models, executing an adversarial intervention strategy, and directly selecting the token with the highest probability in the security-aware twin expert model as the output of the current step ... n When the probability distribution of the two models reaches a consensus, a cooperative decoding strategy is executed, the probability distribution of the two models is weighted and interpolated, and the current step output is generated by sampling from the fused distribution.
7. The method according to claim 6, characterized in that, The weighted interpolation formula in the cooperative decoding strategy is: P n (·)=(1-λ)*p θ (·)+λ*p θ′ (·), where p θ (·) is the probability distribution of the basic large language model, p θ′ (·) represents the probability distribution of the security-aware twin expert model, and λ is the weighting coefficient for controlling the influence of experts.
8. A large language model security defense system, used to perform the method as described in any one of claims 1 to 7, characterized in that, include: The input receiving module is used to receive user input queries; The risk reasoning gateway module is used to calculate the risk probability of the input query and perform three-way traffic splitting; The security-aware twin expert module is used to generate responses on low-risk paths and provide security guidance on uncertain-risk paths. The distribution-guided decoding module is used to calculate the model distribution similarity and perform dynamic interventions under uncertain risk paths; The response output module is used to output the final generated text or a standard rejection template.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.