A Security Decision-Making Method Based on Large Language Models Using Machine Learning
By generating a security posture vector from the hidden state sequence of multi-turn dialogues, and utilizing a security bias generation network and an attention-additive bias mask, the problems of lag and high false positive rate in multi-turn dialogue security defense are solved, achieving precise defense against progressive inducing attacks and ensuring the fluency of normal dialogue.
Patent Information
- Application Number
- CN202611116177.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-27
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies for multi-turn dialogue security defense suffer from problems such as delayed defense response and high false positive rate for legitimate requests. They are unable to effectively identify and defend against progressively induced attacks, and existing methods cannot impose fine-grained selective constraints on the generated content during the model decoding process.
By acquiring the contextual hidden state sequence of multi-turn dialogues, a security posture vector is generated. Then, by using a security bias generation network and an attention additive bias mask, the decoding process of the model is dynamically adjusted to achieve proactive awareness and selective defense against progressively induced attacks.
It achieves accurate detection and defense against progressively induced attacks, reduces the impact of false alarms on normal dialogue, balances security and usability, and introduces minimal latency in defense computation, making it easy to deploy at low cost in existing models.
Smart Images

Figure CN123069449A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of large language model technology, and in particular to a secure decision-making method for large language models based on machine learning. Background Technology
[0002] With the widespread application of Large-Scale Language Models (LLMs) in fields such as intelligent question answering, medical consultation, and legal assistance, multi-turn dialogue services have become a core method for complex information interaction between humans and machines. In such interactions, the model utilizes contextual representations accumulated over consecutive turns to understand user intent and generate coherent responses, thereby effectively supporting deep reasoning and progressive problem-solving. However, this context-dependent generation mechanism also provides a new attack surface for malicious users. Attackers can construct multi-turn progressively induced dialogues, maintaining normal and harmless interactions in the first few turns to lull the model into a false sense of security. Once the dialogue context has reached a state of security desensitization, attackers can suddenly embed illegal commands in subsequent turns, inducing the model to output dangerous content that would otherwise be blocked by security mechanisms.
[0003] Current mainstream defense technologies mainly include single-turn review modules independent of the generation process, preset global security prompts, and output filters based on discriminative models. Among these, single-turn review mechanisms make decisions based solely on the input and output of the current turn, lacking the ability to track the evolution of attack intent across turns; global security prompts attempt to constrain model behavior through static instructions, but their effectiveness diminishes with increasing dialogue context length and is easily bypassed by carefully designed adversarial prompts; output filters can only intercept or rewrite content after the model has generated complete content, unable to intervene during the decoding process. In summary, existing technologies have two prominent problems in multi-turn dialogue security defense: First, the defense response exhibits severe lag. Because security review and the protected generation process are independent, the system only performs danger assessment after the model outputs. By this time, attack-induced dangerous content, and even some intermediate inference steps, have already been generated and may enter the model's internal state as context for subsequent turns, causing the continuous accumulation and propagation of security risks. When attackers employ segmented attack strategies such as splitting and replacing sensitive entities, delayed review becomes even more difficult to identify global attack intent through local feature matching of single-round outputs, leading to defense failure. Secondly, existing methods for proactively adjusting model behavior often result in high false positive rates, meaning that while improving security, they significantly sacrifice the availability of legitimate requests. Whether by injecting prompts that strengthen security constraints or by imposing forced biases on model outputs, these methods operate uniformly across all decoding processes globally, failing to distinguish between normal semantic needs and potential attack semantics. This causes a large number of legitimate users' legitimate inquiries to be incorrectly rejected or the quality of responses to be significantly reduced, which is particularly unacceptable in professional fields such as medicine and law, where semantic accuracy is extremely important.
[0004] Chinese Patent Publication No. CN120542579A discloses a method for detecting and defending against multi-sample attacks on large-scale language models. This method includes constructing a multi-layered text screening and intervention mechanism. By deduplicating input content, adjusting formatting, and perturbing context, the impact of multi-sample attacks is reduced. Similarity analysis is performed on the text to identify highly repetitive examples, which are then filtered using an efficient matching strategy. A pre-trained model is used to map the semantic features of the text, combined with historical data comparison, to determine if there is any potential for misleading attacks. The method also analyzes multi-turn dialogue structures, extracts and analyzes user-system interaction patterns, and dynamically adjusts detection rules to improve adaptability to different format-based misleading attacks. For high-risk texts, deletion and truncation strategies are adopted to retain a small amount of core content. For some medium-risk texts, the order is adjusted or interfering information is inserted to reduce their impact. Therefore, the multi-sample attack detection and defense method for large-scale language models has the following problems: its security defense is based on external filtering and rule statistics at the input end, which cannot dynamically perceive and continuously track the evolution of intentions in progressively induced attacks across rounds. Faced with attackers' strategies of gradually desensitizing text through multiple rounds of normal dialogue before implanting illegal commands, it lacks the ability to proactively intervene before the attack intent escalates, resulting in a delayed response to long-term preparatory attacks. Furthermore, the deletion, truncation, and sentence order adjustment methods used are applied indiscriminately to the entire input text once triggered, failing to impose fine-grained selective constraints on the generated content during model decoding, thus posing a high risk of mistakenly penalizing normal dialogue. Summary of the Invention
[0005] To address this, the present invention provides a security decision-making method based on a large language model using machine learning, which overcomes the technical problems of delayed defense response of large language models to multi-round progressive inducing attacks and high false positive rate of normal requests due to global security constraints in the prior art.
[0006] To achieve the above objectives, this invention provides a security decision-making method based on a large language model using machine learning, comprising: Step S1: Obtain the context hidden state sequence corresponding to the current round of dialogue. The context hidden state sequence is obtained by encoding the multi-round dialogue history by an open-source large language model decoder. Step S2: Based on the context hidden state sequence, determine the cumulative situational security vector representing the current dialogue in the multi-dimensional risk dimension; Step S3: Input the security situation vector into the security bias generation network to obtain the antibody bias vector sequence. The security bias generation network is obtained by training with joint loss. The joint loss includes security loss and subspace projection constraint loss. The subspace projection constraint loss is used to constrain the projection strength of the antibody bias vector sequence on the main subspace of normal dialogue semantics to meet preset conditions. Step S4: When the next round of decoding starts, an attention additive bias mask is generated according to the antibody bias vector sequence, and the attention additive bias mask is applied to the self-attention matrix of each layer of the open source large language model. The degree of application of the attention additive bias mask is adjusted by the content-aware gating coefficient, which is determined according to the relationship between the semantic features of the token to be generated and the main subspace of the normal dialogue semantics. In step S5, during security monitoring in parallel with the decoding process, when the security risk score of the token being generated is detected to be greater than or equal to the emergency threshold, a generation termination and rollback operation is performed, and a security response sequence is regenerated based on the self-attention mechanism with the attention additive bias mask applied.
[0007] Further, step S2 includes: Step S21: Obtain a predefined learnable situation query vector; Step S22: Using the situation query vector as the query and the context hidden state sequence as the key and value, perform cross-attention calculation to obtain the context-weighted representation; Step S23: Determine the security situation vector based on the context-weighted representation.
[0008] Furthermore, the multidimensional risk dimensions include the intensity of inducing tendency, the ambiguity of semantic boundaries, and the activation degree of dangerous entities.
[0009] Further, in step S3, the joint loss training includes: Construct a normal dialogue dataset and obtain the normal dialogue hidden state sequence extracted from the dialogues in the normal dialogue dataset by an open-source large language model decoder; Principal component analysis is performed on the hidden state sequence of the normal dialogue to extract principal components and construct the semantic principal subspace of the normal dialogue. The subspace projection constraint loss is determined based on the normal dialogue semantic principal subspace, and the parameters of the safety bias generation network are updated based on the joint loss.
[0010] Furthermore, the subspace projection constraint loss is determined as follows: Project the antibody bias vector sequence onto the normal dialogue semantic principal subspace and calculate the norm of the projection result; When the norm of the projection result is greater than a preset threshold, a subspace projection constraint loss is generated, and the subspace projection constraint loss is positively correlated with the norm of the projection result. When the norm of the projection result is less than or equal to a preset threshold, the subspace projection constraint loss is zero.
[0011] Further, in step S4, determining the content-aware gating coefficient includes: Get the hidden state of the token to be generated in the open-source large language model decoder; The semantic deviation metric is determined based on the projection intensity of the hidden state onto the semantic principal subspace of the normal dialogue and the total intensity of the hidden state. Based on the semantic deviation metric, the content-aware gating coefficient is obtained through a monotonically increasing mapping, wherein the larger the semantic deviation metric, the larger the content-aware gating coefficient.
[0012] Further, in step S4, generating the attention additive bias mask includes: The antibody bias vector sequence is linearly mapped, and the dimension of the mapped vector is adjusted to match the number and dimension of the attention heads in the open-source large language model. The adjusted bias vector is split into multiple additive bias sub-masks according to the attention head; The multiple additive bias submasks are superimposed onto the self-attention matrix of the corresponding attention head to determine the modified attention matrix.
[0013] Furthermore, step S5 includes the following steps: after decoding and generating each token, if the security risk score of the token is lower than the security threshold, the current position is recorded as a security checkpoint; if the security risk score of the currently generated token is detected to be greater than or equal to the emergency threshold, decoding is paused and the decoding state is rolled back to the most recently recorded security checkpoint.
[0014] Furthermore, in step S5, the security monitoring is performed by a lightweight security classifier; The lightweight security classifier takes the hidden state of the currently generated token as input and outputs a security risk score. The lightweight security classifier runs asynchronously and in parallel with the decoding process of the open-source large language model.
[0015] Furthermore, in step S5, starting from the hidden state at the security checkpoint, the autoregressive decoding is re-executed based on the self-attention mechanism with the attention additive bias mask applied, to generate a security response sequence.
[0016] Compared with the prior art, the beneficial effects of the present invention are that it transforms the hidden states in the multi-turn dialogue history into a security situation vector and dynamically generates an antibody bias vector sequence under the orthogonal constraints of the main subspace of normal dialogue semantics. Combined with content-aware gating and emergency rollback resampling mechanism, it achieves forward perception and selective defense against progressively induced attacks. This effectively blocks the generation of danger while significantly reducing the impact of accidental damage to normal dialogue, thus balancing security and usability.
[0017] Furthermore, this invention, through cross-attention situational extraction based on the native hidden state of a large language model, can fully preserve early background information in the dialogue history, accurately perceive the evolution of progressive attack intentions, and overcome the risk of missed detections caused by long-term forgetting.
[0018] Furthermore, this invention restricts the energy of the antibody bias vector outside the normal semantic direction by projecting constraints on the principal subspace of normal dialogue semantics, thereby eliminating the interference of defense bias on legitimate requests from the source of the generated probability distribution.
[0019] Furthermore, this invention quantifies in real time the degree to which the generated token deviates from normal semantics through a content-aware gating coefficient, and dynamically adjusts the intervention intensity of the antibody bias accordingly, achieving precise intervention only on high-risk semantic paths, thereby further ensuring the fluency of normal dialogue and the quality of responses.
[0020] Furthermore, this invention, through a parallel lightweight security classifier and a security checkpoint fallback mechanism, can block the dangerous token at the moment of its generation and regenerate the response from the secure state, thus preventing the spread of dangerous content in the context. The entire process strictly follows autoregressive causal logic.
[0021] Furthermore, this invention uses an asynchronous computing architecture to execute security situation awareness and antibody bias vector generation in parallel with the main model inference pipeline, resulting in minimal additional latency introduced by defense computation and facilitating low-cost plug-in deployment within existing open-source large language model inference frameworks. Attached Figure Description
[0022] Figure 1 This is a flowchart of the security decision-making method based on a large language model using machine learning, as described in this invention. Figure 2 This is a flowchart of step S2 of the security decision-making method based on a large language model using machine learning according to the present invention; Figure 3 This is a logical block diagram of the subspace projection constraint loss calculation of the security bias generation network in the security decision-making method based on a large language model of machine learning according to the present invention. Figure 4 This is a logic block diagram of the content-aware gating coefficient determination and attention additive bias mask application in the security decision-making method based on a large language model using machine learning, as described in this invention. Detailed Implementation
[0023] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.
[0024] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.
[0025] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.
[0026] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.
[0027] Please see Figure 1 The flowchart shown is a security decision-making method for a large language model based on machine learning, according to the present invention. The present invention provides a security decision-making method for a large language model based on machine learning, comprising: Step S1: Obtain the context hidden state sequence corresponding to the current round of dialogue. The context hidden state sequence is obtained by encoding the multi-round dialogue history by an open-source large language model decoder. This invention provides a security decision-making system for open-source large language models, including a context hidden state extraction unit, a cross-attention pooler, a security bias generation network, an attention additive bias mask generation unit, a content-aware gating unit, a security monitoring unit, and an asynchronous computation unit.
[0028] The context hidden state extraction unit is used to obtain the context hidden state sequence corresponding to the current round of multi-turn dialogue history from the open-source large language model decoder; Cross-attention pooling is used to generate a security posture vector based on the contextual hidden state sequence; Security bias generation network is used to map security posture vectors to antibody bias vector sequences; The attention additive bias mask generation unit is used to convert the antibody bias vector sequence into an additive bias mask acting on the self-attention matrix; The content-aware gating unit is used to adjust the strength of the additive bias mask based on the projection degree of the hidden state of the currently generated token onto the normal dialogue semantic master subspace. The security monitoring unit is used to assess the security risks of the generated token in parallel during the decoding process and trigger generation termination and rollback operations when the risks exceed the emergency threshold; The asynchronous computation unit is used to pre-generate and cache the bias parameters required for the next round in a computation process independent of the inference engine.
[0029] This system is used to implement the security decision-making method of the large language model based on machine learning of the present invention.
[0030] In this embodiment, the specific implementation process of step S1 is as follows: The open-source large language model maintains a dialogue history buffer in the service process, which stores text pairs of user input and model response in turn order.
[0031] When the system receives input from the user in the current round, it concatenates the entire dialogue history in the buffer with the current user input to form a complete input sequence, which is then sent to the open-source large language model decoder.
[0032] The open-source large language model decoder performs token-by-token autoregressive processing on the input sequence based on the Transformer architecture. Through multi-layer self-attention mechanism and stacked computation of multi-layer perceptron, it generates a set of hidden state vectors for each token in the sequence. These hidden state vectors are distributed in the hidden layers of the decoder.
[0033] The context hidden state extraction unit obtains the hidden states of all tokens from the preset extraction layer of the decoder and arranges them in the order of the original positions of the tokens in the input sequence to form a context hidden state sequence.
[0034] The vector at each position in the sequence not only contains the independent semantic information of the corresponding token, but also fully integrates the contextual semantics of other tokens in the entire dialogue history through attention calculation, so that the sequence as a whole can represent the complete semantic state and intention progression relationship of the current multi-turn dialogue with high fidelity.
[0035] The obtained contextual hidden state sequence is then passed to the cross-attention pooler as the basic input for subsequent security situation awareness.
[0036] This invention establishes security situation awareness directly on the native hidden state sequence of a large language model, avoiding the information compression loss and additional computational latency caused by an independent encoder. Since the large language model has already learned strong modeling capabilities for long-range semantic dependencies during the pre-training phase, the preparatory information of early dialogue rounds in its hidden state sequence is effectively preserved. When an attacker engages in normal interactions in the first few rounds to relax security constraints, the progressive correlation between these masquerading semantics and subsequent dangerous intentions can be fully captured by the hidden state sequence, eliminating the risk of misjudgment of the situation caused by forgetting early attack preparations due to information bottlenecks.
[0037] It is understood that the preset extraction layer for hidden state extraction in this embodiment is not limited to the last layer of the decoder. It can also select several layers from the end of the decoder and concatenate or weightedly fuse the hidden states of each layer, or extract only the tokens within a specific window related to the current user input. All of the above substitution methods can achieve the purpose of obtaining the global semantic representation of multi-turn dialogue, and this embodiment does not make specific limitations on this.
[0038] Step S2: Based on the context hidden state sequence, determine the cumulative situational security vector representing the current dialogue in the multi-dimensional risk dimension; Please continue reading. Figure 2 The diagram shows a flowchart of step S2 of the security decision-making method based on a large language model using machine learning according to the present invention. Specifically, step S2 includes: Step S21: Obtain a predefined learnable situation query vector; Step S22: Using the situation query vector as the query and the context hidden state sequence as the key and value, perform cross-attention calculation to obtain the context-weighted representation; Step S23: Determine the security situation vector based on the context-weighted representation.
[0039] Specifically, the multidimensional risk dimensions include the intensity of inducing tendency, the ambiguity of semantic boundaries, and the activation degree of dangerous entities.
[0040] In this embodiment, step S2 is performed by the cross-attention pooler; The cross-attention pooler reads a learnable situation query vector from a pre-defined parameter storage area. The vector has the same dimension as the hidden state. Its initial value is determined randomly before training and continuously optimized using gradient descent in subsequent end-to-end joint training to learn the ability to focus on risk-related information from multi-turn dialogue history.
[0041] Furthermore, using the situation query vector as the query and the contextual hidden state sequence obtained in step S1 as both the key and value, cross-attention calculation is performed. Specifically, the query, key, and value are linearly transformed using a trainable projection matrix, the dot product of the query and key is calculated, and divided by a scaling factor to stabilize the training process. After processing with a normalized exponential function, the attention weight distribution at each position is obtained. The value vector is then weighted and summed using these weights to generate a weighted representation that incorporates contextual risk information, enabling the query vector to adaptively aggregate key information related to the security situation.
[0042] Furthermore, the aforementioned context-weighted representation is input into a feedforward mapping network. This network contains three independent linear projectors, corresponding to the inducing tendency strength, semantic boundary ambiguity, and dangerous entity activation, respectively. Each projector maps the representation to an evaluation value for the corresponding dimension. These three evaluation values are concatenated in sequence to obtain the security posture vector. The magnitude of each component of this vector reflects the cumulative degree of the current dialogue in the corresponding dimension in real time.
[0043] Specifically, the parameters of the aforementioned situation query vector, projection matrices, and feedforward mapping network all need to be determined through pre-supervised training. During training, a batch of dialogue samples is used, each of which has a pre-constructed multi-dimensional risk label value. The label values are constructed as follows: the highest risk dimension is determined based on whether the final round of the dialogue contains a violation command, and combined with the backtracking analysis of the attack steps, the induction tendency strength and semantic boundary ambiguity score of the intermediate rounds are manually defined using rules. During training, the weighted sum of the squares of the differences between the predicted values of each dimension and their corresponding label values is used as the risk dimension prediction loss. This loss is then combined with the security loss of the subsequent security bias generation network, and all parameters are updated uniformly until convergence through gradient backpropagation in an end-to-end manner. This ensures that each component of the security situation vector can independently and accurately characterize different risk categories.
[0044] This invention parameterizes the query vector used in cross-attention into a learnable form and performs end-to-end joint training with the overall defense target under the supervision of multi-dimensional risk scoring. This allows the extraction of the security situation vector to no longer rely on manually designed feature rules, but can automatically discover implicit semantic clues related to security threats from large-scale dialogue samples. Under this mechanism, the situation query vector is equivalent to an adjustable risk probe. After training, it can actively focus on segments in the dialogue history that have important indicative significance for security decisions, forming accurate situational awareness. Furthermore, by using multiple independent risk dimension projection heads, the situational information is explicitly decoupled into key components such as the strength of inducing tendencies, semantic boundary ambiguity, and activation degree of dangerous entities. This allows the system to finely distinguish different stages of attack evolution, such as identifying the difference between inducing preludes and direct violation commands. This provides a discriminative modulation signal for the subsequent antibody bias generation network, applying appropriate defensive biases before danger occurs while avoiding unnecessary interference with normal dialogue content that is still within the security boundary.
[0045] It is understood that, in this embodiment, the cross-attention calculation can be replaced by an additive attention mechanism or a variant based on bilinear attention, in addition to dot product attention. The initialization method for the situation query vector can also utilize the output of a semantic encoder pre-trained on large-scale security data to accelerate training convergence, which will not be elaborated further here.
[0046] Specifically, the multidimensional risk labeling values of the training samples can be constructed in the following way: the highest risk dimension is assigned based on whether the final round of the dialogue contains illegal instructions, and then the intensity of the inducing tendency and the semantic boundary ambiguity score of the intermediate rounds are defined manually or with the help of rules based on the back-tracking analysis of the attack steps. Alternatively, an independently trained risk assessment model can be used to automatically score each round of the historical dialogue, and the scores can be normalized and used as training pseudo-labels.
[0047] Those skilled in the art will understand that the specific source of the annotation does not affect the model structure of the present invention, and no strict limitation is imposed here.
[0048] This invention sets the query vector of cross-attention as a learnable parameter and performs end-to-end training under multi-dimensional risk supervision signals. This allows the extraction of the situation vector to no longer rely on manual feature definitions, but instead automatically discovers implicit semantic clues in the dialogue history that pose a threat to security. The explicit decoupling of the three dimensions further enables the defense system to distinguish between different stages of risk evolution, such as differentiating between inductive preludes and direct dangerous commands. This provides fine-grained modulation basis for the subsequent antibody bias generation network, thereby ensuring security while avoiding accidental intervention in dialogues still within the security boundary.
[0049] Understandably, the number of output heads in a feedforward mapping network is not limited to three; other risk dimensions, such as the strength of social engineering intent, can be added based on specific security requirements. The cross-attention calculation method can also be replaced with an additive attention mechanism or a variant based on bilinear attention. Furthermore, the training initialization method for the situation query vector can utilize the output of a semantic encoder pre-trained on large-scale security data to accelerate convergence.
[0050] Step S3: Input the security situation vector into the security bias generation network to obtain the antibody bias vector sequence. The security bias generation network is obtained by training with joint loss. The joint loss includes security loss and subspace projection constraint loss. The subspace projection constraint loss is used to constrain the projection strength of the antibody bias vector sequence on the main subspace of normal dialogue semantics to meet preset conditions. Specifically, in step S3, the joint loss training includes: Construct a normal dialogue dataset and obtain the normal dialogue hidden state sequence extracted from the dialogues in the normal dialogue dataset by an open-source large language model decoder; Principal component analysis is performed on the hidden state sequence of the normal dialogue to extract principal components and construct the semantic principal subspace of the normal dialogue. The subspace projection constraint loss is determined based on the normal dialogue semantic principal subspace, and the parameters of the safety bias generation network are updated based on the joint loss.
[0051] Please continue reading. Figure 3 As shown, this is a logical block diagram of the subspace projection constraint loss calculation of the security bias generation network in the security decision-making method based on a large language model of machine learning according to the present invention; specifically, the subspace projection constraint loss is determined as follows: Project the antibody bias vector sequence onto the normal dialogue semantic principal subspace and calculate the norm of the projection result; When the norm of the projection result is greater than a preset threshold, a subspace projection constraint loss is generated, and the subspace projection constraint loss is positively correlated with the norm of the projection result. When the norm of the projection result is less than or equal to a preset threshold, the subspace projection constraint loss is zero.
[0052] In this embodiment, step S3 is performed by the security bias generation network, which transforms the security posture vector obtained in step S2 into an antibody bias vector sequence. The security bias generation network is a feedforward neural network whose structure includes an input layer, at least one hidden fully connected layer, and a set of output projection layers. The input layer receives the security posture vector and maps it into a dense intermediate feature representation using the hidden layer and a nonlinear activation function. The output projection layers consist of multiple independent linear mappings, with the number of mappings equal to the number of layers in the open-source large language model decoder. Each linear mapping directly transforms the intermediate feature representation into a vector with the same dimension as the hidden state, serving as the corresponding decoder layer's antibody bias vector. The vectors from all layers are arranged in decoder layer order, thus forming the antibody bias vector sequence.
[0053] The parameters of the safety bias generation network are predetermined through offline joint loss training, and the training process is as follows: Construct a sufficiently large dataset of normal dialogues, collecting multi-turn dialogue samples without any malicious intent. Feed these samples one by one into an open-source large language model decoder, extract the hidden state vector of each token at a preset layer, and aggregate the hidden state vectors of all tokens from all samples into a sequence of hidden states for normal dialogues.
[0054] Furthermore, principal component analysis is performed on the collected hidden state vectors of normal dialogue: first, the mean vector is calculated and centered, and a covariance matrix is constructed. Eigenvalues and corresponding eigenvectors are obtained through eigenvalue decomposition. According to the order of eigenvalues from largest to smallest, the top few principal components whose cumulative variance contribution rate reaches the preset coverage threshold are selected, and their corresponding eigenvectors are used as basis vectors to jointly span the principal subspace of normal dialogue semantics.
[0055] Furthermore, training is performed using dialogue samples that incorporate progressively induced attacks. For each training sample, a security posture vector is obtained through steps S1 and S2 and input into the security bias generation network to obtain a set of antibody bias vector sequences. The joint loss during training consists of two parts: security loss and subspace projection constraint loss.
[0056] Specifically, the security loss is calculated as follows: the antibody bias vector sequence is temporarily converted into an attention-additive bias mask and applied to the open-source large language model. Under the condition of inputting a dangerous request, the model generates the expected log probability of a preset safe rejection response. The negative of this value is taken as the security loss, which aims to prompt the network to generate bias vectors that can suppress dangerous outputs. The subspace projection constraint loss is calculated as follows: for the obtained antibody bias vector sequence, each bias vector is projected onto the normal dialogue semantic master subspace to obtain its own projection vector; the Euclidean length of each projection vector is calculated, and the square root of the sum of the squares of these lengths is taken to obtain the total projection intensity. The total projection intensity is compared with a preset threshold. When the total projection intensity is greater than the threshold, a subspace projection constraint loss is generated, and the loss value is positively correlated with the amount by which the total projection intensity exceeds the threshold, that is, the greater the total projection intensity, the greater the loss value; when the total projection intensity is less than or equal to the threshold, the loss value is zero. This loss constraint forces the energy of the bias vector to concentrate in directions outside the normal semantic master subspace to reduce interference with normal dialogue.
[0057] The specific value of the preset projection threshold in the joint loss can be adjusted according to the trade-off between security and availability in the target application scenario. When there are higher requirements for security, the threshold can be appropriately reduced to enhance the constraint.
[0058] The safety loss and subspace projection constraint loss are weighted and summed according to preset weight coefficients to form the joint loss. To minimize the joint loss, a gradient descent optimizer is used to update all trainable parameters of the safety bias generation network. This iterative process is repeated until the joint loss converges, at which point the network parameters are fixed. In the inference phase of a real multi-turn dialogue, whenever step S2 outputs the safety situation vector for the current turn, the safety bias generation network directly calculates and outputs the antibody bias vector sequence through a single forward propagation, which is then passed to the subsequent step S4.
[0059] This invention directly shapes the spatial distribution characteristics of the antibody bias vector through subspace projection constraints, severely suppressing the component of the defense signal in the normal dialogue semantic direction. During the decoding process along the normal semantic path, the hidden state of the token to be generated is highly concentrated in the main subspace of the normal dialogue semantics. At this time, the component of the bias vector in this space is extremely small, the interaction between the defense signal and the normal semantic representation is very weak, and the influence of the bias mask on the attention distribution approaches zero, thus ensuring the fluency and response quality of the normal dialogue. When decoding faces induced or dangerous expressions, the semantic representation deviates significantly from the main subspace, allowing the antibody bias vector to play a significant directional corrective role in the direction deviating from normal semantics, driving the model towards a safe response. This subspace-based selective defense mechanism fundamentally decouples security intervention from legitimate semantic processing, enabling the model to accurately block progressive attacks in complex multi-turn dialogues while significantly reducing the risk of false positives on legitimate requests, thus balancing security and usability.
[0060] Understandably, the specific number of hidden layers and the specific type of nonlinear activation function in the security bias generator network can be adjusted according to the computing power and model size of the actual deployment environment, as long as the mapping from the security posture vector to the layer-matching bias vector can be achieved. The specific value of the preset projection threshold in the joint loss can be adjusted according to the trade-off between security and availability in the target application scenario. When higher security is required, the threshold can be appropriately reduced to enhance the constraint. The optimizer used for training can also be replaced by other adaptive learning rate optimizers or stochastic gradient descent optimizers with momentum, as long as stable convergence of training is guaranteed.
[0061] Step S4: When the next round of decoding starts, an attention additive bias mask is generated according to the antibody bias vector sequence, and the attention additive bias mask is applied to the self-attention matrix of each layer of the open source large language model. The degree of application of the attention additive bias mask is adjusted by the content-aware gating coefficient, which is determined according to the relationship between the semantic features of the token to be generated and the main subspace of the normal dialogue semantics. Please continue reading. Figure 4 As shown, this is a logic block diagram of the content-aware gating coefficient determination and attention additive bias mask application in the security decision-making method based on a large language model using machine learning according to the present invention; specifically, in step S4, the determination of the content-aware gating coefficient includes: Get the hidden state of the token to be generated in the open-source large language model decoder; The semantic deviation metric is determined based on the projection intensity of the hidden state onto the semantic principal subspace of the normal dialogue and the total intensity of the hidden state. Based on the semantic deviation metric, the content-aware gating coefficient is obtained through a monotonically increasing mapping, wherein the larger the semantic deviation metric, the larger the content-aware gating coefficient.
[0062] Specifically, in step S4, generating the attention additive bias mask includes: The antibody bias vector sequence is linearly mapped, and the dimension of the mapped vector is adjusted to match the number and dimension of the attention heads in the open-source large language model. The adjusted bias vector is split into multiple additive bias sub-masks according to the attention head; The multiple additive bias submasks are superimposed onto the self-attention matrix of the corresponding attention head to determine the modified attention matrix.
[0063] In this embodiment, step S4 is executed collaboratively by the attention additive bias mask generation unit and the content-aware gating unit. The antibody bias vector sequence obtained in step S3 is transformed into a dynamic bias mask that acts on the self-attention matrix of each layer of the open-source large language model, and the application strength of the bias is adjusted by the real-time semantic deviation of the token to be generated.
[0064] The attention additive bias mask generation unit receives the antibody bias vector sequence output from step S3. Each level of antibody bias vector in this sequence corresponds to a self-attention layer of the open-source large language model decoder. For each layer, the unit decomposes the layer's antibody bias vector into multiple head-level bias vectors using a preset linear mapping relationship. The number of decompositions is equal to the number of attention heads in that layer, and the dimension of each head-level bias vector is consistent with the latent space dimension of its corresponding attention head. Then, for each attention head, the head-level bias vector is multiplied by the key embedding matrix of the current decoding sequence under that head to obtain a bias score vector with a length equal to the length of the current decoding sequence. Next, this bias score vector is copied along the query dimension to form an additive bias sub-mask with the exact same shape as the self-attention score matrix, where the bias value in each row of this sub-mask is equal to the bias score vector. Through this copying method, the defense bias applies a consistent additive effect to all keys independently of the query position, thereby effectively suppressing information from dangerous keys during the attention aggregation process.
[0065] During token-by-token generation in the decoding process, the content-aware gating unit dynamically adjusts the applied strength of the bias mask. When the model is ready to generate the token for the current position, the content-aware gating unit reads the hidden state vector at that moment from the decoder. Using the basis of the normal dialogue semantic principal subspace constructed in step S3, the Euclidean length of the projection vector of the hidden state onto the principal subspace is calculated as the projection strength; simultaneously, the Euclidean length of the hidden state itself is calculated as the total strength.
[0066] Furthermore, a semantic deviation metric is determined based on the ratio of projected intensity to total intensity. This semantic deviation metric satisfies the following conditions: the more concentrated the hidden states are within the normal semantic principal subspace, the smaller the deviation metric; conversely, the more the hidden states deviate from the normal semantic principal subspace, the larger the deviation metric. Subsequently, the semantic deviation metric is converted into a content-aware gating coefficient using a pre-defined monotonically increasing mapping function. This coefficient ranges from zero to one, with the gating coefficient approaching one as the semantic deviation metric increases and approaching zero as the semantic deviation metric decreases.
[0067] Furthermore, in the self-attention calculation stage, the attention additive bias mask generation unit multiplies the additive bias sub-masks of each layer and head by the current unified gating coefficient to obtain the gated bias values, which are then superimposed on the original self-attention score matrix of the corresponding attention head according to their element positions to form the corrected attention matrix. The open-source large language model completes subsequent context aggregation and token prediction based on the corrected attention scores.
[0068] Throughout the sequence generation process, for tokens in the normal semantic region, the projection intensity of their hidden state is high, the deviation metric is small, the gating coefficient is close to zero, and the bias mask has almost no effect on the attention distribution. For tokens that gradually deviate from the normal semantic region, the deviation metric of the hidden state increases, the gating coefficient automatically increases, the bias mask significantly changes the attention weights, and guides the model to generate a safe response.
[0069] This invention achieves fine-grained delivery and selective intervention of defense signals by decomposing the hierarchical antibody bias vector according to attention heads and applying content-aware gating. The antibody bias vector is split into various attention head spaces after hierarchical linear mapping, allowing defense energy to deeply influence the attention interactions of multiple representation subspaces within the model, enhancing the global suppression of dangerous outputs. Simultaneously, the gating coefficient is calculated in real-time based on the projection ratio of the currently generated token onto the normal semantic master subspace. When decoding along the normal semantic path, the bias influence is automatically reduced to a very low level to ensure smooth dialogue, while the intervention intensity is automatically increased when the semantic path deviates in a dangerous direction. This establishes a precise positive correlation between defense adjustment and the degree of safety deviation of the generated content, thereby achieving precise security intervention without sacrificing model usability.
[0070] It is understandable that the specific construction method of the semantic deviation metric can be determined not only by using the ratio of projected intensity to total intensity, but also by the length of the projected residual vector or the ratio after nonlinear scaling, as long as it can reflect the degree to which the hidden state deviates from the normal semantic principal subspace. The mapping function of the gating coefficients can be a smooth sigmoid function, a piecewise linear function, or a hard-threshold step function, but all should maintain the monotonically increasing property between the semantic deviation metric and the gating coefficients. The gating coefficients can be shared among different attention heads, or they can be calculated independently for each attention head. This embodiment uses shared coefficients to reduce computational overhead.
[0071] In step S5, during security monitoring in parallel with the decoding process, when the security risk score of the token being generated is detected to be greater than or equal to the emergency threshold, a generation termination and rollback operation is performed, and a security response sequence is regenerated based on the self-attention mechanism with the attention additive bias mask applied.
[0072] Specifically, step S5 includes the following steps: after decoding and generating each token, if the security risk score of the token is lower than the security threshold, the current position is recorded as a security checkpoint; if the security risk score of the currently generated token is detected to be greater than or equal to the emergency threshold, decoding is paused and the decoding state is rolled back to the most recently recorded security checkpoint.
[0073] Specifically, in step S5, the security monitoring is performed by a lightweight security classifier; The lightweight security classifier takes the hidden state of the currently generated token as input and outputs a security risk score. The lightweight security classifier runs asynchronously and in parallel with the decoding process of the open-source large language model.
[0074] Specifically, in step S5, starting from the hidden state at the security checkpoint, the autoregressive decoding is re-executed based on the self-attention mechanism with the attention additive bias mask applied, to generate a security response sequence.
[0075] In this embodiment, step S5 is completed collaboratively by the security monitoring unit, the attention additive bias mask generation unit, and the content-aware gating unit. Through parallel token-level security review, dynamic maintenance of checkpoints, and causal backoff and resampling under emergency conditions, the immediate blocking of danger generation and the replacement output of security response are achieved.
[0076] After a new output token is generated at each time step of the autoregressive decoding of the open-source large language model, a lightweight security classifier deployed in the security monitoring unit immediately obtains the hidden state vector of that token output at the last layer of the decoder. This lightweight security classifier consists of a single hidden layer perceptron with the same input dimension as the hidden state dimension. Its output node is activated by a function to produce a scalar value, which is the security risk score. The higher the score, the greater the likelihood that the current token belongs to a dangerous semantic. The lightweight security classifier runs asynchronously and in parallel with the decoding process, without blocking the token generation rhythm of the main inference pipeline. Its parameters are obtained through binary classification training using pre-collected normal and dangerous dialogue samples.
[0077] The security monitoring unit also maintains a security checkpoint pointer, pointing to the most recent decoding position deemed sufficiently secure. Specifically, after each token is generated and a security risk score is obtained, the system compares this score with a preset security threshold: if the current token's security risk score is less than the security threshold, it indicates that the currently generated content is in a reliable and secure state, and the current position is updated as a security checkpoint; if the current token's security risk score is greater than or equal to the emergency threshold, it indicates that decoding has entered a high-risk area and immediate intervention is required. The security threshold and the emergency threshold satisfy the relationship that the security threshold is less than the emergency threshold, so as to retain a more recent reliable fallback position in the early stages of risk accumulation.
[0078] When a security risk score is detected to be greater than or equal to the emergency threshold, the security monitoring unit issues a pause command to the open-source large language model inference engine, pausing the decoding process at the current token. The system rolls back the entire decoding state to the most recently recorded security checkpoint: restoring the hidden state, key-value cache, and other context states corresponding to that checkpoint, and discarding all generated tokens after that checkpoint. After the rollback is complete, the decoding engine restarts the autoregressive generation process starting from the state of that checkpoint. At this time, the content-aware gating unit and the attention-based additive bias mask generation unit remain active in the regeneration step. The self-attention calculation during decoding continuously superimposes the gated additive bias mask generated in step S4 to ensure that the regenerated sequence completes the security response under the guidance of antibody bias. The entire rollback resampling process strictly follows the autoregressive causal order and does not perform any destructive overwriting of generated tokens.
[0079] Specifically, state transfer between the lightweight security classifier and the decoding process can be achieved through memory sharing or message queues. Each time the decoder generates a token, it writes a copy of the corresponding hidden state to a circular buffer. The classifier reads data from this buffer and asynchronously calculates the security risk score. The score result is fed back through the same mechanism. When the decoding process detects that the risk score exceeds the emergency threshold, it performs a pause and rollback operation. Those skilled in the art will understand that such inter-process coordination methods are mature technologies in the field and will not be elaborated upon here.
[0080] This invention achieves early detection and precise braking of progressive attacks by introducing a lightweight security classifier and a hierarchical threshold monitoring mechanism that runs parallel to decoding. When an attacker suddenly inserts a high-risk command after multiple rounds of preparation, the decoding process often triggers the emergency threshold at the first few dangerous entity words, at which point the nearest checkpoint is still in the boundary zone between preparation and danger. The system backtracks to this boundary zone and resamples based on jointly applied defense bias, causing semantic branches that might have pointed to dangerous answers to be corrected into safe rejection paths, thus cutting off dangerous outputs at the source. Simultaneously, because the security risk score of the token remains below the security threshold throughout normal dialogue, the system does not trigger a backtrack, ensuring smooth and low-latency normal responses.
[0081] Understandably, the structure of a lightweight safety classifier is not limited to a single hidden-layer perceptron; it can also employ a shallow Transformer encoder or a linear discriminator based on word embeddings. The specific values of the safety threshold and the emergency threshold can be dynamically determined by retaining a validation set, based on the trade-off between false positives and false negatives in the target scenario.
[0082] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.
Claims
1. A security decision-making method based on a large language model using machine learning, characterized in that, include: Step S1: Obtain the context hidden state sequence corresponding to the current round of dialogue. The context hidden state sequence is obtained by encoding the multi-round dialogue history by an open-source large language model decoder. Step S2: Based on the context hidden state sequence, determine the cumulative situational security vector representing the current dialogue in the multi-dimensional risk dimension; Step S3: Input the security situation vector into the security bias generation network to obtain the antibody bias vector sequence. The security bias generation network is obtained by training with joint loss. The joint loss includes security loss and subspace projection constraint loss. The subspace projection constraint loss is used to constrain the projection strength of the antibody bias vector sequence on the main subspace of normal dialogue semantics to meet preset conditions. Step S4: When the next round of decoding starts, an attention additive bias mask is generated according to the antibody bias vector sequence, and the attention additive bias mask is applied to the self-attention matrix of each layer of the open source large language model. The degree of application of the attention additive bias mask is adjusted by the content-aware gating coefficient, which is determined according to the relationship between the semantic features of the token to be generated and the main subspace of the normal dialogue semantics. In step S5, during security monitoring in parallel with the decoding process, when the security risk score of the token being generated is detected to be greater than or equal to the emergency threshold, a generation termination and rollback operation is performed, and a security response sequence is regenerated based on the self-attention mechanism with the attention additive bias mask applied.
2. The security decision-making method based on a large language model using machine learning according to claim 1, characterized in that, Step S2 includes: Step S21: Obtain a predefined learnable situation query vector; Step S22: Using the situation query vector as the query and the context hidden state sequence as the key and value, perform cross-attention calculation to obtain the context-weighted representation; Step S23: Determine the security situation vector based on the context-weighted representation.
3. The security decision-making method based on a large language model using machine learning according to claim 2, characterized in that, The multidimensional risk dimensions include the intensity of inducing tendency, the ambiguity of semantic boundaries, and the activation degree of dangerous entities.
4. The security decision-making method based on a large language model using machine learning according to claim 3, characterized in that, In step S3, the joint loss training includes: Construct a normal dialogue dataset and obtain the normal dialogue hidden state sequence extracted from the dialogues in the normal dialogue dataset by an open-source large language model decoder; Principal component analysis is performed on the hidden state sequence of the normal dialogue to extract principal components and construct the semantic principal subspace of the normal dialogue. The subspace projection constraint loss is determined based on the normal dialogue semantic principal subspace, and the parameters of the safety bias generation network are updated based on the joint loss.
5. The security decision-making method based on a large language model using machine learning according to claim 4, characterized in that, The subspace projection constraint loss is determined as follows: Project the antibody bias vector sequence onto the normal dialogue semantic principal subspace and calculate the norm of the projection result; When the norm of the projection result is greater than a preset threshold, a subspace projection constraint loss is generated, and the subspace projection constraint loss is positively correlated with the norm of the projection result. When the norm of the projection result is less than or equal to a preset threshold, the subspace projection constraint loss is zero.
6. The security decision-making method based on a large language model using machine learning according to claim 5, characterized in that, In step S4, determining the content-aware gating coefficient includes: Get the hidden state of the token to be generated in the open-source large language model decoder; The semantic deviation metric is determined based on the projection intensity of the hidden state onto the semantic principal subspace of the normal dialogue and the total intensity of the hidden state. Based on the semantic deviation metric, the content-aware gating coefficient is obtained through a monotonically increasing mapping, wherein the larger the semantic deviation metric, the larger the content-aware gating coefficient.
7. The security decision-making method based on a large language model using machine learning according to claim 6, characterized in that, In step S4, generating an attention additive bias mask includes: The antibody bias vector sequence is linearly mapped, and the dimension of the mapped vector is adjusted to match the number and dimension of the attention heads in the open-source large language model. The adjusted bias vector is split into multiple additive bias sub-masks according to the attention head; The multiple additive bias submasks are superimposed onto the self-attention matrix of the corresponding attention head to determine the modified attention matrix.
8. The security decision-making method based on a large language model using machine learning according to claim 7, characterized in that, Step S5 includes the following steps: after decoding and generating each token, if the security risk score of the token is lower than the security threshold, the current position is recorded as a security checkpoint; if the security risk score of the currently generated token is detected to be greater than or equal to the emergency threshold, decoding is paused and the decoding state is rolled back to the most recently recorded security checkpoint.
9. The security decision-making method based on a large language model using machine learning according to claim 8, characterized in that, In step S5, the security monitoring is performed by a lightweight security classifier; The lightweight security classifier takes the hidden state of the currently generated token as input and outputs a security risk score. The lightweight security classifier runs asynchronously and in parallel with the decoding process of the open-source large language model.
10. The security decision-making method based on a large language model using machine learning according to claim 9, characterized in that, In step S5, starting from the hidden state at the security checkpoint, the autoregressive decoding is re-executed based on the self-attention mechanism with the attention additive bias mask applied, to generate a security response sequence.
Citation Information
Patent Citations
Multi-sample attack detection and defense method oriented to large language model
CN120542579A
