Defense method, system and equipment for jailbreak attack of large language model and storage medium

Abstract

The invention relates to the technical field of model defense, and discloses a defense method, system and device for large language model jailbreak attacks and a storage medium, and the method comprises the steps: collecting high-dimensional intermediate features outputted by each decoder layer through a probe collection model, and carrying out the dimension reduction, and obtaining low-dimensional intermediate features; calculating a first distance between the low-dimensional middle feature and the center of the harmful feature and a second distance between the low-dimensional middle feature and the center of the security feature, obtaining a vector distance difference according to a difference value between the first distance and the second distance, and judging whether a prison break attack risk exists or not; and in response to the existence of the risk, taking the maximization of the vector distance difference of the key layer as a target function, constructing a disturbance optimization model, obtaining an optimal disturbance term, and adding the optimal disturbance term to the corresponding high-dimensional intermediate feature to obtain a safe intermediate feature. According to the method, the real-time dynamic defense capability, interpretability and calculation efficiency of the large language model in the power system are improved, and effective balance between confrontation adjustment and stability of the large language model is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of model defense, and in particular to a defense method, system, device and storage medium for large language model jailbreak attacks. Background Art

[0002] With the deep integration of large language models in the intelligent construction of power systems, their applications in key scenarios such as power grid dispatching instruction generation, equipment operation and maintenance Q&A, and fault diagnosis report writing are becoming increasingly widespread. However, the model faces severe risks of jailbreak attacks in an open interaction environment. Attackers construct concealed instructions through means such as semantic camouflage and context misleading, inducing the model to generate incorrect outputs that violate power safety regulations. Such attacks may trigger systematic risks such as incorrect dispatching instructions and inaccurate operation and maintenance responses, seriously threatening the stable operation of the power grid and personnel safety.

[0003] Currently, the defense technologies of large language models are mainly divided into four categories: static input detection, post-output processing, heuristic rule matching, and feature matching. Among them, static input detection refers to filtering sensitive inputs based on a keyword library or classifier, but this method is difficult to resist semantic recombination attacks; post-output processing refers to filtering the generated content afterwards, but this method has significant defense lag and cannot block real-time instruction errors from being sent to power execution terminals; heuristic rule matching refers to relying on an artificial rule library to intercept abnormal instructions, and this rule library-dependent method is prone to insufficient generalization ability and is difficult to effectively handle new attack variants such as cross-modal instruction nesting; the feature matching technology refers to identifying anomalies by dynamically analyzing the internal state of the model. Although this method has high accuracy, it requires manual annotation of a large number of samples and has high computational overhead, making it difficult to adapt to the low-latency and high-concurrency business requirements of the power system. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides a defense method, system, device and storage medium for large language model jailbreak attacks, achieving the technical effects of improving the computational efficiency, interpretability and real-time dynamic defense ability of large language models through a collaborative mechanism of feature propagation trajectory analysis, lightweight perturbation intervention and selective parameter fine-tuning.

[0005] In a first aspect, the present invention provides a defense method for large language model jailbreak attacks, the method comprising: Inputting text data into a large language model, collecting high-dimensional intermediate features output by each decoder layer of the large language model through a pre-constructed probe collection model and performing dimensionality reduction to obtain a plurality of low-dimensional intermediate features, the probe collection model comprising a plurality of feature extraction modules, and the large language model being constructed based on a transformer model; Calculate the first distance between the low-dimensional intermediate feature and the preset harmful feature center, and the second distance between the low-dimensional intermediate feature and the preset safe feature center. Obtain the vector distance difference according to the difference between the first distance and the second distance, and determine whether there is a risk of jailbreak attack according to the comparison result between the vector distance difference and the preset difference threshold; In response to the existence of a jailbreak attack risk, take the last several decoder layers in the large language model as key layers, construct a perturbation optimization model with the maximization of the vector distance difference of the key layers as the objective function, perform iterative solution on the perturbation optimization model to obtain the optimal perturbation term, and add the optimal perturbation term to the corresponding high-dimensional intermediate feature to obtain a safe intermediate feature.

[0006] Further, the step of collecting the high-dimensional intermediate features output by each decoder layer of the large language model through a pre-constructed probe acquisition model and performing dimensionality reduction to obtain a plurality of low-dimensional intermediate features includes: Embed each feature extraction module into the output position of each decoder layer of the large language model. The feature extraction module includes a linear transformation module and a compression and dimensionality reduction module, and the decoder layer includes a multi-head attention module and a feed-forward neural network module; Collect the high-dimensional intermediate features output by each decoder layer through the linear transformation module, and map the high-dimensional intermediate features to a low-dimensional feature space to obtain the first low-dimensional intermediate feature; Perform feature compression on the first low-dimensional intermediate feature through the compression and dimensionality reduction module to obtain the low-dimensional intermediate features corresponding to each decoder layer.

[0007] Further, the step of constructing a perturbation optimization model with the maximization of the vector distance difference of the key layers as the objective function, performing iterative solution on the perturbation optimization model to obtain the optimal perturbation term includes: Construct a perturbation optimization model with the maximization of the vector distance difference of the key layers as the objective function, the perturbation term as the optimization variable, and the perturbation amplitude less than or equal to the perturbation amplitude threshold as the constraint condition; Take the zero vector as the initial value of the perturbation term, perform iterative solution on the perturbation optimization model to obtain the optimal perturbation term; Among them, the objective function is expressed by the following formula: In the formula, D1 represents the first distance, D2 represents the second distance, represents the maximum value parameter function, represents the perturbation amplitude threshold, represents the perturbation term.

[0008] Further, the calculation steps of the perturbation amplitude threshold include: Calculate the reference distance between the harmful feature center and the safety feature center corresponding to the key layer, and determine whether the reference distance is within the distance threshold range. If so, according to the initial range of the perturbation amplitude, use the linear interpolation method to calculate the first perturbation amplitude, and use the first perturbation amplitude as the perturbation amplitude threshold; Otherwise, compare the reference distance with the upper limit value and the lower limit value of the distance threshold range respectively; In response to the reference distance being greater than the upper limit value of the distance threshold range, according to the initial range of the perturbation amplitude and the reference distance, use the perturbation reduction strategy to calculate the second perturbation amplitude, and use the second perturbation amplitude as the perturbation amplitude threshold; In response to the reference distance being less than the lower limit value of the distance threshold range, then according to the initial range of the perturbation amplitude and the reference distance, use the perturbation amplification strategy to calculate the third perturbation amplitude, and use the third perturbation amplitude as the perturbation amplitude threshold.

[0009] Further, the first perturbation amplitude is expressed by the following formula: In the formula, represents the first perturbation amplitude, represents the lower limit value of the initial range of the perturbation amplitude, represents the upper limit value of the initial range of the perturbation amplitude, represents the upper limit value of the distance threshold range, represents the lower limit value of the distance threshold range, and d represents the reference distance; The second perturbation amplitude is expressed by the following formula: In the formula, represents the second perturbation amplitude, and γ represents the attenuation coefficient; The third perturbation amplitude is expressed by the following formula: In the formula, represents the third perturbation amplitude.

[0010] Further, after the step of obtaining the safe intermediate feature, it further includes: Construct a loss function according to the difference between the high-dimensional intermediate feature and the safe intermediate feature of the key layer and the adjustment amplitude of the model parameters of the key layer; Fine-tune the model parameters of the key layer according to the loss function.

[0011] Further, the loss function is expressed by the following formula: Wherein, θ represents the model parameters of the adjusted critical layer, θ0 represents the model parameters of the critical layer before adjustment, and β represents the regularization coefficient. represents the secure intermediate feature of the t-th critical layer. represents the high-dimensional intermediate feature of the (t-1)-th critical layer. represents the forward mapping function under the action of the model parameters θ.

[0012] In a second aspect, the present invention provides a defense system against large language model jailbreak attacks, and the system includes: A feature acquisition module, configured to input text data into a large language model, and collect and reduce the high-dimensional intermediate features output by each decoder layer of the large language model through a pre-constructed probe acquisition model to obtain a plurality of low-dimensional intermediate features. The probe acquisition model includes several feature extraction modules, and the large language model is constructed based on a transformer model; An attack determination module, configured to calculate a first distance between the low-dimensional intermediate feature and a preset harmful feature center, and a second distance between the low-dimensional intermediate feature and a preset secure feature center, obtain a vector distance difference according to a difference between the first distance and the second distance, and determine whether there is a risk of jailbreak attack according to a comparison result between the vector distance difference and a preset difference threshold; An active defense module, configured to respond to the existence of a risk of jailbreak attack, use the last several decoder layers in the large language model as critical layers, construct a perturbation optimization model with the maximization of the vector distance difference of the critical layers as an objective function, iteratively solve the perturbation optimization model to obtain an optimal perturbation term, and add the optimal perturbation term to the corresponding high-dimensional intermediate feature to obtain a secure intermediate feature.

[0013] In a third aspect, an embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0014] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0015] The present invention provides a defense method, system, device, and storage medium for large language model jailbreak attacks. By using probes to collect the hidden state features of each layer of the large language model in real time and combining with the low-dimensional feature propagation trajectory, the present invention can dynamically locate harmful semantic evolution paths during the forward inference process of the model; by quantifying risk assessment through the feature center distance, the interpretability of model defense can be improved; through an active defense mechanism based on controllable perturbations, the real-time dynamic defense ability of the large language model can be enhanced; through feature dimensionality reduction and local parameter fine-tuning mechanisms, the computational efficiency is effectively improved, and while blocking attacks, the original knowledge base of the model is maximally retained to avoid the problem of catastrophic forgetting. Through the collaborative mechanism of feature propagation trajectory analysis, lightweight perturbation intervention, and selective parameter fine-tuning, the present invention improves the real-time dynamic defense ability and interpretability of large language models in power systems, effectively reduces computational resource consumption, and achieves an effective balance between the adversarial adjustment and stability of large language models. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic flowchart of a defense method for large language model jailbreak attacks in an embodiment of the present invention; Figure 2 is a schematic structural diagram of a defense system for large language model jailbreak attacks in an embodiment of the present invention; Figure 3 is an internal structural diagram of a computer device in an embodiment of the present invention; Reference Signs: 10. Feature Acquisition Module; 20. Attack Determination Module; 30. Active Defense Module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0018] Please refer to Figure 1 , a defense method for large language model jailbreak attacks proposed in the first embodiment of the present invention, including steps S10 to S30: Step S10, input text data into the large language model, and use a pre-constructed probe collection model to collect high-dimensional intermediate features output by each decoder layer of the large language model and perform dimensionality reduction to obtain a plurality of low-dimensional intermediate features. The probe collection model includes several feature extraction modules, and the large language model is constructed based on a transformer model; Step S20, calculate the first distance between the low-dimensional intermediate feature and the preset harmful feature center, and the second distance between the low-dimensional intermediate feature and the preset safe feature center. Obtain the vector distance difference according to the difference between the first distance and the second distance, and determine whether there is a risk of jailbreak attack according to the comparison result between the vector distance difference and the preset difference threshold; Step S30, in response to the existence of a jailbreak attack risk, take the last several decoder layers in the large language model as key layers, construct a perturbation optimization model with the maximization of the vector distance difference of the key layers as the objective function, perform iterative solution on the perturbation optimization model to obtain the optimal perturbation term, and add the optimal perturbation term to the corresponding high-dimensional intermediate feature to obtain the safe intermediate feature.

[0019] The present invention provides a defense method against jailbreak attacks on large language models in the application scenario of power systems. Among them, the large language model is constructed based on the Transformer model. The large language model can be deployed in a power grid intelligent question answering assistant and an automatic dispatching instruction generation model to parse the natural language instructions of power grid operators and generate corresponding operation suggestions or dispatching operation instructions. The conventional Transformer model is an encoder-decoder architecture. Considering model performance, the large language model usually adopts a pure decoder architecture to generate text autoregressively, and its hidden states are passed layer by layer to support real-time content generation. Therefore, the large language model in this embodiment also adopts a conventional pure decoder architecture, including an input embedding layer and multiple decoder layers. Each decoder layer includes a multi-head attention module and a feed-forward neural network module. Among them, the input embedding layer is used to convert the input text into a high-dimensional vector representation and incorporate positional encoding to retain sequence order information; the multi-head attention module only allows the current position to focus on the previous tokens in the sequence through the self-attention mechanism to avoid information leakage; the feed-forward neural network module is used to perform feature transformation through a fully connected layer and a non-linear activation function (such as GELU); in addition, the encoder layer also includes residual connection and layer normalization to improve training stability and alleviate gradient disappearance. The specific structure of the large language model can refer to the conventional model structure based on the decoder architecture, and will not be limited here too much.

[0020] For the above architecture of the large language model, this embodiment constructs a probe collection model composed of multiple feature extraction modules. By deploying the probe collection model in the large language model, the acquisition and processing of the high-dimensional hidden states, that is, high-dimensional intermediate features, output by each layer of the large language model are realized. The specific steps include: Embed each feature extraction module into the output positions of each decoder layer of the large language model. The feature extraction module includes a linear transformation module and a compression and dimensionality reduction module, and the decoder layer includes a multi-head attention module and a feed-forward neural network module; The high-dimensional intermediate features output by each decoder layer are collected through the linear transformation module, and the high-dimensional intermediate features are mapped to a low-dimensional feature space to obtain the first low-dimensional intermediate features; The first low-dimensional intermediate features are subjected to feature compression through the compression and dimensionality reduction module to obtain the low-dimensional intermediate features corresponding to each decoder layer.

[0021] In this embodiment, the probe acquisition model is composed of multiple lightweight feature extraction modules. Each feature extraction module includes a linear transformation module and a compression and dimensionality reduction module. To accurately identify harmful features inside the large language model and model the propagation path, in this embodiment, each feature extraction module is respectively embedded at the output positions of each decoder layer of the large language model to collect the high-dimensional intermediate features output by each layer of the large language model. Since each encoder layer includes a multi-head attention module and a feed-forward neural network module, the feature extraction module is actually deployed at the output positions of the multi-head attention module and the feed-forward neural network module of each encoder layer. The high-dimensional intermediate features output by each decoding layer include the high-dimensional intermediate features output by the multi-head attention module and the high-dimensional intermediate features output by the feed-forward neural network module.

[0022] After the probe acquisition model is embedded, the high-dimensional intermediate features output at each output position are collected through the linear transformation module, and parametric mapping is performed through the fully connected layer of the linear transformation module to map the high-dimensional intermediate features to a low-dimensional feature space for preliminary compression. For example, the dimension is reduced from 1024 to 32, so as to obtain the first low-dimensional intermediate features. Although the linear transformation module can reduce the dimension of high-dimensional data to reduce the subsequent calculation amount, redundant information may still be retained. Therefore, the first low-dimensional intermediate features will continue to be input into the compression and dimensionality reduction module, and principal component analysis or linear projection is performed in the compression and dimensionality reduction module to further compress the first low-dimensional intermediate features and eliminate redundant information, obtaining lower-dimensional intermediate features, that is, obtaining low-dimensional intermediate features. For example, the dimension is compressed from 32 to 2D or 3D.

[0023] In this embodiment, the probe acquisition model is tightly coupled with the structure of the large language model and is embedded at the output end of the network backbone according to hierarchical distribution, thereby constructing a hierarchical feature acquisition system. All the extracted low-dimensional intermediate features will be organized and stored in the form of a time series according to the forward propagation order, and a complete feature propagation trajectory is constructed through vertical aggregation, so as to show the semantic evolution path of the input text among the layers inside the large language model.

[0024] To measure the risk level of each low-dimensional intermediate feature, this embodiment introduces a risk assessment index based on vector distance. Specifically, for the low-dimensional intermediate features output by each layer, through the clustering analysis of jailbreak attack data and security data in the sample database, the harmful feature center and security feature center corresponding to the features of this layer are pre-constructed. Then, the first distance between the low-dimensional intermediate feature and the harmful feature center and the second distance between the low-dimensional intermediate feature and the security feature center are calculated. The distance value here can be calculated using Euclidean distance, cosine distance or other metric algorithms, and no excessive limitation is made here.

[0025] Then subtract the second distance D2 from the first distance D1 to obtain the vector distance difference D, that is, D = D1 - D2, and use the vector distance difference D as the risk assessment index. Through the risk assessment index, it can be determined which type of feature the low-dimensional intermediate feature is closer to, so as to achieve the accurate capture of harmful features. To better characterize the difference between the distances of the low-dimensional intermediate feature to the two feature centers, in this embodiment, first normalize the first distance and the second distance and scale them to the range [0, 1], and then calculate the vector distance difference. According to the distance metric, if the low-dimensional intermediate feature belongs to the security feature, then the low-dimensional intermediate feature should be far from the harmful feature center and close to the security feature center. Therefore, D1 should be greater than D2, that is, D should be greater than zero. Therefore, in this embodiment, the difference threshold is set to zero. If D is greater than zero, it is considered that the low-dimensional intermediate feature is a security feature. On the contrary, if D is less than zero, it means that the low-dimensional intermediate feature is closer to the harmful feature center. At this time, it is considered that the low-dimensional intermediate feature is a harmful feature, and there is a jailbreak attack risk in the large language model. And the smaller the D value, the higher the jailbreak attack risk.

[0026] This embodiment performs feature trajectory modeling through a detection and acquisition model, and quantitatively evaluates the risk of the feature trajectory, thereby providing accurate data support for subsequent defense mechanisms.

[0027] When it is determined through the risk assessment index that there is a jailbreak attack risk in the large language model, this embodiment provides an interpretable feature defense mechanism to actively intervene in the intermediate features of the large language model. A perturbation optimization model is constructed through the vector distance difference, the optimal perturbation term is calculated, and it is superimposed on the corresponding high-dimensional intermediate feature, so as to achieve active intervention in harmful intermediate features. The specific steps include: Construct a perturbation optimization model with the maximization of the vector distance difference of the key layer as the objective function, the perturbation term as the optimization variable, and the perturbation amplitude less than or equal to the perturbation amplitude threshold as the constraint condition; Use the zero vector as the initial value of the perturbation term, and iteratively solve the perturbation optimization model to obtain the optimal perturbation term.

[0028] In this embodiment, key layers in the large language model are first selected. The key layers refer to the network layers that play a dominant role in the model semantic synthesis process and are highly sensitive to the understanding of the input semantics and the generation of the output. Taking the large language model with a pure decoder architecture as an example, for the large language model with a multi-decoder layer structure, the last few decoder layers in the model structure will have a strong impact on the finally generated content. Therefore, the last few layers are defined as key layers. Assuming that the large language model is composed of 24 decoders with the same structure stacked, that is, the model has 24 layers, and the hidden state is passed through forward propagation between layers, that is, the intermediate features are passed, then the 21st to 24th layers can be defined as key layers. The low-dimensional intermediate features of these key layers include the low-dimensional intermediate features of the multi-head attention module and the low-dimensional intermediate features of the feed-forward neural network module.

[0029] In this embodiment, by introducing controllable perturbations at these key positions, the representation space of the large language model can be adjusted at a small computational cost, so that it moves away from the harmful semantic representation and approaches the safe representation. Among them, the perturbation term can be obtained by optimizing the perturbation model calculation.

[0030] In this embodiment, with the maximization of the vector distance difference corresponding to the low-dimensional intermediate features of the key layer as the objective function, the perturbation term as the optimization variable, and the constraint that the perturbation amplitude is less than or equal to the preset perturbation amplitude threshold, a perturbation optimization model is constructed: In the formula, D1 represents the first distance, and D2 represents the second distance. represents the maximum value parameter function. represents the perturbation amplitude threshold. represents the perturbation term.

[0031] According to the harmful feature capture process, the closer the low-dimensional intermediate feature is to the harmful feature center, the smaller D1 is, and the corresponding D2 is larger, and the more harmful the low-dimensional intermediate feature is. On the contrary, when D1 is larger and D2 is smaller, the safer the low-dimensional intermediate feature is. Therefore, in this embodiment, with the maximization of the vector distance difference of the key layer as the objective function, and through iterative solution to find the optimal perturbation term, when the optimal perturbation term is superimposed on the high-dimensional intermediate feature output by this key layer, the output feature of this layer of the large language model can be farthest from the harmful feature center and closest to the safe feature center to the greatest extent.

[0032] When solving the model, the perturbation term is initialized as a zero vector and iterative solution is carried out. Taking the k-th iteration calculation as an example, the gradient of the objective function with respect to the perturbation term is calculated, and the perturbation direction is updated according to the gradient and the learning rate: In the formula, α is the preset learning rate. is the perturbation term gradient of the k-th iteration. is the perturbation term for the (k + 1)-th iteration.

[0033] For the perturbation term of the (k + 1)-th iteration, determine whether the projection constraint is satisfied. If not: Then project it back to the constraint range: Stop when the objective function converges or reaches the maximum number of iterations, and output the optimal perturbation term.

[0034] Finally, add the obtained optimal perturbation term to the high-dimensional intermediate feature output by this key layer to obtain the secure intermediate feature, thereby adjusting the intermediate representation of the large language model: In the formula, is the secure intermediate feature of the t-th key layer, is the high-dimensional intermediate feature of the t-th key layer, is the optimal perturbation term.

[0035] This embodiment intuitively reflects the perturbation direction and intensity through the feature center distance metric, has strong interpretability, and only operates on a small number of nodes in the key layer. Therefore, the overall computational complexity is extremely small, improving the real-time dynamic defense ability of the large language model. Moreover, the active defense mechanism provided in this embodiment is applicable to various large language models with the Transformer architecture and has good compatibility.

[0036] In a preferred embodiment, to ensure that the features of the large language model are not overly changed by perturbations and affect the normal output, the present invention adopts an adaptive perturbation constraint strategy to adjust the perturbation amplitude threshold for different semantic difference metrics. The specific steps include: Calculate the reference distance between the harmful feature center and the secure feature center corresponding to the key layer, and determine whether the reference distance is within the distance threshold range. If so, calculate the first perturbation amplitude using linear interpolation according to the initial range of the perturbation amplitude, and use the first perturbation amplitude as the perturbation amplitude threshold; Conversely, compare the reference distance with the upper limit value and the lower limit value of the distance threshold range respectively; In response to the reference distance being greater than the upper limit value of the distance threshold range, calculate the second perturbation amplitude using a perturbation reduction strategy according to the initial range of the perturbation amplitude and the reference distance, and use the second perturbation amplitude as the perturbation amplitude threshold; In response to the reference distance being less than the lower limit value of the distance threshold range, according to the initial range of the perturbation amplitude and the reference distance, a perturbation amplification strategy is adopted to calculate the third perturbation amplitude, and the third perturbation amplitude is used as the perturbation amplitude threshold.

[0037] In this embodiment, for a certain critical layer, first calculate the reference distance between the harmful feature center and the safety feature center of this critical layer, and then set the distance threshold range of this reference distance. This range can be a preset value or determined by statistical distribution analysis or the quantile threshold method. When using statistical distribution analysis, calculate the mean and standard deviation of the reference distances of all critical layers, and assume that the reference distance follows a normal distribution, then set the distance threshold range according to the mean and standard deviation; when using the quantile threshold method, calculate the interquartile range of this reference distance, and set the distance threshold range according to the upper quartile, lower quartile and interquartile range. Specifically, the setting steps of the conventional threshold setting method based on normal distribution and the quantile threshold method can be referred to, and will not be elaborated here one by one.

[0038] Then determine whether the reference distance is within the distance threshold range. If so, dynamically calculate the first perturbation amplitude by linear interpolation, and use the first perturbation amplitude as the perturbation amplitude threshold in the objective function, where the first perturbation amplitude can be expressed as: In the formula, represents the first perturbation amplitude, represents the lower limit value of the initial range of the perturbation amplitude, represents the upper limit value of the initial range of the perturbation amplitude, represents the upper limit value of the distance threshold range, represents the lower limit value of the distance threshold range, d represents the reference distance; Among them, and are preset values, and are the upper limit value and lower limit value of the distance threshold range obtained by statistically analyzing the reference distances of all critical layers.

[0039] If the reference distance is not within the normal range, then according to the comparison relationship between the reference distance and the upper limit value and lower limit value of the distance threshold range, determine whether the reference distance is too large or too small. If the reference distance is greater than the upper limit value of the distance threshold range, it means that the feature boundary is clear. At this time, it is necessary to reduce the perturbation amplitude threshold and suppress the perturbation intensity to avoid semantic damage. At this time, the following formula is used to calculate the second perturbation amplitude and use it as the perturbation amplitude threshold: In the formula, represents the second perturbation amplitude, and γ represents a preset attenuation coefficient; If the reference distance is less than the lower limit value of the distance threshold range, it indicates that the feature boundary is blurred. At this time, it is necessary to increase the perturbation amplitude threshold and enhance the perturbation intensity to improve the defense ability. At this time, the following formula is used to calculate the third perturbation amplitude and use it as the perturbation amplitude threshold: In the formula, represents the third perturbation amplitude.

[0040] In a preferred embodiment, in order to further improve the defense ability of the large language model, the present invention also provides a feature fine-tuning mechanism. On the premise of maintaining the overall knowledge structure of the model, only the key parameters strongly related to harmful features are fine-tuned to improve the defense ability of the model. The specific steps include: Construct a loss function according to the difference between the high-dimensional intermediate feature and the safe intermediate feature of the key layer and the adjustment amplitude of the model parameters of the key layer; Fine-tune the model parameters of the key layer according to the loss function.

[0041] In this embodiment, the core idea of the feature fine-tuning mechanism is to guide the internal representation of the model to evolve in a safer direction by minimizing the difference between the model output and the "safe intervention target", thereby effectively blocking the propagation path of harmful features. The safe intervention target here refers to the safe intermediate feature generated by the above-mentioned active defense mechanism. Therefore, in this embodiment, a loss function is constructed according to the difference between the high-dimensional intermediate feature and the safe intermediate feature of the key layer and the adjustment amplitude of the model parameters of the key layer, and the model parameters are fine-tuned by minimizing the loss function. Among them, the loss function can be expressed as: In the formula, θ represents the model parameters of the adjusted key layer, θ0 represents the model parameters of the key layer before adjustment, β represents the regularization coefficient, represents the safe intermediate feature of the t-th key layer, represents the high-dimensional intermediate feature of the (t - 1)-th key layer, represents the forward mapping function under the action of the model parameters θ.

[0042] In the loss function, the forward mapping function is used to calculate the high-dimensional intermediate feature The output after the action of model parameter θ, the secure intermediate feature is the secure intermediate feature adjusted by the active defense mechanism, which is used to guide the model output to align towards a more secure representation. The regularization coefficient is used to control the offset amplitude between the current parameter and the initial parameter, and its value is generally set within a small range to ensure that while enhancing security, the original knowledge structure of the model is retained to the greatest extent, thereby achieving an effective balance between adversarial adjustment and model stability.

[0043] Through the above loss function, the model parameters of the key layers of the large language model are fine-tuned. During model training, a lightweight optimizer is preferably used, and a low learning rate is set to avoid large fluctuations in parameters. Through the secure fine-tuning mechanism of the constraint loss function, the model realizes dynamic blocking of harmful features while retaining its original capabilities, effectively taking into account the defense real-time performance, semantic coherence, and engineering feasibility, thereby providing an interpretable technical path for the security reinforcement of the large language model in the power system.

[0044] A defense method against large language model jailbreak attacks provided in this embodiment can dynamically locate the harmful semantic evolution path during the forward inference process of the model by collecting the hidden state features of each layer of the large language model in real time through a probe and combining the low-dimensional feature propagation trajectory; the risk assessment is quantified through the feature center distance, improving the interpretability of model defense; the real-time dynamic defense ability of the large language model is effectively improved through the active defense mechanism based on controllable perturbation; the consumption of computing resources is effectively reduced and the computing efficiency is improved through the feature dimension reduction and local parameter fine-tuning mechanism, and the original knowledge base of the model is retained to the greatest extent while blocking the attack, avoiding the problem of catastrophic forgetting. Through the collaborative mechanism of feature propagation trajectory analysis, lightweight perturbation intervention, and selective parameter fine-tuning, the present invention realizes the integration of real-time precise defense, interpretable decision-making, and low resource consumption, providing a solution that takes into account both efficiency and practicality for the secure deployment of large language models in the power system.

[0045] Please refer to Figure 2 , based on the same inventive concept, a defense system against large language model jailbreak attacks proposed in the second embodiment of the present invention includes: A feature collection module 10, configured to input text data into the large language model, collect and reduce the dimensionality of the high-dimensional intermediate features output by each decoder layer of the large language model through a pre-constructed probe collection model to obtain a plurality of low-dimensional intermediate features. The probe collection model includes a plurality of feature extraction modules, and the large language model is constructed based on a transformer model; An attack determination module 20 is configured to calculate a first distance between the low-dimensional intermediate feature and a preset harmful feature center, and a second distance between the low-dimensional intermediate feature and a preset safe feature center, obtain a vector distance difference according to a difference between the first distance and the second distance, and determine whether there is a jailbreak attack risk according to a comparison result between the vector distance difference and a preset difference threshold; An active defense module 30 is configured to, in response to the existence of a jailbreak attack risk, use the last several decoder layers in the large language model as key layers, construct a perturbation optimization model with the maximization of the vector distance difference of the key layers as an objective function, iteratively solve the perturbation optimization model to obtain an optimal perturbation term, and add the optimal perturbation term to the corresponding high-dimensional intermediate feature to obtain a safe intermediate feature.

[0046] The technical features and technical effects of the defense system against large language model jailbreak attacks proposed in the embodiments of the present invention are the same as those of the method proposed in the embodiments of the present invention, and will not be elaborated here. Each module in the above defense system against large language model jailbreak attacks can be implemented in whole or in part by software, hardware, and their combinations. The above-mentioned modules can be embedded in the processor of the computer device in hardware form or be independent of it, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0047] In addition, an embodiment of the present invention also proposes a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0048] Please refer to Figure 3 , the internal structure diagram of a computer device in an embodiment. The computer device may specifically be a terminal or a server. The computer device includes a processor, a memory, a network interface, a display, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. The computer program, when executed by the processor, implements the defense method against large language model jailbreak attacks. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device may be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, a touchpad, or a mouse, etc.

[0049] Those of ordinary skill in the art can understand that Figure 3 The structure shown in Figure 3 is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computing device may include more or fewer components than those shown in the figure, or combine some components, or have the same component arrangement.

[0050] In addition, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0051] In summary, a defense method, system, device, and storage medium for large language model jailbreak attacks proposed by an embodiment of the present invention. The method inputs text data into a large language model, and uses a pre-constructed probe acquisition model to collect high-dimensional intermediate features output by each decoder layer of the large language model and perform dimensionality reduction to obtain multiple low-dimensional intermediate features. The probe acquisition model includes several feature extraction modules, and the large language model is constructed based on a transformer model; calculate a first distance between the low-dimensional intermediate feature and a preset harmful feature center, and a second distance between the low-dimensional intermediate feature and a preset safe feature center, and obtain a vector distance difference according to the difference between the first distance and the second distance, and judge whether there is a risk of jailbreak attack according to the comparison result between the vector distance difference and a preset difference threshold; in response to the existence of a jailbreak attack risk, take the last several decoder layers in the large language model as key layers, construct a perturbation optimization model with the maximization of the vector distance difference of the key layers as the objective function, perform iterative solution on the perturbation optimization model to obtain an optimal perturbation term, and add the optimal perturbation term to the corresponding high-dimensional intermediate feature to obtain a safe intermediate feature. The present invention improves the real-time dynamic defense ability and interpretability of large language models in power systems through a collaborative mechanism of feature propagation trajectory analysis, lightweight perturbation intervention, and selective parameter fine-tuning, effectively reduces the consumption of computing resources, and achieves an effective balance between the adversarial adjustment and stability of the model.

[0052] Each embodiment in this specification is described in a progressive manner. For parts that are the same or similar in each embodiment, they can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiment. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not conflict, it should be considered as the scope described in this specification.

[0053] The above-described embodiments merely represent several preferred embodiments of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the technical principle of the present invention, several improvements and substitutions can be made, and these improvements and substitutions should also be regarded as the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the protection scope of the claims described above.

Claims

1. A defense method against large language model jailbreak attacks, characterized in that, Including: Input text data into a large language model. Use a pre - constructed probe collection model to collect high - dimensional intermediate features output by each decoder layer of the large language model and perform dimensionality reduction to obtain multiple low - dimensional intermediate features. The probe collection model includes several feature extraction modules, and the large language model is constructed based on a transformer model; Calculate the first distance between the low - dimensional intermediate features and a preset harmful feature center, and the second distance between the low - dimensional intermediate features and a preset safe feature center. Obtain a vector distance difference based on the difference between the first distance and the second distance, and determine whether there is a risk of jailbreak attack according to the comparison result between the vector distance difference and a preset difference threshold; In response to the existence of a jailbreak attack risk, take the last several decoder layers in the large language model as key layers. Construct a perturbation optimization model with the maximization of the vector distance difference of the key layers as the objective function, perform iterative solution on the perturbation optimization model to obtain an optimal perturbation term, and add the optimal perturbation term to the corresponding high - dimensional intermediate features to obtain safe intermediate features.

2. The defense method against large language model jailbreak attacks according to claim 1, characterized in that The step of using a pre - constructed probe collection model to collect high - dimensional intermediate features output by each decoder layer of the large language model and perform dimensionality reduction to obtain multiple low - dimensional intermediate features includes: Embed each feature extraction module into the output position of each decoder layer of the large language model. The feature extraction module includes a linear transformation module and a compression and dimensionality reduction module, and the decoder layer includes a multi - head attention module and a feed - forward neural network module; Collect high - dimensional intermediate features output by each decoder layer through the linear transformation module and map the high - dimensional intermediate features to a low - dimensional feature space to obtain first low - dimensional intermediate features; Perform feature compression on the first low - dimensional intermediate features through the compression and dimensionality reduction module to obtain low - dimensional intermediate features corresponding to each decoder layer.

3. The defense method against large language model jailbreak attacks according to claim 1, wherein The step of constructing a perturbation optimization model with the maximization of the vector distance difference of the key layers as the objective function, performing iterative solution on the perturbation optimization model to obtain an optimal perturbation term includes: Construct a perturbation optimization model with the maximization of the vector distance difference of the key layers as the objective function, the perturbation term as the optimization variable, and the constraint that the perturbation amplitude is less than or equal to the perturbation amplitude threshold; Take the zero vector as the initial value of the perturbation term and perform iterative solution on the perturbation optimization model to obtain an optimal perturbation term; Among them, the objective function is represented by the following formula: Wherein, D1 represents the first distance, and D2 represents the second distance. represents the maximum value parameter function, represents the perturbation amplitude threshold, represents the perturbation term.

4. The defense method against large language model jailbreak attacks according to claim 3, characterized in that The calculation steps of the perturbation amplitude threshold include: Calculate the reference distance between the harmful feature center and the safe feature center corresponding to the key layer, and determine whether the reference distance is within the distance threshold range. If so, calculate the first perturbation amplitude using linear interpolation according to the initial range of the perturbation amplitude, and take the first perturbation amplitude as the perturbation amplitude threshold; Otherwise, compare the reference distance with the upper limit value and the lower limit value of the distance threshold range respectively; In response to the reference distance being greater than the upper limit value of the distance threshold range, according to the initial range of the perturbation amplitude and the reference distance, a perturbation reduction strategy is adopted to calculate a second perturbation amplitude, and the second perturbation amplitude is used as the perturbation amplitude threshold; In response to the reference distance being less than the lower limit value of the distance threshold range, according to the initial range of the perturbation amplitude and the reference distance, a perturbation amplification strategy is adopted to calculate a third perturbation amplitude, and the third perturbation amplitude is used as the perturbation amplitude threshold.

5. The defense method against large language model jailbreak attacks according to claim 4, wherein, The first perturbation amplitude is represented by the following formula: In the formula, represents the first perturbation amplitude, represents the lower limit value of the initial range of the perturbation amplitude, represents the upper limit value of the initial range of the perturbation amplitude, represents the upper limit value of the distance threshold range, represents the lower limit value of the distance threshold range, and d represents the reference distance; The second perturbation amplitude is represented by the following formula: In the formula, represents the second perturbation amplitude, and γ represents the attenuation coefficient; The third perturbation amplitude is represented by the following formula: In the formula, represents the third perturbation amplitude.

6. The defense method against large language model jailbreak attacks according to claim 1, wherein After the step of obtaining the secure intermediate feature, it further includes: Construct a loss function according to the difference between the high-dimensional intermediate feature of the key layer and the secure intermediate feature and the adjustment amplitude of the model parameters of the key layer; Fine-tune the model parameters of the key layer according to the loss function.

7. The defense method against large language model jailbreak attacks according to claim 6, wherein The loss function is represented by the following formula: Where, θ represents the model parameters of the adjusted key layer, θ0 represents the model parameters of the key layer before adjustment, and β represents the regularization coefficient. represents the secure intermediate feature of the t-th key layer. represents the high-dimensional intermediate feature of the (t - 1)-th key layer. represents the forward mapping function under the action of the model parameters θ.

8. A defense system against large language model jailbreak attacks, characterized in that, It includes: A feature acquisition module, configured to input text data into a large language model, and collect and reduce the high-dimensional intermediate features output by each decoder layer of the large language model through a pre-constructed probe acquisition model to obtain a plurality of low-dimensional intermediate features. The probe acquisition model includes several feature extraction modules, and the large language model is constructed based on a transformer model; An attack determination module, configured to calculate a first distance between the low-dimensional intermediate feature and a preset harmful feature center, and a second distance between the low-dimensional intermediate feature and a preset secure feature center, obtain a vector distance difference according to the difference between the first distance and the second distance, and determine whether there is a risk of jailbreak attack according to the comparison result between the vector distance difference and a preset difference threshold; An active defense module, configured to respond to the existence of a jailbreak attack risk, take the last several decoder layers in the large language model as key layers, construct a perturbation optimization model with the maximization of the vector distance difference of the key layers as the objective function, iteratively solve the perturbation optimization model to obtain an optimal perturbation term, and add the optimal perturbation term to the corresponding high-dimensional intermediate feature to obtain a secure intermediate feature.

9. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images