Defense method, system, device and storage medium for large language model jailbreak attack

Abstract

The present invention relates to the field of model defense technology, and discloses a defense method, system, device, and storage medium for jailbreak attacks on a large language model, including collecting high-dimensional intermediate features output by each decoder layer through a probe acquisition model and performing dimensionality reduction to obtain low-dimensional intermediate features; calculating a first distance between the low-dimensional intermediate features and the center of harmful features, and a second distance between the low-dimensional intermediate features and the center of safe features, and obtaining a vector distance difference based on the difference between the first distance and the second distance to determine whether there is a risk of jailbreak attack; in response to the existence of a risk, constructing a disturbance optimization model with the maximization of the vector distance difference of the key layer as the objective function, obtaining an optimal disturbance term, and adding it to the corresponding high-dimensional intermediate feature to obtain a safe intermediate feature. The present invention improves the real-time dynamic defense capability, interpretability, and computational efficiency of large language models in power systems, and achieves an effective balance between adversarial adjustment and stability of large language models.

Description

Technical Field

[0001] The present invention relates to the field of model defense technology, and in particular to a defense method, system, device and storage medium for jailbreak attacks on a large language model. Background Art

[0002] With the deep integration of large language models in the intelligent development of power systems, their application in key scenarios such as generating grid dispatch instructions, answering equipment operation and maintenance questions, and writing fault diagnosis reports is becoming increasingly widespread. However, in open interactive environments, these models face a severe risk of jailbreak attacks. Attackers can construct hidden instructions through semantic disguise and contextual misdirection, tricking the model into generating erroneous outputs that violate power safety regulations. Such attacks can lead to systemic risks such as incorrect dispatch instructions and inaccurate operation and maintenance responses, seriously threatening the stable operation of the power grid and the safety of personnel.

[0003] At present, the defense technologies of large language models are mainly divided into four categories, namely static input detection, output post-processing, heuristic rule matching and feature matching. Among them, static input detection refers to filtering sensitive inputs based on keyword libraries or classifiers, but this method is difficult to resist semantic reorganization attacks; output post-processing refers to post-filtering of generated content, but this method has significant defense lag and cannot prevent real-time instructions from being erroneously sent to power execution terminals; heuristic rule matching refers to relying on manual rule libraries to intercept abnormal instructions. This method of relying on rule libraries is prone to insufficient generalization capabilities and is difficult to effectively respond to new attack variants such as cross-modal instruction nesting; feature matching technology refers to identifying anomalies by dynamically analyzing the internal state of the model. Although this method has high accuracy, it requires manual labeling of massive samples and has high computational overhead, making it difficult to adapt to the power system's business needs for low latency and high concurrency. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a defense method, system, device and storage medium for jailbreak attacks on large language models. Through the collaborative mechanism of feature propagation trajectory analysis, lightweight perturbation intervention and selective parameter fine-tuning, the technical effect of improving the computational efficiency, interpretability and real-time dynamic defense capabilities of large language models is achieved.

[0005] In a first aspect, the present invention provides a method for defending against a large language model jailbreak attack, the method comprising:

[0006] Input text data into a large language model, collect high-dimensional intermediate features output by each decoder layer of the large language model through a pre-built probe acquisition model and perform dimensionality reduction to obtain multiple low-dimensional intermediate features, the probe acquisition model includes several feature extraction modules, and the large language model is constructed based on the transformer model;

[0007] calculating a first distance between the low-dimensional intermediate feature and a preset harmful feature center, and a second distance between the low-dimensional intermediate feature and a preset safe feature center, obtaining a vector distance difference based on a difference between the first distance and the second distance, and determining whether there is a jailbreak attack risk based on a comparison result between the vector distance difference and a preset difference threshold;

[0008] In response to the risk of jailbreak attacks, the last several decoder layers in the large language model are used as key layers, and the maximization of the vector distance difference of the key layers is used as the objective function. A perturbation optimization model is constructed, and the perturbation optimization model is iteratively solved to obtain the optimal perturbation term. The optimal perturbation term is then added to the corresponding high-dimensional intermediate feature to obtain a secure intermediate feature.

[0009] Furthermore, the step of collecting high-dimensional intermediate features output by each decoder layer of the large language model through the pre-built probe collection model and performing dimensionality reduction to obtain multiple low-dimensional intermediate features includes:

[0010] Each feature extraction module is embedded in the output position of each decoder layer of the large language model, wherein the feature extraction module includes a linear transformation module and a compression dimension reduction module, and the decoder layer includes a multi-head attention module and a feedforward neural network module;

[0011] The linear transformation module collects high-dimensional intermediate features output by each decoder layer, and maps the high-dimensional intermediate features to a low-dimensional feature space to obtain a first low-dimensional intermediate feature;

[0012] The first low-dimensional intermediate features are compressed by the compression and dimensionality reduction module to obtain low-dimensional intermediate features corresponding to each decoder layer.

[0013] Furthermore, the steps of constructing a disturbance optimization model with maximizing the vector distance difference of the key layer as the objective function, and iteratively solving the disturbance optimization model to obtain the optimal disturbance term include:

[0014] A disturbance optimization model is constructed by taking maximization of the vector distance difference of the key layer as the objective function, taking the disturbance term as the optimization variable, and taking the disturbance amplitude being less than or equal to the disturbance amplitude threshold as the constraint condition;

[0015] Taking the zero vector as the initial value of the disturbance term, iteratively solving the disturbance optimization model to obtain the optimal disturbance term;

[0016] The objective function is expressed by the following formula:

[0017]

[0018] Where D1 represents the first distance, D2 represents the second distance, represents the maximum parameter function, represents the disturbance amplitude threshold, represents the disturbance term.

[0019] Furthermore, the step of calculating the disturbance amplitude threshold comprises:

[0020] Calculating a reference distance between a harmful feature center and a safe feature center corresponding to the key layer, determining whether the reference distance is within a distance threshold range, and if so, calculating a first disturbance amplitude using a linear interpolation method based on an initial range of the disturbance amplitude, and using the first disturbance amplitude as a disturbance amplitude threshold;

[0021] Otherwise, the reference distance is compared with the upper limit value of the distance threshold range and the lower limit value of the distance threshold range respectively;

[0022] In response to the reference distance being greater than an upper limit of the distance threshold range, a disturbance reduction strategy is adopted according to the initial range of the disturbance amplitude and the reference distance to calculate a second disturbance amplitude, and the second disturbance amplitude is used as the disturbance amplitude threshold;

[0023] In response to the reference distance being less than the lower limit of the distance threshold range, a disturbance amplification strategy is adopted according to the initial range of the disturbance amplitude and the reference distance to calculate a third disturbance amplitude, and the third disturbance amplitude is used as the disturbance amplitude threshold.

[0024] Furthermore, the first disturbance amplitude is expressed by the following formula:

[0025]

[0026] Where, represents the first disturbance amplitude, Indicates the lower limit of the initial range of disturbance amplitude, Indicates the upper limit of the initial range of the disturbance amplitude, Indicates the upper limit of the distance threshold range. represents the lower limit of the distance threshold range, and d represents the reference distance;

[0027] The second disturbance amplitude is expressed by the following formula:

[0028]

[0029] Where, represents the second disturbance amplitude, γ represents the attenuation coefficient;

[0030] The third disturbance amplitude is expressed by the following formula:

[0031]

[0032] Where, represents the third disturbance amplitude.

[0033] Furthermore, after the step of obtaining the security intermediate feature, the method further includes:

[0034] Constructing a loss function based on the difference between the high-dimensional intermediate features and the security intermediate features of the key layer and the adjustment amplitude of the model parameters of the key layer;

[0035] According to the loss function, the model parameters of the key layer are fine-tuned.

[0036] Furthermore, the loss function is expressed by the following formula:

[0037]

[0038] In the formula, θ represents the model parameters of the key layer after adjustment, θ0 represents the model parameters of the key layer before adjustment, β represents the regularization coefficient, represents the secure intermediate feature of the t-th key layer, represents the high-dimensional intermediate features of the t-1th key layer, Represents the forward mapping function under the action of model parameters θ.

[0039] In a second aspect, the present invention provides a defense system for large language model jailbreak attacks, the system comprising:

[0040] A feature acquisition module is used to input text data into the large language model, collect high-dimensional intermediate features output by each decoder layer of the large language model through a pre-built probe acquisition model, and perform dimensionality reduction to obtain multiple low-dimensional intermediate features. The probe acquisition model includes several feature extraction modules. The large language model is constructed based on the transformer model;

[0041] an attack determination module, configured to calculate a first distance between the low-dimensional intermediate feature and a preset harmful feature center, and a second distance between the low-dimensional intermediate feature and a preset safe feature center, obtain a vector distance difference based on the difference between the first distance and the second distance, and determine whether there is a jailbreak attack risk based on a comparison result between the vector distance difference and a preset difference threshold;

[0042] An active defense module is used to respond to the risk of jailbreak attacks by taking the last several decoder layers in the large language model as key layers, maximizing the vector distance difference of the key layers as the objective function, constructing a perturbation optimization model, iteratively solving the perturbation optimization model to obtain the optimal perturbation term, and adding the optimal perturbation term to the corresponding high-dimensional intermediate features to obtain secure intermediate features.

[0043] In a third aspect, an embodiment of the present invention further provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0044] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0045] The present invention provides a defense method, system, device and storage medium for jailbreak attacks on a large language model. The present invention collects hidden state features of each layer of the large language model in real time through a probe acquisition model, and combined with low-dimensional feature propagation trajectories, it can dynamically locate harmful semantic evolution paths during the model's forward reasoning process; through feature center distance quantitative risk assessment, it can improve the interpretability of model defense; through an active defense mechanism based on controllable disturbances, it can improve the real-time dynamic defense capability of the large language model; through feature dimensionality reduction and local parameter fine-tuning mechanisms, it effectively improves computing efficiency, and while blocking attacks, it retains the model's original knowledge base to the maximum extent, avoiding catastrophic forgetting problems. The present invention improves the real-time dynamic defense capability and interpretability of large language models in power systems through the collaborative mechanism of feature propagation trajectory analysis, lightweight disturbance intervention and selective parameter fine-tuning, effectively reduces computing resource consumption, and achieves an effective balance between adversarial adjustment and stability of large language models. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 1 is a flow chart of a method for defending against a large language model jailbreak attack according to an embodiment of the present invention;

[0047] Figure 2 2 is a schematic diagram of the structure of a defense system against jailbreak attacks on a large language model according to an embodiment of the present invention;

[0048] Figure 3 is an internal structural diagram of a computer device according to an embodiment of the present invention;

[0049] Reference numerals:

[0050] 10. Feature collection module; 20. Attack determination module; 30. Active defense module. DETAILED DESCRIPTION

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0052] See also Figure 1 The first embodiment of the present invention proposes a method for defending against a large language model jailbreak attack, comprising steps S10 to S30:

[0053] Step S10: Input text data into a large language model, collect high-dimensional intermediate features output by each decoder layer of the large language model through a pre-built probe acquisition model and perform dimensionality reduction to obtain multiple low-dimensional intermediate features. The probe acquisition model includes several feature extraction modules, and the large language model is constructed based on the converter model.

[0054] Step S20: Calculate a first distance between the low-dimensional intermediate feature and a preset harmful feature center, and a second distance between the low-dimensional intermediate feature and a preset safe feature center; obtain a vector distance difference based on the difference between the first distance and the second distance; and determine whether there is a jailbreak attack risk based on a comparison result between the vector distance difference and a preset difference threshold;

[0055] In step S30, in response to the risk of jailbreak attacks, the last several decoder layers in the large language model are used as key layers, and the maximization of the vector distance difference of the key layers is used as the objective function. A perturbation optimization model is constructed, and the perturbation optimization model is iteratively solved to obtain the optimal perturbation term. The optimal perturbation term is added to the corresponding high-dimensional intermediate feature to obtain a secure intermediate feature.

[0056] The present invention provides a defense method for jailbreak attacks on large language models in power system application scenarios, wherein the large language model is constructed based on the transformer model. The large language model can be deployed in the power grid intelligent question-and-answer assistant and the automatic dispatch instruction generation model to parse the natural language instructions of power grid operators and generate corresponding operation suggestions or dispatch operation instructions. The conventional Transformer model is an encoder-decoder architecture. Based on model performance considerations, large language models usually adopt a pure decoder architecture, generate text through autoregression, and its hidden state is passed layer by layer to support real-time content generation. Therefore, the large language model in this embodiment also adopts a conventional pure decoder architecture, including an input embedding layer and multiple decoder layers. Each decoder layer includes a multi-head attention module and a feedforward neural network module. The input embedding layer is used to convert the input text into a high-dimensional vector representation, incorporating position encoding to retain sequence order information; the multi-head attention module uses a self-attention mechanism to only allow the current position to pay attention to the previous tokens in the sequence to avoid information leakage; the feedforward neural network module is used to perform feature transformation through fully connected layers and nonlinear activation functions (such as GELU); in addition, the encoder layer also includes residual connections and layer normalization to improve training stability and alleviate gradient disappearance. The specific structure of the large language model can refer to the conventional model structure based on the decoder architecture, and is not overly limited here.

[0057] In view of the architecture of the aforementioned large language model, this embodiment constructs a probe acquisition model consisting of multiple feature extraction modules. By deploying the probe acquisition model in the large language model, it realizes the acquisition and processing of high-dimensional hidden states, i.e., high-dimensional intermediate features, output by each layer of the large language model. The specific steps include:

[0058] Each feature extraction module is embedded in the output position of each decoder layer of the large language model, wherein the feature extraction module includes a linear transformation module and a compression dimension reduction module, and the decoder layer includes a multi-head attention module and a feedforward neural network module;

[0059] The linear transformation module collects high-dimensional intermediate features output by each decoder layer, and maps the high-dimensional intermediate features to a low-dimensional feature space to obtain a first low-dimensional intermediate feature;

[0060] The first low-dimensional intermediate features are compressed by the compression and dimensionality reduction module to obtain low-dimensional intermediate features corresponding to each decoder layer.

[0061] In this embodiment, the probe acquisition model is composed of multiple lightweight feature extraction modules, each feature extraction module includes a linear transformation module and a compression dimensionality reduction module. In order to achieve accurate identification of harmful features within the large language model and propagation path modeling, this embodiment embeds each feature extraction module into the output position of each decoder layer of the large language model, and collects high-dimensional intermediate features output by each layer of the large language model. Since each encoder layer includes a multi-head attention module and a feedforward neural network module, the feature extraction module is actually deployed at the output position of the multi-head attention module and the output position of the feedforward neural network module of each encoder layer. The high-dimensional intermediate features output by each decoding layer include the high-dimensional intermediate features output by the multi-head attention module and the high-dimensional intermediate features output by the feedforward neural network module.

[0062] After the probe acquisition model is embedded, the high-dimensional intermediate features output at each output position are collected through the linear transformation module, and parameterized mapping is performed through the fully connected layer of the linear transformation module to map the high-dimensional intermediate features to the low-dimensional feature space for preliminary compression, such as reducing the dimension from 1024 to 32, thereby obtaining the first low-dimensional intermediate features. Although the linear transformation module can reduce the dimensionality of high-dimensional data to reduce the subsequent calculation amount, it may still retain redundant information. Therefore, the first low-dimensional intermediate features will continue to be input into the compression and dimensionality reduction module, and principal component analysis or linear projection will be performed in the compression and dimensionality reduction module to further compress the first low-dimensional intermediate features, eliminate redundant information, and obtain lower-dimensional intermediate features, that is, low-dimensional intermediate features, such as compressing the dimension from 32 to 2D or 3D.

[0063] In this embodiment, the probe acquisition model is tightly coupled with the structure of the large language model and is embedded at the output end of the network backbone in a hierarchical distribution, thereby constructing a hierarchical feature acquisition system. All extracted low-dimensional intermediate features will be organized and stored in the form of time series in the order of forward propagation. A complete feature propagation trajectory is constructed through vertical aggregation, thereby showing the semantic evolution path of the input text between the layers within the large language model.

[0064] In order to measure the risk level of each low-dimensional intermediate feature, this embodiment introduces a risk assessment indicator based on vector distance. Specifically, for the low-dimensional intermediate features output by each layer, the harmful feature center and the security feature center corresponding to the features of this layer are pre-constructed through cluster analysis of jailbreak attack data and security data in the sample database. Then, the first distance between the low-dimensional intermediate feature and the harmful feature center, and the second distance between the low-dimensional intermediate feature and the security feature center are calculated. The distance value here can be calculated using Euclidean distance, cosine distance or other measurement algorithms, and no further restrictions are made here.

[0065] The first distance D1 is then subtracted from the second distance D2 to obtain the vector distance difference D, i.e., D = D1 - D2. This vector distance difference D is used as a risk assessment indicator. This risk assessment indicator can be used to determine which type of feature the low-dimensional intermediate feature is closer to, thereby accurately capturing harmful features. To better characterize the difference between the distances between the low-dimensional intermediate feature and the two feature centers, in this embodiment, the first and second distances are first normalized and scaled to the range [0, 1]. Then, the vector distance difference is calculated. According to the distance metric, if the low-dimensional intermediate feature is a safe feature, it should be far from the center of the harmful feature and close to the center of the safe feature. Therefore, D1 should be greater than D2, that is, D should be greater than zero. Therefore, in this embodiment, the difference threshold is set to zero. If D is greater than zero, the low-dimensional intermediate feature is considered a safe feature. Conversely, if D is less than zero, it indicates that the low-dimensional intermediate feature is closer to the center of the harmful feature and is therefore considered a harmful feature. Large language models are at risk of jailbreak attacks, and smaller D values ​​indicate a higher jailbreak attack risk.

[0066] This embodiment models characteristic trajectories through a detection and acquisition model, and performs risk quantification assessment on the characteristic trajectories, thereby providing accurate data support for subsequent defense mechanisms.

[0067] When a large language model is determined to be at risk of jailbreak attacks based on risk assessment indicators, this embodiment provides an explainable feature defense mechanism that proactively intervenes in the intermediate features of the large language model. A perturbation optimization model is constructed based on vector distance differences, and the optimal perturbation term is calculated and superimposed on the corresponding high-dimensional intermediate features, thereby proactively intervening in harmful intermediate features. The specific steps include:

[0068] A disturbance optimization model is constructed by taking maximization of the vector distance difference of the key layer as the objective function, taking the disturbance term as the optimization variable, and taking the disturbance amplitude being less than or equal to the disturbance amplitude threshold as the constraint condition;

[0069] The zero vector is used as the initial value of the disturbance term, and the disturbance optimization model is iteratively solved to obtain the optimal disturbance term.

[0070] In this embodiment, the key layers in the large language model are first selected. The key layers refer to the network layers that play a leading role in the semantic synthesis process of the model and are highly sensitive to the understanding of input semantics and output generation. Taking the large language model with a pure decoder architecture as an example, for the large language model with multiple decoder layer structures, the last few decoder layers in the model structure will have a strong impact on the final content generated. Therefore, the last few layers are defined as key layers. Assuming that the large language model is composed of 24 decoders stacked with the same structure, that is, the model has 24 layers, and the hidden state is transferred between layers through forward propagation, that is, the intermediate features are transferred, then the 21st to 24th layers can be defined as key layers. The low-dimensional intermediate features of these key layers include the low-dimensional intermediate features of the multi-head attention module and the low-dimensional intermediate features of the feedforward neural network module.

[0071] By introducing controllable perturbations at these key locations, this embodiment can adjust the representation space of a large language model at a low computational cost, thereby moving it away from harmful semantic representations and closer to safe representations. The perturbation terms can be calculated using a perturbation optimization model.

[0072] In this embodiment, a perturbation optimization model is constructed with the maximization of the vector distance difference corresponding to the low-dimensional intermediate features of the key layer as the objective function, the disturbance term as the optimization variable, and the perturbation amplitude being less than or equal to the preset perturbation amplitude threshold as the constraint condition:

[0073]

[0074] Where D1 represents the first distance, D2 represents the second distance, represents the maximum parameter function, represents the disturbance amplitude threshold, represents the disturbance term.

[0075] According to the harmful feature capture process, the closer the low-dimensional intermediate feature is to the harmful feature center, the smaller D1 is, and the corresponding larger D2 is, and the more harmful the low-dimensional intermediate feature is. Conversely, when D1 is larger and D2 is smaller, the safety of the low-dimensional intermediate feature is higher. Therefore, in this embodiment, the objective function is to maximize the vector distance difference of the key layer, and an iterative solution is performed to find the optimal disturbance term. When the optimal disturbance term is superimposed on the high-dimensional intermediate feature output of the key layer, the output feature of this layer of the large language model can be kept away from the harmful feature center and close to the safety feature center to the greatest extent.

[0076] When solving the model, the perturbation term is initialized to a zero vector and iteratively solved. Taking the kth iteration calculation as an example, the gradient of the objective function with respect to the perturbation term is calculated, and the perturbation direction is updated according to the gradient and learning rate:

[0077]

[0078] In the formula, α is the preset learning rate, is the gradient of the perturbation term at the kth iteration, is the disturbance term for the k+1th iteration.

[0079] For the perturbation term of k+1 iterations, determine whether the projection constraint is satisfied. If not:

[0080]

[0081] Then project it back into the constraint range:

[0082]

[0083] When the objective function converges or the maximum number of iterations is reached, the optimal perturbation term is output.

[0084] Finally, the optimal perturbation term is added to the high-dimensional intermediate features output by the key layer to obtain the safe intermediate features, thereby adjusting the intermediate representation of the large language model:

[0085]

[0086] Where, is the secure intermediate feature of the t-th key layer, is the high-dimensional intermediate feature of the t-th key layer, is the optimal disturbance term.

[0087] This embodiment intuitively reflects the disturbance direction and intensity through the feature center distance metric, has strong interpretability, and only operates on a small number of nodes in the key layer, so the overall computational complexity is extremely small, which improves the real-time dynamic defense capability of large language models. In addition, the active defense mechanism provided by this embodiment is applicable to various large language models of the Transformer architecture and has good compatibility.

[0088] In a preferred embodiment, to ensure that the features of the large language model are not excessively disturbed and thus affect normal output, the present invention adopts an adaptive disturbance constraint strategy to adjust the disturbance amplitude threshold for different semantic difference metrics. The specific steps include:

[0089] Calculating a reference distance between a harmful feature center and a safe feature center corresponding to the key layer, determining whether the reference distance is within a distance threshold range, and if so, calculating a first disturbance amplitude using a linear interpolation method based on an initial range of the disturbance amplitude, and using the first disturbance amplitude as a disturbance amplitude threshold;

[0090] Otherwise, the reference distance is compared with the upper limit value of the distance threshold range and the lower limit value of the distance threshold range respectively;

[0091] In response to the reference distance being greater than an upper limit of the distance threshold range, a disturbance reduction strategy is adopted according to the initial range of the disturbance amplitude and the reference distance to calculate a second disturbance amplitude, and the second disturbance amplitude is used as the disturbance amplitude threshold;

[0092] In response to the reference distance being less than the lower limit of the distance threshold range, a disturbance amplification strategy is adopted according to the initial range of the disturbance amplitude and the reference distance to calculate a third disturbance amplitude, and the third disturbance amplitude is used as the disturbance amplitude threshold.

[0093] In this embodiment, for a certain key layer, the baseline distance between the harmful feature center and the safety feature center of the key layer is first calculated, and then the distance threshold range of the baseline distance is set. The range can be a preset value, or it can be determined by statistical distribution analysis or quantile threshold method. When statistical distribution analysis is used, the mean and standard deviation of the baseline distances of all key layers are counted, and it is assumed that the baseline distance is normally distributed, and the distance threshold range is set according to the mean and standard deviation; when the quantile threshold method is used, the interquartile range of the baseline distance is calculated, and the distance threshold range is set according to the upper quartile, lower quartile and interquartile range. For details, please refer to the conventional threshold setting method based on normal distribution and the setting steps of the quantile threshold method, which will not be repeated here.

[0094] Then, it is determined whether the reference distance is within the distance threshold range. If so, the first disturbance amplitude is dynamically calculated by linear interpolation, and the first disturbance amplitude is used as the disturbance amplitude threshold in the objective function. The first disturbance amplitude can be expressed as:

[0095]

[0096] Where, represents the first disturbance amplitude, Indicates the lower limit of the initial range of disturbance amplitude, Indicates the upper limit of the initial range of the disturbance amplitude, Indicates the upper limit of the distance threshold range. represents the lower limit of the distance threshold range, and d represents the reference distance;

[0097] in, and is the default value, and The upper limit and lower limit of the distance threshold range are obtained by statistically analyzing the benchmark distances of all key layers.

[0098] If the reference distance is not within the normal range, the reference distance is judged to be too large or too small based on the comparison relationship between the reference distance and the upper and lower limits of the distance threshold range. If the reference distance is greater than the upper limit of the distance threshold range, it means that the feature boundary is clear. At this time, it is necessary to reduce the perturbation amplitude threshold and suppress the perturbation intensity to avoid semantic destruction. At this time, the second perturbation amplitude is calculated using the following formula and used as the perturbation amplitude threshold:

[0099]

[0100] Where, represents the second disturbance amplitude, γ represents the preset attenuation coefficient;

[0101] If the baseline distance is less than the lower limit of the distance threshold range, it means that the feature boundary is blurred. In this case, it is necessary to increase the perturbation amplitude threshold and enhance the perturbation intensity to improve defense capabilities. In this case, the third perturbation amplitude is calculated using the following formula and used as the perturbation amplitude threshold:

[0102]

[0103] Where, represents the third disturbance amplitude.

[0104] In a preferred embodiment, to further enhance the defense capabilities of large language models, the present invention also provides a feature fine-tuning mechanism. While maintaining the overall knowledge structure of the model, only key parameters strongly correlated with harmful features are fine-tuned to enhance the defense capabilities of the model. The specific steps include:

[0105] Constructing a loss function based on the difference between the high-dimensional intermediate features and the security intermediate features of the key layer and the adjustment amplitude of the model parameters of the key layer;

[0106] According to the loss function, the model parameters of the key layer are fine-tuned.

[0107] In this embodiment, the core idea of ​​the feature fine-tuning mechanism is to guide the internal representation of the model to evolve in a safer direction by minimizing the difference between the model output and the "safety intervention target", thereby effectively blocking the propagation path of harmful features. The safety intervention target here refers to the safe intermediate features generated by the above-mentioned active defense mechanism. Therefore, in this embodiment, based on the difference between the high-dimensional intermediate features and the safe intermediate features of the key layer and the adjustment amplitude of the model parameters of the key layer, a loss function is constructed, and the model parameters are fine-tuned by minimizing the loss function. Among them, the loss function can be expressed as:

[0108]

[0109] In the formula, θ represents the model parameters of the key layer after adjustment, θ0 represents the model parameters of the key layer before adjustment, β represents the regularization coefficient, represents the secure intermediate feature of the t-th key layer, represents the high-dimensional intermediate features of the t-1th key layer, Represents the forward mapping function under the action of model parameters θ.

[0110] In the loss function, the forward mapping function is used to calculate the high-dimensional intermediate features The output after the model parameter θ is affected by the model parameter θ. The secure intermediate feature is the secure intermediate feature adjusted by the active defense mechanism, which is used to guide the model output towards a safer representation alignment. The regularization coefficient is used to control the offset between the current parameter and the initial parameter. Its value is generally set within a small range to ensure that while enhancing security, the original knowledge structure of the model is retained to the maximum extent, thereby achieving an effective balance between adversarial adjustment and model stability.

[0111] Using this loss function, the model parameters of the key layers of the large language model are fine-tuned. During model training, a lightweight optimizer is preferably used, and a low learning rate is set to avoid large parameter fluctuations. This embodiment, through the secure fine-tuning mechanism of the constrained loss function, enables the model to dynamically block harmful features while retaining its original capabilities. This effectively balances real-time defense, semantic coherence, and engineering feasibility, thus providing an explainable technical path for the security reinforcement of large language models in power systems.

[0112] This embodiment provides a defense method for jailbreaking attacks on large language models. The probe acquisition model collects hidden state features of each layer of the large language model in real time, and combined with the low-dimensional feature propagation trajectory, it can dynamically locate harmful semantic evolution paths during the model's forward reasoning process; through the quantitative risk assessment of feature center distance, the interpretability of the model defense is improved; through the active defense mechanism based on controllable disturbances, the real-time dynamic defense capability of the large language model is effectively improved; through the feature dimensionality reduction and local parameter fine-tuning mechanism, the consumption of computing resources is effectively reduced, and the computing efficiency is improved. While blocking the attack, the original knowledge base of the model is retained to the maximum extent, avoiding the problem of catastrophic forgetting. The present invention realizes the integration of real-time and precise defense, explainable decision-making and low resource consumption through the collaborative mechanism of feature propagation trajectory analysis, lightweight disturbance intervention and selective parameter fine-tuning, providing a solution that takes into account both efficiency and practicality for the safe deployment of large language models in power systems.

[0113] See also Figure 2 Based on the same inventive concept, a second embodiment of the present invention proposes a defense system for large language model jailbreak attacks, including:

[0114] A feature acquisition module 10 is used to input text data into a large language model, collect high-dimensional intermediate features output by each decoder layer of the large language model through a pre-built probe acquisition model, and perform dimensionality reduction to obtain multiple low-dimensional intermediate features. The probe acquisition model includes several feature extraction modules. The large language model is constructed based on the transformer model;

[0115] an attack determination module 20 for calculating a first distance between the low-dimensional intermediate feature and a preset harmful feature center, and a second distance between the low-dimensional intermediate feature and a preset safe feature center, obtaining a vector distance difference based on the difference between the first distance and the second distance, and determining whether there is a jailbreak attack risk based on a comparison result between the vector distance difference and a preset difference threshold;

[0116] The active defense module 30 is used to respond to the risk of jailbreak attacks by taking the last several decoder layers in the large language model as key layers, maximizing the vector distance difference of the key layers as the objective function, constructing a perturbation optimization model, iteratively solving the perturbation optimization model to obtain the optimal perturbation term, and adding the optimal perturbation term to the corresponding high-dimensional intermediate features to obtain secure intermediate features.

[0117] The technical features and technical effects of the defense system for large language model jailbreak attacks proposed in the embodiment of the present invention are the same as those of the method proposed in the embodiment of the present invention, and will not be described in detail here. The various modules in the above-mentioned defense system for large language model jailbreak attacks can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0118] In addition, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0119] See also Figure 3, an internal structure diagram of a computer device in one embodiment, the computer device can specifically be a terminal or a server. The computer device includes a processor, a memory, a network interface, a display and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a defense method for large language model jailbreak attacks is implemented. The display screen of the computer device can be a liquid crystal display or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0120] It can be understood by those skilled in the art that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computing device may include more or fewer components than shown in the figure, or combine certain components, or have the same component arrangement.

[0121] In addition, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above method when the computer program is executed by a processor.

[0122] In summary, the embodiments of the present invention propose a method, system, device and storage medium for defending against jailbreak attacks on a large language model. The method inputs text data into a large language model, collects high-dimensional intermediate features output by each decoder layer of the large language model through a pre-built probe acquisition model and performs dimensionality reduction to obtain multiple low-dimensional intermediate features. The probe acquisition model includes several feature extraction modules, and the large language model is constructed based on a converter model; calculates a first distance between the low-dimensional intermediate feature and a preset harmful feature center, and a second distance between the low-dimensional intermediate feature and a preset security feature center, obtains a vector distance difference based on the difference between the first distance and the second distance, and judges whether there is a jailbreak attack risk based on a comparison result between the vector distance difference and a preset difference threshold; in response to the existence of a jailbreak attack risk, takes the last several decoder layers in the large language model as key layers, and constructs a perturbation optimization model with the maximization of the vector distance difference of the key layers as the objective function, iteratively solves the perturbation optimization model to obtain an optimal perturbation term, and adds the optimal perturbation term to the corresponding high-dimensional intermediate feature to obtain a safe intermediate feature. The present invention improves the real-time dynamic defense capability and interpretability of large language models in power systems through the collaborative mechanism of feature propagation trajectory analysis, lightweight disturbance intervention and selective parameter fine-tuning, effectively reduces computing resource consumption, and achieves an effective balance between the model's adversarial adjustment and stability.

[0123] Each embodiment in this specification is described in a progressive manner, and the same or similar parts of each embodiment can be directly referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. It should be noted that the various technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the various technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0124] The above-described embodiments merely represent several preferred implementations of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person skilled in the art could make several improvements and substitutions without departing from the technical principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be based on the scope of protection of the claims.

Claims

1. A defense method for large language model jailbreak attack, characterized in that: include: Input text data into a large language model, collect high-dimensional intermediate features output by each decoder layer of the large language model through a pre-built probe acquisition model and perform dimensionality reduction to obtain multiple low-dimensional intermediate features, the probe acquisition model includes several feature extraction modules, and the large language model is constructed based on the transformer model; calculating a first distance between the low-dimensional intermediate feature and a preset harmful feature center, and a second distance between the low-dimensional intermediate feature and a preset safe feature center, obtaining a vector distance difference based on a difference between the first distance and the second distance, and determining whether there is a jailbreak attack risk based on a comparison result between the vector distance difference and a preset difference threshold; In response to the risk of jailbreak attacks, the last several decoder layers in the large language model are used as key layers, and the maximization of the vector distance difference of the key layers is used as the objective function. A perturbation optimization model is constructed, and the perturbation optimization model is iteratively solved to obtain the optimal perturbation term. The optimal perturbation term is then added to the corresponding high-dimensional intermediate feature to obtain a secure intermediate feature.

2. The method for defending against jailbreak attacks on large language models according to claim 1, characterized in that: The step of collecting high-dimensional intermediate features output by each decoder layer of the large language model through the pre-built probe collection model and performing dimensionality reduction to obtain multiple low-dimensional intermediate features includes: Each feature extraction module is embedded in the output position of each decoder layer of the large language model, wherein the feature extraction module includes a linear transformation module and a compression dimension reduction module, and the decoder layer includes a multi-head attention module and a feedforward neural network module; The linear transformation module collects high-dimensional intermediate features output by each decoder layer, and maps the high-dimensional intermediate features to a low-dimensional feature space to obtain a first low-dimensional intermediate feature; The first low-dimensional intermediate features are compressed by the compression and dimensionality reduction module to obtain low-dimensional intermediate features corresponding to each decoder layer.

3. The method for defending against jailbreak attacks on large language models according to claim 1, characterized in that: The steps of constructing a disturbance optimization model with maximizing the vector distance difference of the key layer as the objective function, and iteratively solving the disturbance optimization model to obtain the optimal disturbance term include: A disturbance optimization model is constructed by taking maximization of the vector distance difference of the key layer as the objective function, taking the disturbance term as the optimization variable, and taking the disturbance amplitude being less than or equal to the disturbance amplitude threshold as the constraint condition; Taking the zero vector as the initial value of the disturbance term, iteratively solving the disturbance optimization model to obtain the optimal disturbance term; The objective function is expressed by the following formula: Where D1 represents the first distance, D2 represents the second distance, represents the maximum parameter function, represents the disturbance amplitude threshold, represents the disturbance term.

4. The method for defending against jailbreak attacks on large language models according to claim 3, characterized in that: The calculation step of the disturbance amplitude threshold comprises: Calculating a reference distance between a harmful feature center and a safe feature center corresponding to the key layer, determining whether the reference distance is within a distance threshold range, and if so, calculating a first disturbance amplitude using a linear interpolation method based on an initial range of the disturbance amplitude, and using the first disturbance amplitude as a disturbance amplitude threshold; Otherwise, the reference distance is compared with the upper limit value of the distance threshold range and the lower limit value of the distance threshold range respectively; In response to the reference distance being greater than an upper limit of the distance threshold range, a disturbance reduction strategy is adopted according to the initial range of the disturbance amplitude and the reference distance to calculate a second disturbance amplitude, and the second disturbance amplitude is used as the disturbance amplitude threshold; In response to the reference distance being less than the lower limit of the distance threshold range, a disturbance amplification strategy is adopted according to the initial range of the disturbance amplitude and the reference distance to calculate a third disturbance amplitude, and the third disturbance amplitude is used as the disturbance amplitude threshold.

5. The method for defending against jailbreak attacks on large language models according to claim 4, characterized in that: The first disturbance amplitude is expressed by the following formula: Where, represents the first disturbance amplitude, Indicates the lower limit of the initial range of disturbance amplitude, Indicates the upper limit of the initial range of the disturbance amplitude, Indicates the upper limit of the distance threshold range. represents the lower limit of the distance threshold range, and d represents the reference distance; The second disturbance amplitude is expressed by the following formula: Where, represents the second disturbance amplitude, γ represents the attenuation coefficient; The third disturbance amplitude is expressed by the following formula: Where, represents the third disturbance amplitude.

6. The method for defending against jailbreak attacks on large language models according to claim 1, characterized in that: After the step of obtaining the security intermediate feature, the method further includes: Constructing a loss function based on the difference between the high-dimensional intermediate features and the security intermediate features of the key layer and the adjustment amplitude of the model parameters of the key layer; According to the loss function, the model parameters of the key layer are fine-tuned.

7. The method for defending against jailbreak attacks on large language models according to claim 6, characterized in that: The loss function is expressed as follows: In the formula, θ represents the model parameters of the key layer after adjustment, θ0 represents the model parameters of the key layer before adjustment, β represents the regularization coefficient, represents the secure intermediate feature of the t-th key layer, represents the high-dimensional intermediate features of the t-1th key layer, Represents the forward mapping function under the action of model parameters θ.

8. A defense system for large language model jailbreak attacks, characterized in that: include: A feature acquisition module is used to input text data into the large language model, collect high-dimensional intermediate features output by each decoder layer of the large language model through a pre-built probe acquisition model, and perform dimensionality reduction to obtain multiple low-dimensional intermediate features. The probe acquisition model includes several feature extraction modules. The large language model is constructed based on the transformer model; an attack determination module, configured to calculate a first distance between the low-dimensional intermediate feature and a preset harmful feature center, and a second distance between the low-dimensional intermediate feature and a preset safe feature center, obtain a vector distance difference based on the difference between the first distance and the second distance, and determine whether there is a jailbreak attack risk based on a comparison result between the vector distance difference and a preset difference threshold; An active defense module is used to respond to the risk of jailbreak attacks by taking the last several decoder layers in the large language model as key layers, maximizing the vector distance difference of the key layers as the objective function, constructing a perturbation optimization model, iteratively solving the perturbation optimization model to obtain the optimal perturbation term, and adding the optimal perturbation term to the corresponding high-dimensional intermediate features to obtain secure intermediate features.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images