Prompt protection method based on large language model

Abstract

The invention provides a prompt protection method for a large language model, which comprises the following steps of: fully reserving safety related characteristics in a hidden state through nonlinear transformation by adopting an auto-encoder; secondly, a feature projection network FPN is introduced for feature decoupling, and a common feature vector and a purified feature vector are separated; and finally, constructing a context-aware multi-head attention mechanism, and enhancing the capture capability of the model on semantic relevance. Through the method, the safety prompt of the large language model is optimized, and the safety of the large language model is protected.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of information security, and particularly relates to a prompt protection method based on a large language model. BACKGROUND

[0003] Adding a security prompt before the input of a large language model is a common practice to protect the large language model from harmful intent queries. However, the underlying working principle of the security prompt has not been revealed, which limits the automatic optimization of them to improve the safety of LLMs. Experiments show that different security prompts move the hidden state of the query in the direction of rejection, in this direction, the model becomes more sensitive to generating a rejection response, even if the query itself is harmless. That is, the security prompt does not make harmful and harmless queries more easily distinguishable, but increases the overall rejection probability of the model.

[0004] Therefore, it is very meaningful to optimize the security prompt, treat the security prompt as a continuous, trainable embedding vector, and move its representation in the rejection direction or the opposite direction according to the harmfulness of the query, in order to improve the effectiveness and robustness of the model. SUMMARY

[0005] In view of this, the present disclosure provides a prompt protection method based on a large language model to optimize the security prompt and establish a more accurate and robust protection mechanism.

[0006] The technical scheme provided by the present disclosure is a prompt protection method based on a large language model, which includes the following steps: in the large language model, a prompt protection module is placed between the user query input and the generation of a response by the large language model, the security prompt embedding is dynamically optimized by the prompt protection module, and then the processed query is input to the large language model; wherein the dynamic optimization of the security prompt embedding by the prompt protection module includes the following:

[0007] S1: using an autoencoder method to project the high-dimensional model hidden state into a low-dimensional space;

[0008] S2: using a feature projection network FPN to decompose the hidden state after dimensionality reduction by the autoencoder, and decomposing the hidden state into a public feature vector and a purification vector;

[0009] S3: generating a context-aware enhancement vector using a multi-head attention mechanism;

[0010] S4: fitting the relationship between the enhancement vector and the rejection probability by logistic regression; defining the regression coefficient as the "rejection direction", which represents the direction in which the model rejection probability increases;

[0011] S5: optimizing the model through a loss function; after optimization, the model judges the new input user query in real time according to the adjusted security prompt embedding, and if the query is identified as malicious, a rejection response is triggered; if it is a normal query, a reply is normally generated, thereby realizing security protection of the large language model.

[0012] Preferably, S1 specifically comprises:

[0013] The hidden state x is normalized:

[0014]

[0015] Wherein, μ and σ represent the mean and standard deviation of the hidden state x respectively, and x' represents the normalized hidden state;

[0016] The hidden state x' is input into the encoder as input data to obtain a low-dimensional representation z:

[0017] z = f enc (x') = σ(W e x' + b e ) (2)

[0018] Wherein, W e is the weight matrix of the encoder, b e is the bias vector of the encoder, σ is the sigmoid activation function, and z is the compressed low-dimensional representation;

[0019] The low-dimensional representation is restored to the original data

[0020]

[0021] Wherein, W d is the weight matrix of the decoder, b d is the bias vector of the decoder, σ is the sigmoid activation function, is the reconstructed high-dimensional vector;

[0022] The difference between the input data x' and the reconstructed data is calculated using a loss function to minimize the difference, wherein the mean square error MSE is used as the loss function;

[0023]

[0024] Wherein, n is the number of samples, x' and represent the normalized hidden state and the reconstructed output data respectively.

[0025] Preferably, S2 specifically comprises:

[0026] S21: generating vector z by two convolutional neural networks with same structure but different parameters respectively common and z p ;

[0027] z common =CNN public (z) (7)

[0028] z p =CNN pure (z) (8)

[0029] wherein CNN public represents a convolutional network for extracting common features, CNN pure represents a convolutional network for extracting individual features, and z common represents a common feature vector;

[0030] S22: the feature projection network comprises two parts: C-net and P-net;

[0031] C-net extracts a common feature vector z common , which is gradually discarded of individual information and only carries common information under the action of gradient reversal layer GRL during the training process;

[0032] GRL(z common ) = z common (9)

[0033]

[0034] wherein GRL represents a gradient reversal layer, λ represents a gradient reversal coefficient, and I represents a unit matrix;

[0035] S23: P-net is a purification vector extraction network, and vector z common is projected in the orthogonal projection layer OPL of P-net together with vector z p to obtain a specific feature vector z pure which removes common features and only retains task-related specific features;

[0036] S24: the purification vector z pure and the common feature vector z common are classified by a Softmax normalization function:

[0037] Y p = softmax(W p ·z pure +b p ) (14)

[0038] Y c = softmax(Wc ·z common +b c (15)

[0039] Where Softmax represents the normalization function, W p and W c Let b represent the weight matrix of the classifier. p and b c Y represents the bias term of the classifier. p and Y c This represents the predicted classification probability distribution;

[0040] S25: Constructing the loss function:

[0041] Loss p =CrossEntropy(Y truth ,Y p (16)

[0042] Loss c =CrossEntropy(Y truth ,Y c (17)

[0043] Where CrossEntropy represents the cross-entropy loss function, Y truth Indicates the true label, Loss p Loss represents the loss for purified feature classification. c This represents the common feature classification loss.

[0044] Preferably, the vector z common With vector z p Projection operations are performed in the orthographic projection layer (OPL) of P-net, including two projections: the first projection is performed on z... p Projected onto z common In the direction of z, we obtain z', that is, in z common The component z' in the direction; the second time z p Projecting onto the orthogonal direction of z', i.e., the direction of z-z', yields z. pure , z pure With common feature vector z common vertical;

[0045] z′=Projection(z p ,z common (11)

[0046] z pure =Projection(z p ,z p -z′) (12)

[0047]

[0048] wherein, Projection denotes a projection operation, z1 denotes a vector to be projected, z2 denotes a target direction vector of the projection,

[0049] |z2| denotes the length of z2.

[0050] Preferably, S3 generates the context-aware enhanced vector using the multi-head attention mechanism, comprising:

[0051] S31: generating a purified vector z pure performing linear transformation to generate query Q, key K and value V matrix:

[0052] Q=W Q z pure K=W K z pure V=W V z pure (18)

[0053] wherein, W Q , W K and W V are linear transformation matrices of query Q, key K and value V respectively, Q, K, V represent the corresponding query, key and value matrices as inputs of the attention mechanism;

[0054] S32: parallel computing multiple attentions: for each head, respectively using scaled dot-product attention mechanism to compute attention output:

[0055]

[0056] wherein, d k denotes the dimension of the key vector for scaling; Q i , K i , V i are inputs of the i-th attention head; Softmax is a normalization function, Attention i denotes the output result of the i-th attention head;

[0057] S33: output concatenating attention heads:

[0058] z attn =MultiHead(Q,K,V)=Concat(Attention1,…,Attention h )·W O (20)

[0059] wherein, h denotes the number of attention heads, WO represents the output linear transformation matrix, z attn is the context-aware augmented vector.

[0060] Preferably, the direction of rejection probability increase is determined by fitting the empirical rejection probability of the anchor data in S4 by logistic regression: the empirical rejection probability is the number of model rejections per query divided by the total number of samples.

[0061]

[0062] where, f r represents the rejection direction function, represents the transpose matrix of the weight vector of the rejection direction, b r represents the bias term, and σ is the sigmoid function.

[0063] Preferably, the model optimization objective in S5 is to move the query representation in the rejection direction, i.e. harmful queries, or in the opposite direction, i.e. harmless queries, by adjusting the embedding of the continuous security cues:

[0064] The harmful query: adjust the security cues so that their hidden state moves in the wr direction → increase the rejection probability;

[0065] The harmless query: adjust the security cues so that their hidden state moves in the opposite direction of wr → reduce the false rejection probability;

[0066] Rejection loss function

[0067]

[0068] Δ r = f r (z′ attn ) - f r (z attn ) (22)

[0069] Harmfulness identification loss

[0070]

[0071] Δ h = f h (z′ attn ) - f h (z attn ) (23)

[0072] Regularization loss function

[0073]

[0074] Total loss:

[0075] Wherein, z attn represents the vector representation of the initial input security prompt, z' attn represents the vector representation after fine-tuning the security prompt, l is a label, 1 represents a harmful query, 0 represents a harmless query, f r is a rejection direction function, f h is a harmfulness identification function, Δ r is the difference in rejection direction before and after optimization, Δ h is the difference in harmful direction before and after optimization, is the transpose matrix of the orthogonal basis matrix, and β is a regularization coefficient.

[0076] The application proposes a prompt protection method based on a large language model, which improves the shortcomings of existing security prompt optimization technology and establishes a more accurate and robust protection mechanism. For the information loss and noise interference affecting direction discrimination caused by the linear dimension reduction method, the proposed scheme: first, an autoencoder method is used to fully retain the security-related features in the hidden state through nonlinear transformation; second, a feature projection network FPN is introduced to decouple the features, separating the public feature vector and the purified feature vector; finally, a multi-head attention mechanism with context awareness is constructed to enhance the model's ability to capture semantic associations.

[0077] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the disclosure of the application. BRIEF DESCRIPTION OF DRAWINGS

[0078] The accompanying drawings incorporated into the specification and forming a part thereof, illustrate embodiments consistent with the present application and, together with the specification, serve to explain the principles of the application.

[0079] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0080] Figure 1 The self-encoder composition framework diagram provided by the disclosed embodiment of the present application;

[0081] Figure 2 The feature projection network FPN model schematic diagram provided by the disclosed embodiment of the present application;

[0082] Figure 3 The working principle diagram of the orthogonal projection layer provided by the disclosed embodiment of the present application;

[0083] Figure 4 A flow chart of a prompt protection method of a large language model is provided for the disclosed embodiment of the present application. DETAILED DESCRIPTION

[0084] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers refer to the same or similar elements throughout the drawings. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they only represent examples of systems consistent with some aspects of the present application, as detailed in the appended claims.

[0085] To solve the deficiencies of the current large language model security prompt technology, the present application provides a prompt protection method of a large language model to achieve the purpose of protecting the security of the large language model.

[0086] First, in order to facilitate subsequent calculation and optimization, the high-dimensional model hidden state needs to be projected into a low-dimensional space. The autoencoder is selected for dimension reduction. The autoencoder (AE) is an unsupervised learning neural network model, mainly used for data dimension reduction and feature learning. The basic idea is to compress the input data into a low-dimensional representation through the encoder (Encoder), and then try to restore the original data from this low-dimensional representation through the decoder (Decoder). Secondly, because there may be some fixed format or high-frequency query words unrelated to security in the query statement, which may affect the subsequent calculation of the hidden state. Therefore, the feature projection network FPN is used here to extract the purification vector in the hidden state, and then the multi-head attention mechanism is used to capture the context information to generate an enhanced vector, and then the relationship between the enhanced hidden state and the rejection probability is fitted through logistic regression. The regression coefficient is defined as the "rejection direction", which represents the direction in which the model increases the rejection probability. Finally, the model is optimized through the loss function.

[0087] Specifically, it includes:

[0088] S0: A large number of malicious instruction templates are collected from open source platforms such as Huggingface, and the collected data is semantically expanded by a large language model to build a data set for model training and evaluation;

[0089] S1: The high-dimensional model hidden state is projected into a low-dimensional space using the autoencoder method;

[0090] The autoencoder is selected for dimensionality reduction, which is a type of neural network that includes an encoder, a hidden layer, and a decoder. The basic idea is to compress the input data into a low-dimensional representation through the encoder, and then restore this low-dimensional representation to the original data through the decoder. The purpose is to make the reconstructed data of the decoder close to the original data. First, the hidden state is normalized and then input into the encoder to obtain a low-dimensional representation, and then reconstructed into the original data through the decoding layer. The loss function is to calculate the difference between the input data x' and the reconstructed data , so as to minimize the difference. The low-dimensional representation z in the hidden layer is the result to be obtained. The detailed process is as follows:

[0091] S11: First, normalize the hidden state x:

[0092]

[0093] where μ and σ represent the mean and standard deviation of the hidden state x respectively, and x' represents the normalized hidden state.

[0094] S12: Encoder: map the input data x' to a low-dimensional representation z:

[0095] z = f enc (x') = σ(W e x' + b e ) (2)

[0096] where W e is the weight matrix of the encoder, b e is the bias vector of the encoder, σ is the sigmoid activation function, and z is the compressed low-dimensional representation.

[0097] S13: Decoder: reconstruct the low-dimensional representation z into the original data

[0098]

[0099] where W d is the weight matrix of the decoder, b d is the bias vector of the decoder, σ is the sigmoid activation function, is the reconstructed high-dimensional vector.

[0100] S14: Loss function: minimize the difference between the input data x' and the reconstructed data , using mean square error (MSE) as the loss function:

[0101]

[0102] where n is the number of samples, x' and respectively represent the normalized hidden state and the reconstructed output data.

[0103] S2: Adopting feature projection network FPN to decompose the hidden state z after dimension reduction by the autoencoder, and decompose z into a common feature vector and a purification vector.

[0104] Because there may be some fixed format or high-frequency query words unrelated to security in the security prompt and query statement, which may affect the subsequent calculation of the hidden state. Therefore, the present embodiment uses the feature projection network FPN to extract the purification vector in the hidden state, and filter irrelevant noise.

[0105] Specifically, the following is included:

[0106] S21: Generating vectors z common and z p from the hidden state by two convolutional neural networks with the same structure but different parameters.

[0107] z common = CNN public (z) (7)

[0108] z p = CNN pure (z) (8)

[0109] Wherein, CNN public represents a convolutional network for extracting common features, CNN pure represents a convolutional network for extracting individual features, z c o mmon represents a common feature vector.

[0110] The feature projection network includes two parts: C-net and P-net. C-net is to extract a common feature vector, and P-net is to extract a purification vector.

[0111] S22: C-net: z commonDuring the training process, the gradient reversal layer GRL gradually discards the individual information and only retains the common information. During the forward propagation process, the gradient reversal layer does not change the input data, but during the backward propagation process, the GRL reverses the gradient passed to the previous node, and the loss function value will continue to increase. (For example, a gradient of +0.5 indicates that an increase in the parameter will increase the loss, and therefore the model will reduce the parameter value to reduce the loss. After the gradient is reversed by the GRL, the optimizer will mistakenly believe that an increase in the parameter can reduce the loss (in fact, it will increase the loss), and therefore increase the parameter value. This causes the classification loss of the C-net to continue to increase, forcing the network to extract common features that are unrelated to the categories.) The purpose of this is to force the C-net to extract features that do not contain information specific to the input sequence, but rather retain common features, because these common features cannot help the classifier to correctly classify, resulting in an increase in the classification loss.

[0112] GRL(z common )=z common (9)

[0113]

[0114] where GRL represents the gradient reversal layer, λ represents the gradient reversal coefficient, and I represents the unit matrix.

[0115] S23: P-net: vector z common is projected in the orthogonal projection layer OPL of the P-net, and the common feature vector z p is obtained, which only retains the specificity feature vector z pure related to the task.

[0116] The OPL performs 2 projections: the first projection is from z p to the direction of z common , obtaining z', which is the component z' in the direction of z common . The second projection is from z p to the orthogonal direction of z' (the direction of z-z'), obtaining z pure .

[0117] The working principle of the orthogonal projection layer is shown in Figure 3 . Since z pure is perpendicular to the common feature vector z common , z pure removes the common features in the original vector z, and uses the vector z pure as the final purified vector.

[0118] z′=Projection(z p ,z common ) (11)

[0119] z pure = Projection(z p , z p - z') (12)

[0120]

[0121] wherein Projection denotes a projection operation, z1 denotes a vector to be projected, z2 denotes a target direction vector of the projection,

[0122] |z2| denotes a norm of z2.

[0123] S24: A Softmax normalization function is applied to the purified vector z pure and the public feature vector z common to classify:

[0124] Y p = softmax(W p ·z pure +b p ) (14)

[0125] Y c = softmax(W c ·z common +b c ) (15)

[0126] wherein Softmax denotes a normalization function, W p and W c denote weight matrices of the classifier, b p and b c denote bias terms of the classifier, Y p and Y c denote predicted classification probability distributions.

[0127] S24: A loss function:

[0128] Loss p = CrossEntropy(Y truth , Y p ) (16)

[0129] Loss c = CrossEntropy(Y truth , Y c ) (17)

[0130] wherein CrossEntropy denotes a cross-entropy loss function, Y truth denotes a true label. Loss p denotes a purified feature classification loss, which is calculated by Lossp Optimization ensures that the model can accurately identify harmfulness. Loss c represents the public feature classification loss, which is Loss c Through the adversarial training of, it is ensured that the public features do not contain harmfulness discrimination information, and the model is prevented from being misjudged due to irrelevant features (such as high-frequency words in harmless queries).

[0131] S3: Generate context-aware enhanced vectors through multi-head attention mechanism;

[0132] Attention mechanism gives different weights to different places, and the more important or more relevant places are given higher weights. Multi-head attention is to divide into multiple attention heads for parallel calculation, and then splice the results. Specifically, after generating Q, K and V matrices, first, multiply Q and K matrices, divide by the dimension of the root to reduce dimension, get the attention score matrix, then multiply the attention score matrix and the V matrix, get the output of a single attention head. Finally, splice the results of all attention heads to get the final enhanced vector Zattn.

[0133] Specifically, it includes: S31: Perform linear transformation on the above obtained purification vector z pure to generate query Q, key K and value V matrices:

[0134] Q=W Q z pure K=W K z pure V=W V z pure (18)

[0135] Where, W Q , W K and W V are linear transformation matrices of query Q, key K and value V, Q, K, V represent the corresponding query, key and value matrices, which are used as inputs of the attention mechanism.

[0136] S32: Parallel computing multiple attentions: for each head, use scaled dot-product attention mechanism to calculate attention output respectively:

[0137]

[0138] Where, d k represents the dimension of the key vector for scaling. Q i , K i , V i are the inputs of the i-th attention head. Softmax is a normalization function, and Attention i represents the output result of the i-th attention head.

[0139] S33: Concatenate the outputs of the attention heads:

[0140] z attn = MultiHead(Q, K, V) = Concat(Attention1, …, Attention h )· W O (20)

[0141] where h denotes the number of attention heads, W O denotes the output linear transformation matrix, and z attn is the enhanced vector of contextual awareness.

[0142] S4: Fit the relationship between the enhanced vector and the rejection probability using logistic regression;

[0143] Fit the empirical rejection probability of the anchor data through logistic regression to determine the direction of the increase in rejection probability. The empirical rejection probability is the proportion of rejections in 20 model answers for each query, and the empirical rejection probability = number of model rejections / total number of samples.

[0144]

[0145] where f r denotes the rejection direction function, denotes the transpose matrix of the weight vector of the rejection direction, b r denotes the bias term, and σ is the sigmoid function.

[0146] S5: Optimize the model through the loss function. Optimization goal: adjust the embedding of the continuous security prompt to make the query representation move in the rejection direction (harmful query) or the reverse direction (harmless query).

[0147] Harmful query: adjust the security prompt to make its hidden state move in the wr direction → increase the rejection probability.

[0148] Harmless query: adjust the security prompt to make its hidden state move in the reverse wr direction → reduce the false rejection probability.

[0149] Adjust the security prompt to control the rejection probability through the rejection loss function Enhance the harmfulness identification ability through the harmfulness identification loss Prevent the security prompt from deviating too far from the original semantics through the regularization loss function

[0150] Rejection loss function

[0151]

[0152] Δ​r = f r (z' attn ) - f r (z attn ) (22)

[0153] Harmfulness identification loss

[0154]

[0155] Δ h = f h (z' attn ) - f h (z attn ) (23)

[0156] Regularization loss function

[0157]

[0158] Total loss:

[0159] where z attn represents the vector representation of the initial input security prompt, z' attn represents the vector representation after fine-tuning the security prompt, l is the label (1 represents harmful query and 0 represents harmless query), f r is the rejection direction function, f h is the harmfulness identification function. Δ r is the difference in rejection direction before and after optimization. Δ h is the difference in harmful direction before and after optimization, is the transpose matrix of the orthogonal basis matrix, used to constrain the semantic shift of the security prompt. β is the regularization coefficient (default is 0.001).

[0160] After optimization, the model judges the new input user query according to the adjusted security prompt embedding. If it is identified as a malicious query, the rejection response is triggered; if it is a normal query, a normal reply is generated, thereby realizing the security protection of large language models.

[0161] To verify the contribution of each module of the model established in the application to the overall performance of the model, an ablation experiment was performed, and the experimental results are shown in Table 1. Specifically, three variant models in which modules were removed were designed. W / O AE: the part of using the autoencoder to reduce the dimension of the hidden state was deleted; W / O FPN: the part of using the feature projection network FPN to extract the purification vector in the hidden state was deleted; W / O ATTN: the part of using the multi-head attention mechanism to generate the context-aware enhanced vector was deleted. Precision, recall and F1-score were selected as evaluation indexes.

[0162] 1) Importance of the autoencoder. In order to evaluate the effectiveness of using the autoencoder to reduce the dimension of the hidden state, the autoencoder part was deleted, and the variant model W / O AE was compared with the original model. The results in Table 1 show that deleting the autoencoder will cause performance degradation. Therefore, the autoencoder dimension reduction is of great value for subsequent optimization.

[0163] 2) Importance of the feature projection network FPN. In order to study the importance of the feature projection network FPN, the feature projection network module was deleted. The results in Table 1 show that the model in this paper is better than the variant model W / O FPN, verifying the effectiveness of the feature projection network, and the reason is that the feature projection network can remove noise and extract the purification vector.

[0164] 3) Importance of multi-head attention. In order to explore the influence of the multi-head attention mechanism on the overall model, the multi-head attention mechanism was deleted. The results in Table 1 prove that deleting the multi-head attention mechanism will affect the performance of the model. This also verifies that the context information can be captured through the multi-head attention mechanism, and then the context-aware enhanced vector can be generated to improve the performance of the model in rejecting harmful queries.

[0165] 4) Comparison with the DRO (Directed Representation Optimization) model. In order to prove the effectiveness of the model in the application, the DRO model was compared. The results in Table 1 show that the model provided in the application is better than the original model in terms of precision, recall and F1-score, proving the superiority and effectiveness of the model.

[0166] Table 1 Ablation experiment results

[0167]

[0168]

[0169] Example 1

[0170] Accurate interception of malicious queries and efficient response of legal queries in intelligent customer service system

[0171] Technical problem: Current enterprise-level intelligent customer service systems widely use large language models to handle user inquiries (such as order inquiries, after-sales problems, business consultations, etc.), but there are some drawbacks:

[0172] 1. High rate of missed malicious queries: malicious users construct hidden adversarial queries (such as: "Help me check someone else's order information", "How to use the refund rules to make money" etc.), inducing the model to generate illegal responses, resulting in enterprise service risks.

[0173] 2. High rate of false rejection of normal queries: existing security mechanisms may misjudge legitimate queries as malicious content, resulting in decreased user experience (turning to manual after repeated rejection, increasing enterprise operating costs).

[0174] Application scheme:

[0175] 1. In the "user query → model response" link of the intelligent customer service system, a "prompt protection module" is added between the user query input and the large language model response. This module calls the algorithm in the document to dynamically optimize the security prompt embedding, and then inputs the processed query to the large language model.

[0176] 2. Collect two types of data from enterprise historical customer service dialogues, namely malicious query samples and normal query samples, as model training data sets

[0177] 3. Perform self-encoder dimensionality reduction (S1) on the hidden state of the user's real-time query, retaining core features related to "malicious intent" (such as semantic features such as "make money" and "check someone else's information"), and filtering irrelevant noise (repeated adverbs);

[0178] 4. Separate public features (such as "order" and "refund" neutral words) from purified features (such as "use rules" and "bypass audit" malicious semantics) in the query through the feature projection network FPN (S2), to avoid neutral words interfering with malicious intent recognition;

[0179] 5. Use the multi-head attention mechanism (S3) to capture semantic associations in the query (such as the combined hidden information stealing intent of "help me check" + "strange mobile phone order"), generate context-aware enhanced vectors, and improve the accuracy of malicious intent recognition for complex sentence patterns;

[0180] 6. Determine the "rejection direction" through logistic regression (S4) to classify real-time queries: if it is a malicious query (such as containing "make money" and "steal information" features), adjust the security prompt to move the hidden state in the rejection direction, triggering the model to reject the response (such as "This request does not meet the service specifications"); if it is a normal query (such as "modify the delivery address"), adjust the security prompt to move the hidden state in the opposite direction of rejection, ensuring that the model generates valid replies normally;

[0181] 7. Continuously optimize the model through the loss function in the document (S5), aiming to reduce the false negative rate of malicious queries (improve recall rate) and the false rejection rate of normal queries (improve precision rate).

[0182] Technical effects:

[0183] (1) Reduce the false negative rate of malicious queries: through purification feature extraction and context association capture, the intelligent customer service system improves the recognition accuracy of hidden malicious queries (such as "how to pay less money with your rules" and other ambiguous malicious queries);

[0184] (2) Reduce the false rejection rate of normal queries: by dynamically adjusting the direction of security prompts, avoid misjudging "order query" and "after-sales consultation" and other legal requests as malicious, so that the false rejection rate is reduced to below 5%;

[0185] (3) Adapt to complex scenarios: support the differentiated needs of multi-business line intelligent customer service (such as e-commerce, finance, government affairs), adjust the "rejection direction" weight, and adapt to different scenarios of security standards (such as the financial customer service's interception of "cash-out" queries is stricter than that of the e-commerce customer service).

Claims

1. A prompt guard method of a large language model, characterized in that, In a large language model, a prompt protection module is placed between user query input and large language model response generation, a safety prompt embedding is dynamically optimized through the prompt protection module, and the processed query input is input to the large language model; wherein the prompt protection module dynamically optimizes the safety prompt embedding, including the following: S1: using an autoencoder method to project high-dimensional model hidden states into a low-dimensional space; S2: using a feature projection network FPN to decompose the hidden state after dimensionality reduction by the autoencoder, and decomposing the hidden state into a public feature vector and a purification vector; S3: generating a context-aware enhancement vector using a multi-head attention mechanism; S4: fitting the relationship between the enhancement vector and the rejection probability through logistic regression; the regression coefficient is defined as the "rejection direction", indicating the direction in which the model increases the rejection probability; S5: optimizing the model through a loss function; after optimization, the model judges the newly input user query in real time according to the adjusted safety prompt embedding, and if it is identified as a malicious query, a rejection response is triggered; if it is a normal query, a reply is generated normally, thereby achieving safety protection for the large language model.

2. The method of claim 1, wherein the method further comprises: S1 specifically includes: Normalizing the hidden state x: Wherein μ and σ represent the mean and standard deviation of the hidden state x respectively, and x' represents the normalized hidden state; Inputting the hidden state x' as input data into the encoder to obtain a low-dimensional representation z: z = f enc (x') = σ(W e x' + b e ) (2) where W e is the weight matrix of the encoder, b e is the bias vector of the encoder, σ is the sigmoid activation function, and z is the compressed low-dimensional representation; restoring the low-dimensional representation to the original data by a decoder where W d is the weight matrix of the decoder, b d is the bias vector of the decoder, σ is the sigmoid activation function, is the reconstructed high-dimensional vector; The loss function is used to calculate the input data x' and the reconstructed data. The difference between them is minimized, where the mean squared error (MSE) is used as the loss function. where n is the number of samples, x' and represent the normalized hidden state and reconstructed output data, respectively.

3. The method of claim 1, wherein the method further comprises: S2 specifically includes: S21 : generating the hidden state vector z through two convolutional neural networks with the same structure but different parameters respectively common and z p ; z common = CNN public (z) (7) z p = CNN pure (z) (8) wherein CNN public represents a convolutional network extracting common features, CNN pure represents a convolutional network extracting individual features, z common represents a common feature vector; S22: the feature projection network includes two parts: C-net and P-net; C-net extracts common feature vectors, z common In the training process, the gradient reverse layer GRL is used to gradually discard the individual information and only retain the common information. GRL(z common ) = z common (9) Wherein GRL represents a gradient reverse layer, λ represents a gradient reversal coefficient, and I represents a unit matrix; S23: P-net is to extract the purification vector, vector z common With vector z p In the orthogonal projection layer OPL of P-net, the projection operation is carried out, and the vector z of specific features related to the task is obtained by removing common features pure ; S24: Normalize the purification vector z by a Softmax function pure and the public feature vector z common Classify: Y p = softmax(W p ·z pure +b p ) (14) Y c = softmax(W c ·z common +b c ) (15) where Softmax denotes a normalization function, W p and W c denote the weight matrix of the classifier, b p and b c denote the bias term of the classifier, Y p and Y c denote the predicted class probability distribution; S25: constructing a loss function: Loss p = CrossEntropy(Y truth , Y p ) (16) Loss c = CrossEntropy(Y truth , Y c ) (17) where CrossEntropy denotes a cross-entropy loss function, Y truth denotes a real label, Loss p denotes a purification feature classification loss, Loss c denotes a common feature classification loss.

4. The method of claim 3, wherein the method further comprises: The vector z common is perpendicular to the common feature vector z p In the orthogonal projection layer OPL of the P-net, the projection operation is performed: including: twice projection in the orthogonal projection layer OPL: the first time, z p is projected into the direction of z common , to obtain z', that is, the component z' in the direction of z common ; the second time, z p is projected into the orthogonal direction of z', that is, the direction of z-z', to obtain z pure , z pure is perpendicular to the common feature vector z common ; z' = Projection(z p ,z common ) (11) z pure = Projection(z p ,z p -z′) (12) Wherein Projection represents a projection operation, z1 represents a vector to be projected, z2 represents a target direction vector for projection, and |z2| represents the length of z2.

5. The method of claim 1, wherein the method further comprises: S3 includes generating a context-aware enhancement vector using a multi-head attention mechanism, including: S31: Generating the query Q, the key K and the value V matrices by the linear transformation vector z pure Performing linear transformation to generate query Q, key K and value V matrix: Q = W Q z pure K = W K z pure V = W V z pure (18Θ) where W Q , W K , and W V are linear transformation matrices of the query Q, the key K, and the value V, respectively, and Q, K, and V represent the corresponding query, key, and value matrices as inputs of the attention mechanism. S32: parallel computing multiple attentions: for each head, respectively using a scaled dot-product attention mechanism to compute attention output: wherein d k represents the dimension of the key vector, used for scaling; Q i , K i , V i is the input of the i-th attention head; Softmax is a normalization function, Attention i represents the output result of the i-th attention head; S33: outputting a concatenated attention head: z attn = MultiHead(Q, K, V) = Concat(Attention1, Attention h ) · W O (20) where h denotes the number of attention heads, W O denotes the output linear transformation matrix, z attn is the context-aware enhancement vector.

6. The method of claim 1, wherein the method further comprises: In S4, the empirical rejection probability of the anchor data is fitted through logistic regression to determine the direction in which the rejection probability increases: the empirical rejection probability is a statistical model for each query, and the empirical rejection probability = model rejection times / total sampling times; where f r denotes the rejection direction function, denotes the transpose matrix of the weight vector of the rejection direction, b r denotes the bias term, and σ is the sigmoid function.

7. The method of claim 1, wherein the method further comprises: The optimization goal of the model in S5 is to adjust the embedding of continuous safety prompts so that the query representation moves in the rejection direction, i.e. harmful queries or the reverse direction, i.e. harmless queries: The harmful query: adjust the safety prompt so that its hidden state moves in the wr direction → increase the rejection probability; The harmless query: adjust the safety prompt so that its hidden state moves in the reverse wr direction → reduce the false rejection probability; loss rejection function Δ r = f r (z′ attn ) - f r (z attn ) (22) Harmfulness recognition loss Δ h = f h (z′ attn ) - f h (z attn ) (23) Regularized loss function Total loss: where z attn represents the vector representation of the initial input security hint, z' attn represents the vector representation after fine-tuning the security hint, l is the label, 1 represents a harmful query, and 0 represents a harmless query, f r is a rejection direction function, f h is a harmfulness identification function, Δ r is the difference in the rejection direction before and after optimization, Δ h is the difference in the harmful direction before and after optimization, is the transpose matrix of the orthogonal basis matrix, and β is a regularization coefficient.

Images