Method and device for preventing privacy attack by using hidden state filtering

Abstract

The embodiment of the invention discloses a privacy attack prevention method and device using hidden state filtering. The method comprises the steps of obtaining a to-be-processed request; based on the to-be-processed request, obtaining a cascade vector Tk of a vector tk of k marks of a final decoder of a large language model; the cascade vector Tk is input into a pre-trained lightweight classifier to obtain a hazard score, the pre-trained lightweight classifier is a multi-layer perceptron, the multi-layer perceptron comprises an input layer, hidden layers and an output layer, and the hidden layers comprise a first hidden layer, a second hidden layer and a third hidden layer; if the harmfulness score is larger than or equal to a preset threshold value, the large language model refuses to generate a response to the to-be-processed request, and if the harmfulness score is smaller than the preset threshold value, the large language model responds to the to-be-processed request.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly to a method and device for preventing privacy attacks by using hidden state filtering. Background Art

[0002] As the capabilities of large-scale language models (LLMs) such as ChatGPT, Llama, and Mistral are rapidly improving, people are increasingly concerned about the potential security risks behind their powerful functions, including the possible spread of biased reports, inaccurate information, and harmful content. Therefore, effective anti-privacy attack methods for large models are crucial for the secure output of large models.

[0003] In the prior art, the defense strategies for large models mainly focus on instant rewriting or detection mechanisms, but the effectiveness of these methods is seriously challenged when facing complex and changeable privacy attacks. Privacy attacks can bypass existing alignment mechanisms through carefully designed prompts and induce LLM to output unsafe or harmful content. However, instant rewriting and detection mechanisms often fail to fully cover all possible attack modes, resulting in limited defense effects. Some defense strategies focus on output control and detection, which rely on the reasoning process of LLM and are therefore computationally expensive. They attempt to prevent the generation of unsafe content by controlling the generation process of the decoder or detecting the final output after LLM generates output. However, a significant disadvantage of this approach is that they can only intervene after the reasoning process, which not only increases the computational burden, but may also fail to prevent potential harm that has already occurred in time. At the same time, current defense methods also face the problems of limited defense range and interruption of benign prompts. For example, some rewriting or interpretation strategies may not be able to effectively distinguish between malicious and benign prompts, resulting in interference with normal and legitimate input requests while defending against privacy attacks. Such false positives and false blocking not only reduce the user experience, but may also affect the reliability and stability of LLM in practical applications.

[0004] Based on this, this specification provides a privacy attack prevention method and device using hidden state filtering. Summary of the invention

[0005] The embodiments of this specification provide a privacy attack prevention method and device using hidden state filtering, which are used to solve the following technical problems: In the prior art, the fusion of multimodal data is generally processed using a machine learning method. This method generally uses a shallow model and relies on a large amount of labeled data for supervised or semi-supervised training. Therefore, it usually requires a lot of preprocessing and manual intervention, and cannot generate missing data.

[0006] To solve the above technical problems, the embodiments of this specification are implemented as follows:

[0007] The embodiment of this specification provides a privacy attack prevention method using hidden state filtering, including:

[0008] Get pending requests;

[0009] Based on the pending request, obtain the vector t of k tags of the final decoder of the large language model k The concatenation vector T k ;

[0010] The concatenated vector T k Inputting into a pre-trained lightweight classifier to obtain a harmfulness score, the pre-trained lightweight classifier is a multi-layer perceptron, the multi-layer perceptron includes: an input layer, a hidden layer and an output layer, the hidden layer includes: a first hidden layer, a second hidden layer and a third hidden layer;

[0011] If the harmfulness score is greater than or equal to a preset threshold, the large language model refuses to generate a response to the pending request; if the harmfulness score is less than the preset threshold, the large language model responds to the pending request.

[0012] The embodiment of this specification also provides an anti-privacy attack device using hidden state filtering, including:

[0013] Request module, obtains pending requests;

[0014] The cascade vector acquisition module obtains the k-tagged vectors t of the final decoder of the large language model based on the pending request. k The concatenation vector T k ;

[0015] The hazard score calculation module converts the cascade vector T k Inputting into a pre-trained lightweight classifier to obtain a harmfulness score, the pre-trained lightweight classifier is a multi-layer perceptron, the multi-layer perceptron includes: an input layer, a hidden layer and an output layer, the hidden layer includes: a first hidden layer, a second hidden layer and a third hidden layer;

[0016] The response module is configured to cause the large language model to refuse to generate a response to the pending request if the harmfulness score is greater than or equal to a preset threshold, and to cause the large language model to respond to the pending request if the harmfulness score is less than the preset threshold.

[0017] The anti-privacy attack method using hidden state filtering provided in the embodiment of this specification uses the hidden state representation of the large language model to distinguish different types of queries, and converts the k-labeled vectors t of the final decoder of the large language model into k The concatenation vector T kAs input, a preset lightweight classifier is used to achieve effective attack detection without significantly increasing the computational burden, thereby ensuring adaptation to different large language models and changing attack strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 A flowchart of a privacy attack prevention method using hidden state filtering provided in an embodiment of this specification;

[0020] Figure 2 A flowchart of a privacy attack prevention method using hidden state filtering provided in an embodiment of this specification;

[0021] Figure 3 A framework diagram of a privacy attack prevention method using hidden state filtering provided in an embodiment of this specification;

[0022] Figure 4 A schematic diagram of an anti-privacy attack device using hidden state filtering provided in an embodiment of this specification;

[0023] in,

[0024] LLM stands for Large Language Model;

[0025] User Input represents user input;

[0026] DecoderLayer represents the decoder layer;

[0027] Hidden states represent hidden states;

[0028] last k tokens means the last k tokens;

[0029] HiddenState Filter represents a hidden state filter;

[0030] harmfulness a scoreα represents harmfulness score α;

[0031] threshold means threshold;

[0032] thresholdβ=0.5 means threshold β=0.5;

[0033] Safe response means safe response;

[0034] Output means output. DETAILED DESCRIPTION

[0035] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0036] In large language models, technical means such as FT (supervised fine-tuning) and RLHF (reinforcement learning based on human feedback) are widely used to strive to safely align these models and ensure that their behavior is consistent with the core values ​​and preferences of human society for security and reliability.

[0037] Although these alignment strategies have given the model the ability to judge the security of inputs to a certain extent, LLMs still face severe challenges from adversarial attacks. With the increasing popularity of LLMs in daily application scenarios such as chatbots and content generation, the task of ensuring that their output content conforms to human values ​​and effectively avoids harmful information has become increasingly urgent. However, complex and varied privacy attack methods continue to threaten the achievement of this goal, which aim to cleverly induce LLMs to produce unsafe outputs. It can be seen that existing technologies have significant disadvantages in defending against LLM privacy attacks, such as limited effectiveness, high computational cost, and limited defense range.

[0038] Based on this, this application aims to propose a privacy-resistant attack method that is pre-inferential, efficient, and widely applicable to ensure the security and reliability of LLM in daily applications such as chatbots and content generation.

[0039] Figure 1 The flowchart of a privacy attack prevention method using hidden state filtering provided in the embodiment of this specification is as follows. From the program perspective, the execution subject of the process can be a program installed on an application server or application terminal. It can be understood that the method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Figure 1 As shown, the method includes:

[0040] Step S101: Obtain pending requests.

[0041] In the embodiments of this specification, the request to be processed is a request from a user to query a large language model based on a terminal device. Specifically, the terminal device is any computing device with computing capabilities, such as various computer devices / systems, terminal devices, servers, etc. The terminal device may involve any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. Servers include but are not limited to mainframes, edge computing nodes, computing devices in cloud environments, and the like.

[0042] In the embodiment of this specification, the request to be processed is a request sent by a user to a large language model.

[0043] Step S103: Based on the pending request, obtain the vectors t of k tags of the final decoder of the large language model k The concatenation vector T k .

[0044] Large Language Model (LLM) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, and dialogue, and are an important path to artificial intelligence.

[0045] When using a large language model for querying, harmful or inappropriate content may be generated. Even after the large language model has been aligned, it may still be subject to adversarial attacks, such as adding specific meaningless tokens to cause the large language model to generate harmful content. This is the case that the embodiments of this specification are aimed at.

[0046] Therefore, in the embodiment of this specification, the large language model is an alignment model. The field involved in the alignment model can be external alignment, internal alignment or interpretability alignment. Further, the method of external alignment, internal alignment or interpretability alignment is not specifically limited here.

[0047] In the embodiments of this specification, the large language model is a decoder model, or an encoder-decoder model. Specifically, the large language model can be ChatGPT, and / or Llama, and / or Mistral. Specifically, the specific type of the large language model does not constitute a limitation of the present application.

[0048] In a large language model, the hidden states intuitively collect all the information about the model’s understanding of the query and its response strategy. These hidden states are also projected through the language modeling head (linear mapping) to predict the next token, which means that they exhibit a linear structure in the corresponding representation space (as assumed by PCA). Therefore, through the data of the hidden states, it is possible to extract the most important features related to the harmfulness of the query.

[0049] In the embodiments of this specification, the large language model is a model to be defended, that is, a large language model that needs to be protected from privacy attacks.

[0050] To facilitate understanding of the method provided in the embodiments of this specification, a large language model is taken as an example, in which the language model has a parameter level of 7B, specifically Llama2-7B-chat.

[0051] In the embodiment of this specification, based on the pending request, the vector t of k tags of the final decoder of the large language model is obtained. k The concatenation vector T k , including:

[0052] Inputting the pending request into the large language model and performing a forward pass to generate a hidden state corresponding to the pending request;

[0053] Based on the last decoder layer of the large language model, obtain the vector t of k tokens corresponding to the last k tokens from the hidden state corresponding to the pending request k ;

[0054] The vector t of the k labels k Connect in the original order and fill zero vectors between every two vectors to form the concatenated vector T k .

[0055] In the embodiment of this specification, the vector t of k labels k ∈R n×k , R n×k represents a matrix of n*k dimensions. Let t1, t2, .., t k is the vector corresponding to the last k tokens, where t i ∈R n .

[0056] In the embodiment of this specification, the vector T k Defined as:

[0057] T k =[t k ,0,t2,0,...,0,t1],(1)

[0058] Among them, 0∈Rn , represents a zero vector of length n.

[0059] In the embodiment of this specification, the vector t of the k labels k The k in depends on the output dimension of the hidden state of the large language model.

[0060] Continuing with the previous example, when the large language model is Llama2-7B-chat, the value of k is k∈{1,2,…,8}.

[0061] As a hyperparameter, the value of k will change with different large language models.

[0062] Determining the hyperparameter k can help capture sufficient contextual information on the one hand, and maintain computational efficiency on the other hand. Therefore, the determination of the hyperparameter k is of great significance.

[0063] In the embodiment of this specification, the model to be defended is a model that needs to be defended, specifically, a large language model that needs to be defended.

[0064] In the embodiment of this specification, the vector t of k tags of the final decoder of the large language model is obtained. k , is obtained by a marker. In one embodiment, the marker is a Llama2-7B-chat marker. Of course, the marker may also be other markers, and the specific type of the marker does not constitute a limitation of the present application.

[0065] Step S105: The cascade vector T k The data is input into a pre-trained lightweight classifier to obtain a harmfulness score. The pre-trained lightweight classifier is a multi-layer perceptron, which includes an input layer, a hidden layer and an output layer. The hidden layer includes a first hidden layer, a second hidden layer and a third hidden layer.

[0066] Multilayer Perceptron (MLP) is a feedforward artificial neural network model that maps multiple input data sets to a single output data set.

[0067] In a multi-layer perceptron, the input layer is used to receive raw data features as input, and each input neuron corresponds to one feature. In the embodiment of this specification, the input of the input layer is a cascade vector T k .

[0068] The hidden layer is located between the input layer and the output layer, and the hidden layer can capture the mutual relationship between the input layer data. In the embodiment of this specification, the hidden layer includes a first hidden layer, a second hidden layer and a third hidden layer. Among them, the number of nodes in the first hidden layer is 1024, the number of nodes in the second hidden layer is 64, and the number of nodes in the third hidden layer is 1.

[0069] In the embodiment of this specification, the loss function of the pre-trained lightweight classifier is a binary cross entropy loss function;

[0070] The expression of the binary cross entropy loss function is:

[0071] L k (θ)=-llog(σ(f k (T k )))-(1-l)log(1-σ(f k (T k )))

[0072] in,

[0073] L k (θ) represents the binary cross entropy loss function;

[0074] l∈{0,1} is a binary label indicating whether the query is safe;

[0075] σ represents the sigmoid function;

[0076] f k Represents a function that reduces the dimension of a matrix of n*k dimensions to a scalar;

[0077] T k Represents a concatenated vector.

[0078] In the embodiment of the present specification, the activation function of the hidden layer is the ReLU activation function, and the activation function of the output layer is the sigmoid function.

[0079] In the embodiment of this specification, the hidden layer is regularized by dropout to prevent overfitting;

[0080] The dropout regularization expression is:

[0081]

[0082] in,

[0083] f k Represents a function that reduces the dimension of a matrix of n*k dimensions to a scalar;

[0084] R n×k Represents a matrix of n*k dimensions;

[0085] R represents the real number space;

[0086] w k ∈R n×k and b k ∈R are the parameters to be fitted by the logistic regression model;

[0087] represents the hidden layer weight;

[0088] b k Indicates the offset value.

[0089] In one embodiment of the present specification, during dropout regularization, the dropout ratio is preferably 0.2.

[0090] In the embodiment of the present specification, the first hidden layer reduces the input dimension to 1024 dimensions, the second hidden layer reduces the 1024 dimensions to 64 dimensions, and the third hidden layer reduces the 64 dimensions to 1 dimension.

[0091] Harmful query data refers to data that does not conform to human values, while benign query data refers to data that conforms to human values. For example, How to make a bomb? belongs to harmful query data, while How to make a cake? belongs to benign query data. In an embodiment of the present specification, a pre-trained lightweight classifier constructs a training sample set with harmful query data and benign query data during the training process, and the training sample set may be data from a large language model or data from multiple large language models. When constructing a training sample set, harmful query data and benign query data should meet a certain ratio, for example, harmful query data: benign query data = 1:2.

[0092] It should be noted that if the model alignment of multiple large language models is inconsistent, a preset proportion of samples are randomly selected from the constructed training sample set as a new training sample set. The selection of the preset proportion can be determined according to the specific business scenario. In one embodiment of this specification, the preset proportion is 10%.

[0093] In one embodiment of the present specification, Llama2-7B-chat and Mistral-instrct-v0.2 are used as example models, 3,000 samples are collected from UltraSafety and PKU SafeRLHF prompt datasets as harmful query datasets, and 6,000 sample datasets are collected as benign query datasets. For the mistral-instrct-v0.2 model with inconsistent model alignment, 750 queries are extracted from the harmful and benign datasets we constructed and attack paradigms are added. The queries and attack paradigms are used as datasets for model training to perform model training.

[0094] In this embodiment of the specification, the vector t of the k tags of the final decoder (i.e., the last decoder) of the large language model is k The concatenation vector T k As input, it is fed into a pre-trained lightweight classifier to obtain a harmfulness score, thereby identifying and rejecting potential attacks. k The concatenation vector T k It is obtained through the hidden state of the final decoder. Therefore, the method provided in the embodiment of this specification can achieve the purpose of preventing privacy attacks by identifying attack patterns in the hidden state representation space of the large language model. Therefore, the pre-trained lightweight classifier is also called a hidden state filter (Hidden State Filtering, HSF).

[0095] The pre-trained lightweight classifier provided in the embodiments of this specification can obtain a harmfulness score, and thus determine the response of the large language model to the request to be processed based on the harmfulness score.

[0096] Step S107: If the harmfulness score is greater than or equal to a preset threshold, the large language model refuses to generate a response to the pending request; if the harmfulness score is less than the preset threshold, the large language model responds to the pending request.

[0097] The harmfulness score is an evaluation result of the possible harmfulness of the pending request, or the evaluation result of the potential harmfulness of the pending request. According to the harmfulness score, harmful attacks of the pending request can be identified and privacy attacks can be prevented. Therefore, the harmfulness score is of great significance.

[0098] In the embodiment of this specification, the harmfulness score is a value between 0 and 1, and the preset threshold is 0.5.

[0099] It should be noted that, in the embodiment of this specification, the vector t of the k labels of the final decoder k The concatenation vector T kIt is obtained from the large language model and is strictly dependent on the large language model. Therefore, the method provided in the embodiment of this specification is aimed at a specific large language model, and different large language models need to be trained separately. It is precisely because of this that the anti-privacy attack method provided in the embodiment of this specification has a more accurate result.

[0100] Figure 2 This is a flowchart of a privacy attack prevention method using hidden state filtering provided in an embodiment of this specification. Figure 2 As shown, the user terminal sends a data processing request (i.e., a pending request) to the server where the large language model is located. After the large language model obtains the pending request, it obtains the cascade vector T k As the input of the pre-trained lightweight encoder, a harmfulness score α is obtained. If the harmfulness score α is greater than or equal to the preset threshold β, the server where the large language model is located rejects the pending request; if the harmfulness score α is less than the preset threshold β, the server where the large language model is located responds to the pending request and outputs it.

[0101] In the embodiments of this specification, the user terminal can be a smart phone, a laptop, a tablet computer, etc. The embodiments of this application do not specifically limit the product type of the user terminal. The user terminal is installed with a chat software, and the specific form of the chat software is not specifically limited here.

[0102] The server is the background server of the large language model. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. The embodiment of the present application does not specifically limit this. The large language model can be deployed directly on the server or on other servers. The server can call the large language model by interacting with other servers.

[0103] In order to further understand the anti-privacy attack method using hidden state filtering provided by the present embodiment, the anti-privacy attack method using hidden state filtering will be described below in conjunction with the framework of the anti-privacy attack method using hidden state filtering.

[0104] Figure 3 This is a framework diagram of a privacy attack prevention method using hidden state filtering provided in an embodiment of this specification. Figure 3 As shown, after the large language model receives the pending request, the large language model forwards the hidden state from the final decoder to the pre-trained HSF, uses the sigmoid function to calculate the harmfulness score α, and compares the harmfulness score α with the preset threshold to determine the response of the large language model, ensuring that the pending request can be output safely and reliably.

[0105] The anti-privacy attack method using hidden state filtering provided in the embodiment of this specification uses the hidden state representation of the large language model to distinguish different types of queries, and converts the k-labeled vectors t of the final decoder of the large language model into k The concatenation vector T k As input, a preset lightweight classifier is used to achieve effective attack detection without significantly increasing the computational burden, thereby ensuring adaptation to different large language models and changing attack strategies.

[0106] The above content describes in detail a privacy attack prevention method using hidden state filtering. Correspondingly, this specification also provides a privacy attack prevention device using hidden state filtering, such as Figure 4 shown. Figure 4 A schematic diagram of an anti-privacy attack device using hidden state filtering provided in an embodiment of this specification, the anti-privacy attack device includes:

[0107] Request module 401, obtaining pending requests;

[0108] The cascade vector acquisition module 403 acquires the vectors t of the k tags of the final decoder of the large language model based on the pending request. k The concatenation vector T k ;

[0109] The criticality score calculation module 405 converts the cascade vector T k Inputting into a pre-trained lightweight classifier to obtain a harmfulness score, the pre-trained lightweight classifier is a multi-layer perceptron, the multi-layer perceptron includes: an input layer, a hidden layer and an output layer, the hidden layer includes: a first hidden layer, a second hidden layer and a third hidden layer;

[0110] The response module 407 may be configured to reject the large language model from generating a response to the pending request if the harmfulness score is greater than or equal to a preset threshold, and to respond to the pending request if the harmfulness score is less than the preset threshold.

[0111] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0112] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, electronic device, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0113] The apparatus, electronic device, non-volatile computer storage medium and method provided in the embodiments of this specification correspond to each other, and therefore, the apparatus, electronic device and non-volatile computer storage medium also have similar beneficial technical effects as the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding apparatus, electronic device and non-volatile computer storage medium will not be repeated here.

[0114] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0115] The specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0116] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0117] The above is only an embodiment of this specification and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A privacy attack prevention method using hidden state filtering, characterized in that: The anti-privacy attack method comprises: Get pending requests; Based on the pending request, obtain the vector t of k tags of the final decoder of the large language model k The concatenation vector T k ; The concatenated vector T k Inputting into a pre-trained lightweight classifier to obtain a harmfulness score, the pre-trained lightweight classifier is a multi-layer perceptron, the multi-layer perceptron includes: an input layer, a hidden layer and an output layer, the hidden layer includes: a first hidden layer, a second hidden layer and a third hidden layer; If the harmfulness score is greater than or equal to a preset threshold, the large language model refuses to generate a response to the pending request; if the harmfulness score is less than the preset threshold, the large language model responds to the pending request.

2. The privacy attack prevention method according to claim 1, characterized in that: Based on the pending request, obtaining the vector t of k tags of the final decoder of the large language model k The concatenation vector T k , specifically including: Inputting the pending request into the large language model and performing a forward pass to generate a hidden state corresponding to the pending request; Based on the last decoder layer of the large language model, obtain the vector t of k tokens corresponding to the last k tokens from the hidden state corresponding to the pending request k ; The vector t of the k labels k Connect in the original order and fill zero vectors between every two vectors to form the concatenated vector T k .

3. The privacy attack prevention method according to claim 1, characterized in that: The vector t of the k labels k The k in depends on the output dimension of the hidden state of the large language model.

4. The privacy attack prevention method according to claim 1, characterized in that: The loss function of the pre-trained lightweight classifier is a binary cross entropy loss function; The expression of the binary cross entropy loss function is: L k (θ)=-llog(σ(f k (T k )))-(1-l)log(1-σ(f k (T k ))) in, L k (θ) represents the binary cross entropy loss function; l∈{0,1} is a binary label indicating whether the query is safe; σ represents the sigmoid function; f k Represents a function that reduces the dimension of a matrix of n*k dimensions to a scalar; T k Represents a concatenated vector.

5. The privacy attack prevention method according to claim 1, characterized in that: The activation function of the hidden layer is the ReLU activation function, and the activation function of the output layer is the sigmoid function.

6. The privacy attack prevention method according to claim 1, characterized in that: The hidden layer is regularized by dropout to prevent overfitting; The dropout regularization expression is: in, f k Represents a function that reduces the dimension of a matrix of n*k dimensions to a scalar; R n×k Represents a matrix of n*k dimensions; R represents the real number space; w k ∈R n×k and b k ∈R are the parameters to be fitted by the logistic regression model; represents the hidden layer weight; b k Indicates the offset value.

7. The privacy attack prevention method according to claim 1, characterized in that: The first hidden layer reduces the input dimension to 1024 dimensions, the second hidden layer reduces the 1024 dimensions to 64 dimensions, and the third hidden layer reduces the 64 dimensions to 1 dimension.

8. The privacy attack prevention method according to claim 1, characterized in that: The large language model is an alignment model.

9. The privacy attack prevention method according to claim 1, characterized in that: The harmfulness score is a value between 0 and 1, and the preset threshold is 0.

5.

10. An anti-privacy attack device using hidden state filtering, characterized in that: The privacy attack prevention device comprises: Request module, obtains pending requests; The cascade vector acquisition module obtains the k-tagged vectors t of the final decoder of the large language model based on the pending request. k The concatenation vector T k ; The hazard score calculation module converts the cascade vector T k Inputting into a pre-trained lightweight classifier to obtain a harmfulness score, the pre-trained lightweight classifier is a multi-layer perceptron, the multi-layer perceptron includes: an input layer, a hidden layer and an output layer, the hidden layer includes: a first hidden layer, a second hidden layer and a third hidden layer; The response module is configured to cause the large language model to refuse to generate a response to the pending request if the harmfulness score is greater than or equal to a preset threshold, and to cause the large language model to respond to the pending request if the harmfulness score is less than the preset threshold.

Images