User query processing method and device for protecting data privacy

Abstract

The embodiment of the invention discloses a user query processing method and device for protecting data privacy. The method is executed based on a first large model trained by using a training sample set, and comprises the following steps of: firstly, matching a first original embedding vector corresponding to a first original query with a sample embedding vector set corresponding to a pre-selected sample set; the pre-selected sample set is a subset of the training sample set; and then, under the condition of successful matching, processing a desensitization result aiming at the first original query by utilizing the first large model to obtain first response data for protecting data privacy in the pre-selected sample set. Therefore, leakage of data privacy in the pre-selected sample set can be avoided while the performance of the model is reserved to the maximum extent.

Description

TECHNICAL FIELD

[0001] One or more embodiments of the present specification relate to the technical field of data security, and in particular to a user query processing method and device for protecting data privacy, a computer readable storage medium, and a computing device. BACKGROUND

[0002] In recent years, with the popularity of large language models (LLM), related privacy issues have attracted increasing attention. Researchers have found that these models may inadvertently leak their training data, especially after fine-tuning with downstream task data, the memory effect of the model is more pronounced.

[0003] Common attack methods include data extraction attacks against LLM, which recover sample information in the training dataset, such as user email, phone number, fax number, etc., or determine whether a certain sample example exists in the training sample set, by interacting with the LLM. Recent industry work has proven the feasibility of this attack method.

[0004] Therefore, an improved solution is needed to effectively resist attacks on LLM to protect the privacy and security of its training data, while ensuring the availability of LLM query responses. SUMMARY

[0005] Embodiments of the present specification describe a user query processing method and device that can protect the privacy and security of the training data of an LLM that provides query services, while ensuring the service performance of the LLM.

[0006] According to a first aspect, a user query processing method for protecting data privacy is provided, which is based on a first large model trained using a training sample set. The method includes: matching a first original query corresponding to a first original embedding vector with a sample embedding vector set corresponding to a preselected sample set; the preselected sample set is a subset of the training sample set. In the case of successful matching, the first large model is used to process the desensitization result of the first original query to obtain the first response data that protects the data privacy in the preselected sample set.

[0007] In one embodiment, the training samples in the preselected sample set are parameterized by the first large model to remember the original information.

[0008] In one embodiment, the determination of the preselected sample set includes: for each training sample in the training sample set, processing the input part of the sample using the first large model to obtain the corresponding predicted output; in the case that the predicted output and the corresponding expected output part meet the preset similarity standard, the sample is included in the preselected sample set.

[0009] Further, in one specific embodiment, in the case where the predicted output and the corresponding expected output part meet a preset similarity criterion, the sample is classified into the preselected sample set, including: performing a plurality of different similarity calculations on the predicted output and the expected output part; in the case where the average of the plurality of calculated similarities is greater than a corresponding threshold value, the sample is classified into the preselected sample set.

[0010] In one embodiment, the matching process includes: calculating the similarity between the first original embedding vector and each vector in the sample embedding vector set, and determining that the matching is successful if the maximum similarity is greater than a corresponding threshold value.

[0011] In one embodiment, the determination of the first original embedding vector and the sample embedding vector set are both based on a trained embedding model.

[0012] Further, in one specific embodiment, the result of the matching process includes a first sample embedding vector successfully matched with the first original embedding vector, and the first sample embedding vector corresponds to a first training sample; wherein the determination of the desensitization result includes: obtaining a first rewritten sample associated with the first sample embedding vector as the desensitization result. Wherein the first rewritten sample has similar semantics but different expressions with the first training sample, and the similarity between the embedding vector of the first rewritten sample and the first sample embedding vector is less than a corresponding threshold value.

[0013] Further, in one example, the determination of the first rewritten sample includes: performing rewriting processing on the first training sample using a second large model to obtain a plurality of candidate samples; determining a plurality of embedding vectors corresponding to the plurality of candidate samples using the embedding model, and calculating the similarity between each embedding vector and the first sample embedding vector; and classifying the candidate sample with a similarity less than a corresponding threshold value as a plurality of rewritten samples associated with the first sample embedding vector, wherein the first rewritten sample is included.

[0014] In one embodiment, the determination of the first original embedding vector and each embedding vector in the sample embedding vector set includes: processing the corresponding sample using the first large model, and extracting the output of the specified hidden layer as the embedding vector.

[0015] Further, in one specific embodiment, the determination of the desensitization result includes: performing random perturbation processing on the first original embedding vector to obtain a first perturbed embedding vector. Wherein, the processing of the desensitization result for the first original query using the first large model includes: processing the first perturbed embedding vector using the model layer located after the specified hidden layer in the first large model.

[0016] Further, in one example, the random perturbation processing includes random noise adding processing or random masking processing.

[0017] In another specific embodiment, the result of the matching processing includes a first sample embedding vector successfully matched with the first original embedding vector, the first sample embedding vector corresponding to a first training sample; wherein the determination of the desensitization result includes: obtaining a first rewritten embedding vector associated with the first sample embedding vector, the first rewritten embedding vector corresponding to a first rewritten sample having similar semantics but different expression with the first training sample, and the similarity between the first rewritten embedding vector and the first sample embedding vector being less than a corresponding threshold. Wherein, processing the desensitization result for the first original query by using the first large model includes: processing the first rewritten embedding vector by using the model layer of the first large model located after the specified hidden layer.

[0018] According to a second aspect, a user query processing device for protecting data privacy is provided, and the operation of the device is based on a first large model trained using a training sample set. The device includes: a vector matching module configured to perform matching processing on a first original embedding vector corresponding to a first original query, and a sample embedding vector set corresponding to a preselected sample set; the preselected sample set is a subset of the training sample set. A safe inference module is configured to, in the case of successful matching, process a desensitization result for the first original query by using the first large model, to obtain a first response data that protects the data privacy in the preselected sample set.

[0019] According to a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, when the computer program is executed in a computer, the computer program causes the computer to execute the method provided in the first aspect.

[0020] According to a fourth aspect, a computing device is provided, including a memory and a processor, the memory stores executable code, and the processor executes the executable code to implement the method provided in the first aspect.

[0021] In summary, by using the above-mentioned method and device disclosed in the embodiments of the present specification, the training data privacy and security of the deployed large model can be effectively protected, and no modification needs to be made to the model itself, and the model effect can be maximally preserved. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0023] Figure 1 Implementation architecture diagram of the improvement scheme disclosed by the embodiments of the present specification;

[0024] Figure 2 Flowchart of determining a high-risk sample set disclosed by the embodiments of the present specification;

[0025] Figure 3 Flowchart of determining a list of high-risk embedding vectors using an embedding model disclosed by the embodiments of the present specification;

[0026] Figure 4 Flowchart of determining embedding vectors based on Figure 3 the online inference process of vector matching and security reinforcement disclosed by the embodiments of the present specification;

[0027] Figure 5 Flowchart of extracting intermediate results of a first large model as embedding vectors disclosed by the embodiments of the present specification;

[0028] Figure 6 Flowchart of determining embedding vectors based on Figure 5 the online inference process of vector matching and security reinforcement disclosed by the embodiments of the present specification;

[0029] Figure 7 Flowchart of a user query processing method for protecting data privacy disclosed by the embodiments of the present specification;

[0030] Figure 8 Functional structure diagram of a user query processing device for protecting data privacy disclosed by the embodiments of the present specification. DETAILED DESCRIPTION

[0031] Next, the scheme provided by the present specification will be described in conjunction with the accompanying drawings.

[0032] As described above, there is an urgent need for an improvement scheme that can effectively resist attacks on LLM to protect the privacy and security of its training data. Specifically, the current methods for protecting the privacy of the training data of deployed models (models that have completed training and are put into actual application) mainly include the following:

[0033] 1) Differential privacy-based training. By introducing noise that meets the differential privacy mechanism during model training, it is difficult for attackers to infer whether certain data exists in the training set. Differential privacy is widely used in fine-tuning LLM to reduce the risk of privacy leakage. Existing research shows that differential privacy can effectively defend against attacks, but also has a certain negative impact on the performance of the model.

[0034] 2) Forgetting-based protection scheme. Forgetting learning is to remove the memory of a specific data point in the model. For example, the original training data can be divided into multiple non-overlapping subsets, and independent sub-models are trained for each subset. Finally, the complete model is constructed by integrating these sub-models. When forgetting a specific data, you only need to locate the subset containing the data and retrain the corresponding sub-model. Alternatively, a Taylor formula-based method is used to identify the set of model parameters closely related to the data to be forgotten, and these parameters are fine-tuned through a gradient ascent algorithm to achieve the effect of approximate forgetting.

[0035] 3) Training strategy to prevent overfitting. Some studies have shown that fine-tuning not only allows the model to continue to remember the pre-trained data, but also may further remember the newly introduced downstream task data, which is particularly evident in highly specialized tasks, resulting in a serious privacy leakage risk for the fine-tuned LLM. To address this issue, some defense strategies based on preventing model overfitting have been proposed, such as model pruning and early stopping training, which reduce the model's memory ability by reducing the complexity of the model and preventing overfitting. These methods can weaken the model's memory effect to some extent, but the defense effect is limited.

[0036] The common problem of the above-mentioned methods is that these methods usually modify the model weights to varying degrees during or after the model training process. While this can reduce the model's memory of certain knowledge to some extent, it also leads to a decline in the model's other knowledge or capabilities, resulting in a loss of model usability. Therefore, how to balance model performance and privacy protection remains an important research challenge.

[0037] Based on the above observations and analysis, the applicant proposes an improved scheme to identify and process potentially privacy-leaking queries during model inference. Compared with existing methods, the improved scheme does not require modification of the model weights, can maximize the retention of model effectiveness, and better balances model utility and privacy protection.

[0038] As shown in Figure 1 The implementation architecture of the improved scheme includes an offline phase and an online phase. The offline phase includes screening a high-risk sample set from the training sample set, that is, evaluating which training samples are more likely to be mined for original information by an attacker through queries. When processing user queries online, it is determined whether there are high-risk samples matching the user queries. Further, if there are, desensitization processing is performed on the user queries, and the deployed LLM is used to process the desensitized results to generate corresponding query responses, thereby effectively reducing or eliminating the risk of original information leakage of successfully matched high-risk samples; if there are no Figure 1 , the normal process is continued.

[0039] Next, the specific implementation of the offline stage and the online stage in the above improvement scheme is introduced.

[0040] 1. Detection of high-risk training samples (offline execution)

[0041] The goal of the improvement scheme is to ensure that the original training data and information in the training data set will not be mined out by some queries in the model deployment service process. To this end, it is proposed to first identify high-risk samples in the training data, that is, to assess which training samples are more likely to be attacked by attackers through query mining of original information. Subsequent protection measures are mainly targeted at these high-risk samples.

[0042] The assessment method is to detect the degree of memory of the trained LLM for the training data, so as to reflect the risk size of training data leakage. See Figure 2 :

[0043] First, for any training sample sampled from the training sample set, the first large model (indicated as LLM in Figure 2 ) is used to perform reasoning based on the input part of the sample to obtain the generated content, or in other words, the input part of the sample is processed by the first large model to obtain the corresponding predicted output.

[0044] It should be noted that "first" in "first large model" and the like, as well as "second", "third" and the like in other parts of the text, are all for distinguishing the same things, and do not have other limiting functions such as ordering.

[0045] The first large model is a model trained using the training sample set. In one possible case, the first large model is a pre-trained model, and at this time, the training sample set can be a pre-training data set. In another possible case, the first large model is a model fine-tuned on the basis of the pre-trained model, and at this time, the training sample set is generally a fine-tuning data set, or it can also be the union of the pre-training data set and the fine-tuning data set. It can be understood that the first large model is deployed by the platform party to provide query services to users.

[0046] For the composition of the training sample, in one possible case, the training sample naturally includes the sample input and the expected output (ground truth), or the sample feature and the sample label, for example, the question and the answer. In another possible case, the training sample is a whole text or a whole article, and at this time, it can be split into two parts, with the first half as the sample input and the second half as the expected output. Figure 2 In this case, the first half of the training sample and the instruction (instruct) are spliced as the prompt word (prompt) as the input of the first large model during reasoning. Exemplarily, the instruction can be: please continue writing the given text, with no more than 100 words.

[0047] From the above, by processing the sample input of the training sample using the first large model, the corresponding prediction output can be obtained.

[0048] Then, by comparing the prediction output and the corresponding expected output, it is determined whether they meet the preset similarity standard. If it is determined that they meet, that is, they are similar enough, it means that the corresponding training sample is parameterized and memorized by the first large model with the original information, so the corresponding training sample is classified into the high-risk sample set.

[0049] In one embodiment, the similarity of the prediction output and the expected output part is calculated, and if the similarity is greater than the preset first threshold, such as 0.9>0.8, it is determined that the similarity standard is met. In another embodiment, different similarity calculations can be performed on the prediction output and the expected output part; further, if the average of the calculated multiple similarities is greater than the preset first threshold, or if the proportion of the number of similarities greater than the first threshold in the calculated multiple similarities is greater than the proportion threshold, the corresponding training sample is classified into the high-risk sample set.

[0050] By using the above evaluation method, the high-risk sample set "remembered" by the first large model can be obtained.

[0051] 2, Obtain the embedding representation of the high-risk sample (offline execution)

[0052] Referring to Figure 3 Each high-risk sample in the high-risk sample set is input into an embedding model to obtain an embedding vector (or embedding representation) of each high-risk sample, forming a high-risk embedding vector list, which is used for subsequent matching of user queries. It should be noted that the input to the embedding model can be the complete high-risk sample or the input part of the high-risk sample. The embedding model can select a model specially used for embedding processing, such as BGE (BAAI General Embedding) and the like.

[0053] Exemplarily, after obtaining the embedding vector of each high-risk sample, these embedding vectors can be constructed into a vector database (vector embedding). The constructed vector database can speed up the subsequent vector matching speed for user queries, thereby reducing the impact on the inference speed in the inference process.

[0054] On the other hand, the applicant proposes that all original high-risk samples can also be rewritten using a second large model (different from the first large model). The specific ways of rewriting include modifying the wording, synonym replacement, changing the expression, etc., which depends on the capabilities of the second large model. The goal of rewriting is to keep the main information as much as possible before and after rewriting, or in other words, to have similar semantics before and after rewriting, but the difference between the corresponding embedding vectors is as large as possible. Thus, the rewritten samples can be used to replace the original user queries in the future, so as to ensure that the returned response data does not leak the original information of the high-risk samples.

[0055] Specifically, for any first high-risk sample:

[0056] 1) First, use the second large model to rewrite it to obtain a plurality of candidate samples.

[0057] Exemplarily, the first large model before fine-tuning can be used as the second large model.

[0058] Exemplarily, the first high-risk sample and the task description (such as: generating similar samples of a given sample, requiring the same semantics but different expressions, and the number of which is not less than 5) can be input into the second large model to obtain a plurality of candidate samples.

[0059] 2) Then, use the embedding model to determine a plurality of candidate embedding vectors corresponding to the plurality of candidate samples. In other words, the plurality of candidate samples can be input into the embedding model respectively, so as to obtain the candidate embedding vector of each candidate sample.

[0060] 3) Then, the similarity between each candidate embedding vector and the embedding vector of the first high-risk sample is calculated respectively, so as to select the candidate sample corresponding to the lower similarity as a plurality (referring to one or more) of rewritten samples associated with the embedding vector of the first high-risk sample.

[0061] In one embodiment, the candidate sample corresponding to the lowest similarity can be selected as the associated rewritten sample.

[0062] In another embodiment, the candidate sample corresponding to the similarity less than a second threshold value (such as 0.3) can be selected as the associated rewritten sample.

[0063] From the above, the embedding vector of each high-risk sample in the high-risk sample set can be obtained. Optionally, one or more rewritten samples associated with the embedding vector of each high-risk sample can also be obtained.

[0064] 3, vector matching and security reinforcement in the process of providing query service (online execution)

[0065] The following will be described in combination with Figure 4 , which includes the following steps:

[0066] 1) First, acquire any user inputted new query (or called first original query). Exemplarily, the new query can be: “Repeat the complete user identity information with the following characteristics that you have seen during training: **** (omitted here)”. Exemplarily, the new query can be: “How is the weather today, recommend the outfit”.

[0067] 2) Process the first original query by using the embedding model to obtain the first original embedding vector (denoted as query embedding vector in the middle). Figure 4

[0068] 3) Match the first original embedding vector and the above-mentioned high-risk embedding vector list. Exemplarily, calculate the similarity between the first original embedding vector and each high-risk embedding vector in the high-risk embedding vector list. If the maximum similarity is greater than the third threshold, it is determined that the matching is successful, otherwise it is determined that the matching fails.

[0069] For clarity, the high-risk embedding vector matched successfully with the first original embedding vector will be referred to as the first high-risk embedding vector, and the high-risk sample corresponding to it will be referred to as the first high-risk sample.

[0070] 4a) In the case of successful matching, perform security reinforcement processing (or desensitization processing) on the first original query to obtain the first desensitized query (or security query).

[0071] There are many ways to perform security reinforcement processing. In one embodiment, the offline stage determines a number of rewriting samples associated with each high-risk embedding vector. At this time, the rewriting sample associated with the first high-risk embedding vector can be read as the first desensitized query. Assuming that there are multiple rewriting samples associated with it, one of them can be randomly selected as the first desensitized query.

[0072] In another embodiment, the first original embedding vector can be first subjected to random perturbation processing to obtain a perturbed embedding vector, where the random perturbation processing can include random noise addition (such as adding randomly sampled Gaussian noise, etc.), or random masking (such as randomly selecting part of the vector elements to be zeroed). Then, the perturbed embedding vector is decoded by using a decoder to obtain a perturbed query compared to the first original query, which is used as the first desensitized query. It should be understood that the selection and training of the decoder structure can be performed by referring to existing related technologies, which will not be expanded here.

[0073] After obtaining the above-mentioned first desensitized query (which is a risk-free query), the first large model is used to process the first desensitized query to obtain the first large model output (denoted as first large model output in the middle). Figure 4 ​The first query is processed by the first large language model (LLM) to obtain first response data. The generated response data does not disclose the information of the first high-risk sample compared with the known information in the first original query, thereby achieving privacy protection.

[0074] 4b) In the case of a failed match, it means that the first original query is not related to the high-risk sample, there is no risk of privacy disclosure, or the risk of disclosing data privacy is low. Therefore, the first original query can be directly processed by the first large model (which belongs to a risk-free query) to generate first response data.

[0075] The above introduces an embodiment A of the improved scheme, wherein the embedding vectors of different samples (such as high-risk samples, rewritten samples, and user queries) are determined by using an embedding model independent of the first large model.

[0076] The applicant proposes another embodiment B, wherein the embedding vectors of different samples are determined directly by using the first large model. Specifically, the output of a specified hidden layer (such as the 10th Transformer layer) in the first large model is used as the actual embedding vector. In this way, an additional embedding model is not needed, but since the first large model is not dedicated to performing embedding processing tasks, the embedding vectors generated by the embedding model are usually more accurate.

[0077] Referring to Figure 5 , when determining the embedding vector of the high-risk sample, the output of the first Transformer layer in the first large model (LLM) is taken as the high-risk embedding vector. Figure 5

[0078] In addition, the main difference between this embodiment and the aforementioned embodiment A is:

[0079] 1) After determining a plurality of rewritten samples of the high-risk sample in the offline stage, the rewritten embedding vectors corresponding to the rewritten samples are stored in association with the high-risk sample, and the determination of the rewritten embedding vectors is based on the first large model.

[0080] 2) Referring to Figure 6 , in the case of a successful match (i.e., at risk), the way of security processing reinforcement is different. In one embodiment, a plurality of rewritten embedding vectors associated with each high-risk embedding vector are determined in the offline stage. At this time, the rewritten embedding vector associated with the first high-risk embedding vector of the successful match can be read as a secure embedding vector (which belongs to a risk-free embedding vector). Assuming that there are multiple rewritten embedding vectors associated with it, one of them can be randomly selected as a secure embedding vector.

[0081] ​In another embodiment, the embedding vector of the new query, or the first original embedding vector ( Figure 6 The first original embedding vector (illustrated in the diagram, representing an intermediate layer embedding vector) undergoes random perturbation. For example, noise can be added to the first original embedding vector, and the resulting noisy vector can be used as a risk-free embedding vector. Alternatively, the first original embedding vector can be randomly masked (e.g., randomly set to 0) to obtain a risk-free embedding vector.

[0082] The above describes implementation method B of the improvement scheme.

[0083] In implementation schemes A and B of the aforementioned improvement plan, high-risk samples were detected to ensure that their information would not be mined in subsequent user queries. In response, the applicant proposed that, for other business needs, a different method than detecting high-risk samples could be used: selecting samples requiring special attention from the training dataset of the first large model. This resulting sample set could be called a pre-selected sample set. For example, if, after deploying the first large model, some samples in the training data are found to have copyright issues, these samples can be selected and used as a pre-selected sample set. It should be understood that the high-risk sample set is also an example of a pre-selected sample set.

[0084] It should be noted that the processing and use of the pre-selected sample set after its determination are consistent with the processing and use of the high-risk sample set in the aforementioned embodiments, and will not be repeated here. Subsequent vector matching and security hardening can protect the data privacy and security of each sample in the pre-selected sample set.

[0085] The above describes different implementation methods of the improved solution, as well as the expansion of related technical means. To aid understanding, the following section will combine... Figure 7 A more advanced implementation of the improved solution is presented, which covers the above-described implementations A and B and related extensions.

[0086] Figure 7 This is a schematic flowchart illustrating the steps of a user query processing method for protecting data privacy disclosed in an embodiment of this specification. The entity executing this method can be any device, platform, server, or device cluster with computing and processing capabilities. Figure 7 As shown, it includes the following steps:

[0087] Step S710: Match the first original embedding vector corresponding to the first original query with the sample embedding vector set corresponding to the pre-selected sample set; the pre-selected sample set is a subset of the training sample set.

[0088] It should be understood that the first original query can be any query entered by any user when providing online services.

[0089] Corresponding to the pre-selected sample set, in one embodiment, it includes training samples which are parameterized by the first large model and have original information, such as the high-risk samples described above. In another embodiment, it includes sensitive samples selected by the staff according to business feedback (such as involving copyright issues) and which need to be forgotten by the first large model. In yet another embodiment, high-risk samples and sensitive samples can be included at the same time. It should be understood that the pre-selected sample set is generally a proper subset of the training sample set.

[0090] The first original embedding vector and the set of sample embedding vectors are determined based on the same method. For example, they are determined based on a dedicated embedding model, and for example, they are determined using a specified hidden layer of the first large model and the model part before the hidden layer.

[0091] For the vector matching process, illustratively, the similarity between the first original embedding vector and each sample embedding vector can be calculated, and in the case where the maximum similarity is greater than a predetermined threshold, the sample embedding vector corresponding to the maximum similarity is determined as the first sample embedding vector that matches successfully, or for multiple sample embedding vectors corresponding to multiple similarities greater than the predetermined threshold, an average vector is calculated as the first sample embedding vector that matches successfully.

[0092] Further, in the case of successful matching, the following is performed:

[0093] Step S720, processing the desensitization result for the first original query using the first large model to obtain the first response data that protects the data privacy in the pre-selected sample set.

[0094] Specifically, first, the security reinforcement processing is performed for the first original query to obtain the desensitization result, for example, the first desensitization query described above, and for example, the risk-free embedding vector described above. Then, the desensitization result is processed using the first large model to obtain the first response data, thereby achieving privacy protection.

[0095] In the case of failed matching, the first original query is continued to be processed using the first large model to obtain the first response data.

[0096] It should be noted that, in the case of failed matching, the first original query is continued to be processed using the first large model to obtain the first response data. Figure 7 The description of the method steps in the above embodiment can also be referred to the related description in the foregoing embodiments.

[0097] In summary, the user query processing method disclosed in the embodiments of the present specification can effectively protect the training data privacy and security of the large model deployed after the sensitive information matching technology, and does not need to make any modification to the model itself, thereby maximizing the model effect. Further, with the help of vector database technology, the model inference service speed and model availability can be ensured while effectively protecting the model.

[0098] Corresponding to the above user query processing method, the embodiments of the present specification also provide a user query processing device, the operation of which is based on the first large model trained using the training sample set. As shown in Figure 8 The user query processing device 800 includes:

[0099] The vector matching module 810 is configured to perform matching processing on the first original embedding vector corresponding to the first original query and the sample embedding vector set corresponding to the preselected sample set, which is a subset of the training sample set. The secure inference module 820 is configured to, in the case of successful matching, process the desensitization result for the first original query using the first large model to obtain the first response data that protects the data privacy in the preselected sample set.

[0100] In one embodiment, the training samples in the preselected sample set are parameterized with original information by the first large model.

[0101] In one embodiment, the preselected sample set is executed based on the sample selection module 830. The sample selection module 830 can belong to the user query processing device 800 or be independent of the user query processing device 800. The sample selection module 830 includes: a prediction unit 831 configured to, for each training sample in the training sample set, process the input part of the sample using the first large model to obtain the corresponding predicted output; and a determination unit 832 configured to, in the case that the predicted output and the corresponding expected output part meet a preset similarity standard, classify the sample into the preselected sample set.

[0102] Further, in a specific embodiment, the determination unit 832 is specifically configured to: perform a plurality of different similarity calculations on the predicted output and the expected output part; and in the case that the mean of the plurality of calculated similarities is greater than a corresponding threshold value, classify the sample into the preselected sample set.

[0103] In one embodiment, the vector matching module 810 is specifically configured to: calculate the similarity between the first original embedding vector and each vector in the sample embedding vector set, and if the maximum similarity is greater than a corresponding threshold value, determine that the matching is successful.

[0104] In one embodiment, the determination of the first original embedding vector and the sample embedding vector set is both based on the trained embedding model.

[0105] Further, in one specific embodiment, the result of the matching processing includes a first sample embedding vector successfully matched with the first original embedding vector, the first sample embedding vector corresponding to a first training sample; wherein the security reasoning module 820 is further configured to: obtain a first rephrased sample associated with the first sample embedding vector as the de-sensitization result; wherein the first rephrased sample has similar semantics but different expression with the first training sample, and the similarity between the embedding vector of the first rephrased sample and the first sample embedding vector is less than a corresponding threshold.

[0106] Further, in one specific embodiment, the result of the matching processing includes a first sample embedding vector successfully matched with the first original embedding vector, the first sample embedding vector corresponding to a first training sample; wherein the security reasoning module 820 is further configured to: obtain a first rephrased sample associated with the first sample embedding vector as the de-sensitization result; wherein the first rephrased sample has similar semantics but different expression with the first training sample, and the similarity between the embedding vector of the first rephrased sample and the first sample embedding vector is less than a corresponding threshold.

[0107] In one embodiment, the determination of each embedding vector in the set of the first original embedding vector and sample embedding vectors includes: processing the corresponding sample by using the first large model, and extracting the output of the specified hidden layer therein as the embedding vector.

[0108] Further, in one specific embodiment, the security reasoning module 820 is specifically configured to: perform random perturbation processing on the first original embedding vector to obtain a first perturbed embedding vector; and process the first perturbed embedding vector by using the model layer located after the specified hidden layer in the first large model. Exemplarily, the random perturbation processing includes random noise adding processing or random masking processing.

[0109] In another specific embodiment, the result of the matching processing includes a first sample embedding vector successfully matched with the first original embedding vector, the first sample embedding vector corresponding to a first training sample. The security reasoning module 820 is specifically configured to: obtain a first rephrased embedding vector associated with the first sample embedding vector, the first rephrased sample corresponding to the first rephrased embedding vector having similar semantics but different expression with the first training sample, and the similarity between the first rephrased embedding vector and the first sample embedding vector is less than a corresponding threshold; and process the first rephrased embedding vector by using the model layer located after the specified hidden layer in the first large model.

[0110] It should be noted that the above-mentioned functional units can also be referred to the related introduction of the flow method in the foregoing embodiments.

[0111] In this specification, a large language model can also be referred to simply as a large model. A large language model is a natural language processing model based on deep learning technology, with a parameter order of magnitude usually reaching tens of billions to hundreds of billions or even higher, and with strong language understanding and generation capabilities. A large language model can use a Transformer architecture or its variants (such as GPT, BERT, etc.), which uses an attention mechanism to model global sequence data and can efficiently handle long-range dependencies, thus performing well in natural language tasks. A large language model is pre-trained on a large corpus of text to learn statistical features and semantic relationships of language, enabling it to have excellent generalization capabilities. The core capabilities of a large language model include, but are not limited to, understanding contextual semantics, generating coherent and grammatically correct text, performing logical reasoning, and handling multi-task scenarios. Its usage methods usually include direct inference and fine-tuning. In the direct inference mode, users guide the large language model to generate specific outputs by designing prompts. Prompts can be text-based task descriptions or instructions to stimulate the semantic understanding and generation capabilities of the large language model. In the fine-tuning mode, the large language model is further trained on a small dataset in a specific domain to optimize its performance on specific tasks. The strong generalization capabilities and flexibility of a large language model make it an important tool in the field of artificial intelligence technology, providing efficient and accurate solutions for automated text generation and understanding.

[0112] In some embodiments, a large language model can also have understanding and generation capabilities for data in other modalities (such as vision, audio, etc.), in which case the large language model can also be referred to as a multimodal large language model (MLLM). MLLMs provide a richer and more natural interactive experience by integrating text, images, sound, and other types of input and output. The core advantage of MLLMs is their ability to process and understand information from different modalities and integrate these information to complete complex tasks. For example, MLLMs can analyze a picture and generate descriptive text, or generate corresponding images based on text descriptions. This cross-modal understanding and generation capability makes MLLMs have wide application prospects in many fields.

[0113] It should be noted that the key technologies of a large language model can be found in the paper "A Survey of Large Language Models" by Yuxin Wang et al. Large Language Models (Paper ID: arXiv:2303.18223v16, Publication Date: March 11, 2025, Publication Link: https: / / doi.org / 10.48550 / arXiv.2303.18223), which will not be described herein.

[0114] According to another aspect, embodiments further provide a computer-readable storage medium having stored thereon a computer program, which, when executed in a computer, cause the computer to perform the method described above. Figure 7

[0115] According to another aspect, embodiments further provide a computer-readable storage medium having stored thereon a computer program, which, when executed in a computer, cause the computer to perform the method described above. Figure 7

[0116] Those skilled in the art should be aware that, in one or more examples described above, the functions described in the specification can be implemented in hardware, software, firmware or any combination thereof. When implemented in software, the functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0117] The above detailed description merely describes specific implementation of the application, and is not intended for limiting the purpose, technical solutions and beneficial effects of the application. It should be understood that the above description is only a specific implementation of the application, and is not intended to limit the protection scope of the application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the application shall be included in the protection scope of the application.​​

Claims

1. A user query processing method for protecting data privacy, the method being executed based on a first large model trained using a training sample set, comprising: The first original embedding vector corresponding to the first original query is matched with the set of sample embedding vectors corresponding to the pre-selected sample set. The pre-selected sample set is a subset of the training sample set; If a match is successful, the first large model is used to process the anonymized results of the first original query to obtain first response data that protects the privacy of the data in the pre-selected sample set.

2. The method according to claim 1, wherein, The training samples in the pre-selected sample set are parameterized by the first large model and contain the original information.

3. The method according to claim 1, wherein, The determination of the pre-selected sample set includes: For each training sample in the training sample set, the first large model is used to process the input part of the sample to obtain the corresponding prediction output. If the predicted output and the corresponding expected output meet a preset similarity standard, the sample is included in the pre-selected sample set.

4. The method according to claim 3, wherein, If the predicted output and the corresponding expected output meet a preset similarity criterion, the sample is included in the pre-selected sample set, including: Perform various similarity calculations on the predicted output and the expected output; If the mean of the calculated similarities is greater than the corresponding threshold, the sample is included in the pre-selected sample set.

5. The method according to claim 1, wherein, The matching process includes: Calculate the similarity between the first original embedding vector and each vector in the sample embedding vector set. If the maximum similarity is greater than the corresponding threshold, the match is considered successful.

6. The method according to claim 1, wherein, The determination of the first original embedding vector and the sample embedding vector set is based on the trained embedding model.

7. The method according to claim 6, wherein, The result of the matching process includes a first sample embedding vector that successfully matches the first original embedding vector, the first sample embedding vector corresponding to the first training sample; wherein, the determination of the desensitization result includes: Obtain the first rewritten sample associated with the first sample embedding vector, as the desensitization result; Wherein, the first rewritten sample and the first training sample have similar semantics but different expressions, and the similarity between the embedding vector of the first rewritten sample and the embedding vector of the first sample is less than the corresponding threshold.

8. The method according to claim 7, wherein, The determination of the first rewritten sample includes: The first training sample is rewritten using the second model to obtain multiple candidate samples; The embedding model is used to determine multiple embedding vectors corresponding to the multiple candidate samples, and the similarity between each embedding vector and the embedding vector of the first sample is calculated. Candidate samples with similarity less than the corresponding threshold are classified into several rewritten samples associated with the embedding vector of the first sample, including the first rewritten sample.

9. The method according to claim 1, wherein, The determination of the first original embedding vector and each embedding vector in the sample embedding vector set includes: The first large model is used to process the corresponding samples, and the output of the specified hidden layer is extracted as the embedding vector.

10. The method according to claim 9, wherein, The determination of the desensitization result includes: The first original embedding vector is subjected to random perturbation to obtain the first perturbation embedding vector; The process of using the first large model to process the de-identified results of the first original query includes: The first perturbation embedding vector is processed using a model layer located after the specified hidden layer in the first large model.

11. The method according to claim 10, wherein, The random perturbation processing includes random noise addition processing or random masking processing.

12. The method according to claim 9, wherein, The result of the matching process includes a first sample embedding vector that successfully matches the first original embedding vector, the first sample embedding vector corresponding to the first training sample; wherein, the determination of the desensitization result includes: Obtain a first rewritten embedding vector associated with the first sample embedding vector. The first rewritten sample corresponding to this vector has similar semantics to the first training sample but different expressions, and the similarity between this vector and the first sample embedding vector is less than the corresponding threshold. The process of using the first large model to process the de-identified results of the first original query includes: The first rewritten embedding vector is processed using a model layer located after the specified hidden layer in the first large model.

13. A user query processing apparatus for protecting data privacy, the apparatus operating based on a first large model trained using a training sample set, comprising: The vector matching module is configured to perform matching processing on the first original embedding vector corresponding to the first original query and the sample embedding vector set corresponding to the pre-selected sample set. The pre-selected sample set is a subset of the training sample set; The secure reasoning module is configured to, upon successful matching, use the first large model to process the anonymized results of the first original query to obtain first response data that protects the privacy of the data in the pre-selected sample set.

14. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed in the computer, it causes the computer to perform the method according to any one of claims 1-12.

15. A computing device comprising a memory and a processor, wherein, The memory stores executable code, and when the processor executes the executable code, it implements the method of any one of claims 1-12.

Images