Method and system for resisting dense language retrieval model backdoor attack

Abstract

The invention belongs to the technical field of artificial intelligence model security, and particularly discloses a method and system for resisting a dense language retrieval model backdoor attack, and the method comprises the steps: calculating learning loss values of training samples, screening out the training samples with the highest loss value and the lowest loss value, forming two agent data sets, and continuously updating the two agent data sets; calculating the gradients of the two agent data sets to the model parameters, and predicting the optimal defense opportunity; at the best defense opportunity, the query of the proxy data set with the highest loss value is marked as a toxic sample, the query of the proxy data set with the lowest loss value is marked as a clean sample, and a toxic sample identifier is trained; afterwards, when the proxy data set is updated each time, the proxy data set is used for carrying out incremental training on the toxic sample identifier, and suspicious samples in the training samples are marked; performing gradient reverse optimization on the suspicious samples, and performing normal gradient descent optimization on the non-suspicious samples. By means of the method and device, the backdoor attack resisting capacity of the dense language retrieval model can be remarkably improved.

Description

Technical Field

[0001] This application belongs to the technical field of artificial intelligence model security, and more specifically, relates to a method and system for resisting backdoor attacks on dense language retrieval models. Background Art

[0002] In recent years, with the rapid development of natural language processing technology, retrieval models based on dense retrieval have gradually replaced traditional sparse retrieval models and have been widely used in fields such as information retrieval, question answering systems, and dialogue systems. In particular, retrieval-augmented generation technology, by combining large-scale pre-trained language models with retrieval models, has significantly improved the accuracy of information acquisition and the reliability of generated content. However, as the application scope of dense retrieval models expands, the security risks they face have gradually become prominent, and among them, backdoor attacks pose a particularly significant threat to retrieval models.

[0003] A backdoor attack is a type of attack that maliciously tampers with the behavior of a model. Attackers implant backdoors during the model training process, causing the model to exhibit abnormal behavior when faced with specific trigger samples, while still maintaining high performance on normal samples. Although a large number of studies have shown that deep neural networks are vulnerable to backdoor attacks in classification tasks, recent research has found that dense retrieval models also face such threats. Specifically, attackers can implant backdoors in retrieval models, causing them to return preset malicious content when processing specific queries, thereby undermining the credibility and security of retrieval results. Such attacks can lead to serious consequences such as the spread of false information, malicious guidance of user behavior, and data leakage, posing a great security risk to practical applications.

[0004] Currently, there are various methods for defending against backdoor attacks on classification models, such as anomaly detection, model detoxification, and training process intervention. However, these methods are difficult to directly apply to dense retrieval models, mainly because the backdoor attacks on retrieval models have the following unique challenges: 1). Convergence difference: The convergence speed of the backdoor task is usually slower than that of the normal task, making it difficult to directly apply traditional detection methods based on the analysis of training dynamics. 2). Feature space mixing: Toxic samples and clean samples are highly mixed in the feature space, making it difficult for defense methods based on outlier detection to effectively distinguish them.

[0005] Therefore, for backdoor attacks on dense language retrieval models, developing an effective defense method and system to improve the robustness and security of the model against malicious queries has become an urgent technical problem to be solved. Summary of the Invention

[0006] Aiming at the deficiencies of the prior art, the purpose of this application is to provide a method and system for resisting backdoor attacks on dense language retrieval models, aiming to solve the problem that dense language retrieval models may be implanted with the threat of backdoor attacks when fine-tuning using publicly available poisoned datasets, so as to improve the robustness and security of dense language retrieval models against malicious queries.

[0007] To achieve the above purpose, in the first aspect, this application provides a method for resisting backdoor attacks on dense language retrieval models, including: Calculate the learning loss values of training samples, screen out the training samples with the highest and lowest loss values to form two proxy datasets; during the training process, continuously update the two proxy datasets to ensure that they always contain the training samples with the largest and smallest loss values; the training samples include a query text and at least one answer text; Calculate the gradients of the two proxy datasets with respect to the model parameters, and predict the optimal defense timing based on the gradient cosine similarity; At the optimal defense timing, mark the queries in the proxy dataset with the highest loss value as toxic samples, mark the queries in the proxy dataset with the lowest loss value as clean samples, and use the marked queries to train the toxic sample identifier; after the optimal defense timing, each time the proxy dataset is updated, incrementally train the toxic sample identifier using the proxy dataset, and use the toxic sample identifier to mark the suspicious samples in the training samples; Perform gradient reverse optimization on the suspicious samples, and use normal gradient descent optimization for the non-suspicious samples.

[0008] In a possible implementation, continuously updating the two proxy datasets includes: For each training batch, calculate the learning loss values of the current batch of training data and the learning loss values of the proxy datasets, sort the loss values, and select the first K data with the highest loss values as the updated proxy dataset with the largest loss value; select the last K data with the lowest loss values as the updated proxy dataset with the smallest loss value; In the first training batch, directly calculate the learning loss values of the current batch of data, sort the loss values, and select the first K data with the highest loss values as the proxy dataset with the largest loss value; select the last K data with the lowest loss values as the proxy dataset with the smallest loss value.

[0009] In a possible implementation, the scale of the proxy dataset is 25% - 100% of the data in each training batch.

[0010] In a possible implementation, predicting the optimal defense timing based on the gradient cosine similarity includes: Calculate the average loss of the surrogate dataset with the largest loss value for the gradient of the model parameters, and at the same time calculate the average loss of the surrogate dataset with the smallest loss value for the gradient of the model parameters, and calculate the cosine similarity of the two gradients; Use a smoothing algorithm to smooth the gradient cosine similarity obtained for each training batch. When the smoothed gradient cosine similarity is lower than a preset threshold , it is considered that this batch is the optimal defense time, where .

[0011] In a possible implementation, training a toxicity sample identifier includes: At the optimal defense time, extract the queries of each piece of data in the surrogate dataset with the largest loss value and label them as toxicity samples; extract the queries of each piece of data in the surrogate dataset with the smallest loss value and label them as clean samples; use the labeled queries to train a binary classification model, and this binary classification model is the toxicity sample identifier; In each subsequent training batch, use the updated surrogate dataset to incrementally train the toxicity sample identifier, that is, on the basis of the original model parameters, update the model parameters using the mini-batch gradient descent method.

[0012] [[ID=]16]In a possible implementation, perform gradient reverse optimization on suspicious samples and use normal gradient descent optimization on non-suspicious samples, including: Input the queries of the data in each training batch into the toxicity sample identifier. The samples classified as toxicity samples by the toxicity sample identifier are suspicious samples, and the samples classified as clean samples by the toxicity sample identifier are non-suspicious samples; Input the suspicious samples into the dense language retrieval model to obtain the learning loss value, multiply the learning loss value of the suspicious samples by the reverse learning rate to obtain the updated loss value; input the non-suspicious samples into the dense language retrieval model to obtain the learning loss value, and the reverse learning rate is negative; Calculate the average loss value according to the updated loss value of the suspicious samples and the loss value of the non-suspicious samples, calculate the gradient of this loss value with respect to the parameters of the dense language retrieval model, and then perform gradient descent to update the parameters of the dense language retrieval model.

[0013] In a possible implementation, multiplying the learning loss value of the suspicious samples by the reverse learning rate to obtain the updated loss value includes: The numerical part of the reverse learning rate Decays according to the formula , where t is the current training step, T is the optimal training time, is the decay coefficient, , where H is the lower bound of, and N is the total number of training steps.

[0014] In a second aspect, the present application provides a system for defending against backdoor attacks on dense language retrieval models, including: A data screening and dynamic update module, configured to calculate the learning loss values of training samples, screen out the training samples with the highest and lowest loss values to form two surrogate data sets; during the training process, continuously update the two surrogate data sets to ensure that they always contain the training samples with the largest and smallest loss values; the training samples include a query text and at least one answer text; An optimal defense timing prediction module, configured to calculate the gradients of the two surrogate data sets with respect to the model parameters and predict the optimal defense timing based on the gradient cosine similarity; A toxic sample identifier training module, configured to, at the optimal defense timing, mark the queries in the surrogate data set with the highest loss value as toxic samples, mark the queries in the surrogate data set with the lowest loss value as clean samples, and use the marked queries to train a toxic sample identifier; after the optimal defense timing, each time the surrogate data set is updated, incrementally train the toxic sample identifier using the surrogate data set, and use the toxic sample identifier to mark suspicious samples in the training samples; A joint gradient optimization module, configured to perform gradient backpropagation optimization on the suspicious samples and perform normal gradient descent optimization on the non-suspicious samples;

[0015] In a third aspect, the present application provides an electronic device, including: at least one memory for storing a program; at least one processor for executing the program stored in the memory, and when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program, and when the computer program runs on a processor, it causes the processor to execute the method described in the first aspect or any possible implementation manner of the first aspect.

[0017] Generally speaking, compared with the prior art through the above technical solutions conceived by the present application, the following beneficial effects are achieved: (1) The present application specifically defends against backdoor attacks on dense language retrieval models, rather than backdoor attack defenses for traditional classification tasks, filling the technical gap in backdoor attack defense methods for dense language retrieval models and providing a brand-new solution for this field.

[0018] (2) This application dynamically screens and constructs a proxy dataset from the toxicity training set, predicts the optimal defense timing based on the proxy dataset, and trains the toxicity sample identifier at the optimal timing. Meanwhile, the toxicity sample identifier is incrementally trained by continuously updating the proxy dataset. The entire process does not rely on any additional data, has wide applicability, and can effectively defend against various backdoor attack methods.

[0019] (3) By combining the strategy of gradient reverse optimization and normal gradient descent, this application can maximize the retrieval performance of the model for normal samples while defending against backdoor attacks. Specifically, gradient reverse optimization is implemented on suspicious samples to weaken the impact of backdoor attacks, while normal gradient descent optimization is used for non-suspicious samples to ensure the model's learning ability for normal data. This dual optimization mechanism significantly enhances the robustness and practicality of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is one of the flow diagrams of the method for defending against backdoor attacks on a dense language retrieval model provided by an embodiment of this application; Figure 2 is another flow diagram of the method for defending against backdoor attacks on a dense language retrieval model provided by an embodiment of this application; Figure 3 is yet another flow diagram of the method for defending against backdoor attacks on a dense language retrieval model provided by an embodiment of this application; Figure 4 is one of the structural diagrams of the system for defending against backdoor attacks on a dense language retrieval model provided by an embodiment of this application; Figure 5 is another structural diagram of the system for defending against backdoor attacks on a dense language retrieval model provided by an embodiment of this application; Figure 6 is the structural diagram of the electronic device provided by an embodiment of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] In order to make the objectives, technical solutions, and advantages of this application clearer, the following further elaborates on this application in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0022] In the embodiments of this application, words such as "exemplary" or "for example" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0023] In the description of the embodiments of the present application, unless otherwise specified, "a plurality of" means two or more. For example, a plurality of processing units means two or more processing units, etc.; a plurality of elements means two or more elements, etc.

[0024] The embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application.

[0025] Refer to Figure 1 , the present application provides a method for resisting backdoor attacks on dense language retrieval models, including: S101. Calculate the learning loss values of the training samples, respectively select the training samples with the highest and lowest loss values to form two proxy data sets. During the training process, continuously update the two proxy data sets to ensure that they always contain the training samples with the largest and smallest loss values; S102. Calculate the gradients of the two proxy data sets with respect to the model parameters, and predict the optimal defense timing based on the cosine similarity of their gradients; S103. At the optimal defense timing, mark the queries of the proxy data set with the highest loss value as toxic samples, mark the queries of the proxy data set with the lowest loss value as clean samples, and use the marked queries to train a toxic sample identifier. After the optimal defense timing, each time the proxy data set is updated, use the updated marked queries to incrementally train the toxic sample identifier; S104. After the optimal defense timing, use the toxic sample identifier to mark the suspicious samples in the training samples, perform gradient reverse optimization on the suspicious samples, and use normal gradient descent optimization on the non-suspicious samples.

[0026] First, in the embodiments of the present application, through S101, in each training batch, calculate the learning loss values of the training data in this batch and the learning loss values of the proxy data set, sort the loss values, and select the first K data with the highest loss values as the updated proxy data set with the largest loss value; select the first K data with the lowest loss values as the updated proxy data set with the smallest loss value; in the first training batch, directly calculate the learning loss values of the data in this batch, sort the loss values, and select the first K data with the highest loss values as the proxy data set with the largest loss value; select the first K data with the lowest loss values as the proxy data set with the smallest loss value.

[0027] Predict the optimal defense timing through S102, calculate the gradient of the average loss of the proxy data set with the largest loss value with respect to the model parameters, and at the same time calculate the gradient of the average loss of the proxy data set with the smallest loss value with respect to the model parameters, and calculate the cosine similarity of the two gradients; use a smoothing algorithm to smooth the gradient cosine similarity obtained in each training batch. When the smoothed gradient cosine similarity is lower than the preset threshold , where , it is considered that this batch is the optimal defense timing.

[0028] The so-called "optimal defense timing" refers to the situation where the model has fully learned the semantic features shared by all samples and is about to start learning the personalized features unique to each sample, including the toxicity features contained in the toxic samples.

[0029] Experiments show that during the training process of the dense language retrieval model, the average loss value of toxic samples is generally higher than that of clean samples. However, simply relying on the magnitude of the average loss cannot accurately distinguish toxic samples from clean samples.

[0030] On the contrary, at this "optimal defense timing" stage, the toxicity features have not been fully captured by the model, and at this time, the proportion of toxic samples in the proxy dataset with the largest loss value is the highest. Therefore, use the sample set with the largest loss value in this stage as the proxy set for toxic samples, and use the sample set with the smallest loss value as the proxy set for clean samples.

[0031] The smoothed gradient cosine similarity is lower than the preset threshold , which means that the gradient directions of toxic samples and clean samples tend to be orthogonal, and the model has shifted from learning common semantic features to learning individual differences features. This is exactly the optimal timing to activate the defense mechanism.

[0032] Through S103, a toxicity sample identifier with high recall and low false alarm is gradually obtained. At the optimal defense timing, extract the queries of each piece of data in the proxy dataset with the largest loss value and label them as toxic samples; extract the queries of each piece of data in the proxy dataset with the smallest loss value and label them as clean samples; use the labeled queries to train a binary classification model, and this binary classification model is the toxicity sample identifier; in each subsequent training batch, use the updated proxy dataset to perform incremental training on the toxicity sample identifier, that is, on the basis of the original model parameters, update the model parameters using the mini-batch gradient descent method.

[0033] In the dense language retrieval task, each training sample usually consists of a query and one or more answers. The query is a natural language question or information requirement issued by the user, and the dense language retrieval model is used to retrieve the most relevant answers or documents from the corpus. During the process of constructing the toxicity sample identifier, focus on these query texts.

[0034] And at this "optimal defense timing" stage, the toxicity features have not been fully captured by the model, and at this time, the proportion of toxic samples in the proxy dataset with the largest loss value is the highest. Therefore, use the queries of the sample set with the largest loss value in this stage to construct the toxic samples required for training the toxicity sample identifier.

[0035] Experiments show that the toxicity sample identifier is prone to overfitting the toxicity characteristics, thus solving the problem of high recall and low false positives.

[0036] Through the joint optimization of gradient backpropagation optimization for suspicious samples and normal gradient descent for non-suspicious samples in S104, the query input of the data in each training batch is fed into the toxicity sample identifier. The samples classified as toxicity samples by the toxicity sample identifier are suspicious samples, and the samples classified as clean samples by the toxicity sample identifier are non-suspicious samples; the suspicious samples are input into the dense language retrieval model to obtain the learning loss value, and the learning loss value of the suspicious samples is multiplied by the reverse learning rate (negative value) to obtain the updated loss value; the non-suspicious samples are input into the dense language retrieval model to obtain the learning loss value; the average loss value is calculated based on the updated loss value of the suspicious samples and the loss value of the non-suspicious samples, and the gradient of this loss value with respect to the parameters of the dense language retrieval model is calculated, and then gradient descent is performed to update the parameters of the dense language retrieval model.

[0037] Among them, the numerical part of the reverse learning rate Decays according to the formula where t is the current training step, T is the optimal training time, is the decay coefficient, , where H is the lower bound (a very small value close to 0), and N is the total number of training steps.

[0038] It should be noted that as the joint optimization of the gradients of suspicious samples and non-suspicious samples progresses, the proportion of toxicity samples in the sample set with the largest loss value will become higher and higher. Because the update of the proxy dataset, the incremental training of the toxicity sample identifier, and the joint gradient optimization complement each other, forming a virtuous cycle.

[0039] Referring to Figure 2 , the method for defending against the backdoor attack of the dense language retrieval model provided by the embodiment of the present application includes the following steps: (1) Proxy dataset screening and update step: Calculate the learning loss values of the training samples, respectively screen out the training samples with the highest and lowest loss values to form two proxy datasets, and during the training process, continuously update the two proxy datasets to ensure that they always contain the training samples with the largest and smallest loss values; (2) Optimal defense timing detection step: Calculate the gradients of the two proxy datasets with respect to the model parameters, and predict the optimal defense timing based on their gradient cosine similarity; (3)Toxic sample identifier training step: At the optimal defense time, mark the queries of the proxy dataset with the highest loss value as toxic samples, and mark the queries of the proxy dataset with the lowest loss value as clean samples. Use the marked queries to train a toxic sample identifier. After the optimal defense time, each time the proxy dataset is updated, use the updated marked queries to incrementally train the toxic sample identifier; (4)Gradient joint optimization step: After the optimal defense time, use the toxic sample identifier to mark the suspicious samples in the training samples, perform gradient reverse optimization on the suspicious samples, and use normal gradient descent optimization for the non-suspicious samples In the embodiment of the present application, first, the proxy datasets with the largest and smallest losses are obtained respectively, and the proxy dataset is updated in each training batch. Calculate the gradients of the average losses of the two datasets with respect to the parameters of the dense language retrieval model, calculate the cosine similarity of the two gradients and smooth it. When the smoothed cosine similarity is close to 0 (set a small threshold), it is the optimal defense time. At the optimal defense time, mark the queries of the proxy dataset with the largest loss as toxic, and mark the queries of the proxy dataset with the smallest loss as non-toxic. Use the marked queries to train a binary classification model, which is the toxic sample identifier. In each subsequent training batch when the proxy dataset is updated, use the updated queries to incrementally train the toxic sample identifier. After the optimal defense time, insert the data of each training batch into the toxic sample identifier. If it is toxic, the data is a suspicious sample, otherwise it is a non-suspicious sample. The dense language retrieval model performs gradient reverse optimization on the suspicious samples to weaken the impact of the backdoor attack, and at the same time uses normal gradient descent optimization for the non-suspicious samples to ensure the model's learning ability for normal data. This dual optimization mechanism significantly enhances the robustness and practicality of the dense language retrieval model.

[0040] Refer to Figure 3 , Figure 3 shows the process of generating the initial proxy dataset.

[0041] Generated according to the described initial proxy dataset, used to generate the proxy dataset in the first training batch.

[0042] Specifically, pass the training data of the first batch through the initial dense language retrieval model for forward propagation to obtain the loss value of each data. Sort according to the size of the loss value (from high to low or from low to high). Select the first K data with the largest loss value as the updated proxy dataset with the largest loss value; select the first K data with the smallest loss value as the updated proxy dataset with the smallest loss value; at the same time, the model performs backpropagation to update the parameters.

[0043] The proxy dataset screening and update includes: Calculate the learning loss value of the training samples, filter out the training samples with the highest and lowest loss values, and form two proxy datasets; during the training process, continuously update the two proxy datasets to ensure that they always contain the training samples with the largest and smallest loss values; Specifically, the steps of screening and updating the proxy dataset include: For each training batch, calculate the learning loss value of the training data in that batch and the learning loss value of the proxy dataset, sort the loss values, and select the first K data with the highest loss value as the proxy dataset with the largest updated loss value; select the first K data with the lowest loss value as the proxy dataset with the smallest updated loss value; In the first training batch, the learning loss value of the batch data is directly calculated, the loss values are sorted, and the first K data with the highest loss value are selected as the proxy data set with the largest loss value; the first K data with the lowest loss value are selected as the proxy data set with the smallest loss value.

[0044] Among them, the size of each proxy data set is 25%-100% of the current training batch data, and the specific proportion is selected according to the actual application scenario and effect optimization requirements.

[0045] The optimal defense opportunity detection includes: Calculate the gradients of the two proxy datasets to the model parameters and predict the optimal defense timing based on their gradient cosine similarity; Specifically, the specific steps of predicting the optimal defense opportunity based on the gradient cosine similarity include: Calculate the gradient of the average loss of the proxy dataset with the largest loss value with respect to the model parameters, and at the same time calculate the gradient of the average loss of the proxy dataset with the smallest loss value with respect to the model parameters, and calculate the cosine similarity of the two gradients; Use the smoothing algorithm to smooth the gradient cosine similarity obtained in each training batch. When the smoothed gradient cosine similarity is lower than the preset threshold ,in , that is, this batch is considered to be the best defense time T.

[0046] The toxic sample identifier training includes: At the optimal defense time, the proxy dataset with the highest loss value is marked as a toxic sample, and the proxy dataset with the lowest loss value is marked as a clean sample, and the labeled dataset is used to train the toxic sample identifier; after the optimal defense time, each time the proxy dataset is updated, the toxic sample identifier is incrementally trained using the proxy dataset. Specifically, the specific steps of using the proxy dataset to train the toxic sample identifier include: At the optimal defense time, extract the queries of each piece of data in the proxy dataset with the largest loss value and label them as toxic samples; extract the queries of each piece of data in the proxy dataset with the smallest loss value and label them as clean samples; use the labeled queries to train a binary classification model, and this binary classification model is the toxic sample identifier; In each subsequent training batch, use the updated proxy dataset to incrementally train the toxic sample identifier, that is, on the basis of the original model parameters, use the mini-batch gradient descent method to update the model parameters.

[0047] The gradient joint optimization includes: Use the toxic sample identifier to mark the suspicious samples in the training samples; perform gradient reverse optimization on the suspicious samples, and use normal gradient descent optimization on the non-suspicious samples.

[0048] Specifically, the specific steps of performing gradient reverse optimization on the suspicious samples and using normal gradient descent optimization on the non-suspicious samples include: Input the queries of the data in each training batch into the toxic sample identifier. The samples classified as toxic samples by the toxic sample identifier are suspicious samples, and the samples classified as clean samples by the toxic sample identifier are non-suspicious samples; Input the suspicious samples into the dense language retrieval model to obtain the learning loss value, multiply the learning loss value of the suspicious samples by the reverse learning rate (negative value) to obtain the updated loss value; input the non-suspicious samples into the dense language retrieval model to obtain the learning loss value; Calculate the average loss value according to the updated loss value of the suspicious samples and the loss value of the non-suspicious samples, calculate the gradient of this loss value with respect to the parameters of the dense language retrieval model, and then perform gradient descent to update the parameters of the dense language retrieval model.

[0049] Optionally, the reverse learning rate includes: The numerical part of the reverse learning rate Decay according to the formula where t is the current training step, T is the optimal training time, is the decay coefficient, , where H is the lower bound (a very small value close to 0), and N is the total number of training steps.

[0050] Next, the system for defending against the backdoor attack of the dense language retrieval model provided by this application is described. The system for defending against the backdoor attack of the dense language retrieval model described below can be mutually corresponding and referred to the method for defending against the backdoor attack of the dense language retrieval model described above.

[0051] Refer to Figure 4, this application provides a system for automatically generating and annotating vulnerable programs, including: A data screening and dynamic update module S410, which is used to calculate the learning loss values of training samples, screen out the training samples with the highest and lowest loss values, form two proxy datasets, and continuously update the two proxy datasets during the training process to ensure that they always contain the training samples with the largest and smallest loss values; An optimal defense timing prediction module S420, which is used to calculate the gradients of the two proxy datasets with respect to the model parameters and predict the optimal defense timing based on the gradient cosine similarity; A toxic sample identifier training module S430, which is used to mark the queries of the proxy dataset with the highest loss value as toxic samples and the queries of the proxy dataset with the lowest loss value as clean samples at the optimal defense timing, and use the marked queries to train the toxic sample identifier; after the optimal defense timing, each time the proxy dataset is updated, incremental training is performed on the toxic sample identifier; A joint gradient optimization module S440, which is used to mark the suspicious samples in the training samples with the toxic sample identifier after the optimal defense timing, perform gradient reverse optimization on the suspicious samples, and use normal gradient descent optimization on the non-suspicious samples.

[0052] Refer to Figure 5 , the data screening and dynamic update module includes: A loss calculation and sorting sub-module, which is used to calculate the learning loss values of the current batch and the proxy datasets in each training batch, and then sort the loss values; A proxy dataset sample selection sub-module, which is used to select the data ranked in the top K positions of the loss values to update the proxy dataset with the largest loss value, and the data ranked in the bottom K positions of the loss values to update the proxy dataset with the smallest loss value; Among them, in the first training batch, since there is no historical data, the sub-module directly generates the initial proxy dataset according to the sorting of the loss values of the current batch of data.

[0053] The scale of each proxy dataset in the data screening and dynamic update module is 25% - 100% of the data of the current training batch, and the specific ratio is selected according to the actual application scenario and the optimization requirements of the effect.

[0054] Specifically, the loss calculation and sorting sub-module is used to calculate the learning loss values of the training data and the proxy data of this batch through a dense language retrieval model in each training batch, and then use a sorting algorithm to sort (from high to low or from low to high).

[0055] The proxy dataset sample selection sub-module is used to, at each training batch, select the top K data with the largest loss values and update them as the latest proxy dataset with the largest loss values, and select the bottom K data with the smallest loss values and update them as the latest proxy dataset with the smallest loss values according to the training data of this batch and the loss values of the proxy dataset.

[0056] Since the proxy dataset has not been generated in the first training batch, the proxy dataset is directly selected according to the loss values of the data in the first training batch.

[0057] The size of the proxy dataset is fixed at each training batch, and its scale is 25% - 100% of the data in the current training batch. The specific ratio is selected according to the actual application scenario and the optimization requirements of the effect.

[0058] Furthermore, the optimal defense timing prediction module includes: The gradient calculation sub-module is used to calculate the gradients of the average losses of the two proxy datasets with respect to the model parameters respectively; The cosine similarity analysis sub-module is used to calculate the cosine similarity of the two gradients and smooth the similarity through a smoothing algorithm; The threshold judgment sub-module is used to determine that the current batch is the optimal defense timing when the smoothed cosine similarity is lower than the preset threshold.

[0059] Specifically, the gradient calculation sub-module is used to calculate the average loss values of the two proxy datasets respectively at each training batch before the optimal defense timing, and then calculate the gradients of the average loss values with respect to all the parameters of the dense language retrieval model.

[0060] The cosine similarity analysis sub-module is used to calculate the cosine similarity of the gradients obtained from the two proxy datasets at each training batch before the optimal defense timing, and smooth the obtained cosine similarity through a smoothing algorithm to obtain the changing trend of the cosine similarity as the training progresses.

[0061] The threshold judgment sub-module is used to judge the optimal defense timing, set a threshold close to 0, and when the smoothed cosine similarity is first lower than this threshold, this training batch is the optimal defense timing.

[0062] The toxic sample identifier training module includes: The sample annotation sub-module is used to extract the queries in the proxy dataset with the largest loss values and label them as toxic samples, and extract the queries in the proxy dataset with the smallest loss values and label them as clean samples during the optimal defense timing and subsequent training; The classification model training sub-module is used to train a binary classification model as a toxic sample identifier based on the labeled queries during the optimal defense timing.

[0063] An incremental training sub-module, which is used to continue training the toxicity sample identifier with the labeled queries (based on the updated proxy dataset) after the optimal defense time.

[0064] Specifically, a sample annotation sub-module, which is used to prepare training data for the toxicity sample identifier, extract the queries in the proxy dataset with the largest loss value and label them as toxicity samples, and extract the queries in the proxy dataset with the smallest loss value and label them as clean samples at the optimal defense time and each subsequent training batch; A classification model training sub-module, which is used to train a binary classification model as the toxicity sample identifier based on the labeled queries at the optimal defense time.

[0065] An incremental training sub-module, which is used to continue training on the basis of the toxicity sample identifier obtained in the previous training batch with the updated labeled queries (based on the updated proxy dataset) at each training batch after the optimal defense time, and update the classification model parameters.

[0066] Furthermore, the joint gradient optimization module includes: A sample classification sub-module, which is used to identify the data of each training batch with the toxicity sample identifier at the optimal defense time and later, input the query of each piece of data into the toxicity sample identifier, if it is classified as toxicity, then this piece of data is a suspicious sample, otherwise it is a non-suspicious sample.

[0067] A joint optimization sub-module, which is used to multiply the loss value of the suspicious sample by the reverse learning rate (negative value), and calculate the total loss gradient in combination with the normal loss value of the non-suspicious sample, and update the model parameters.

[0068] A learning rate adjustment sub-module, which is used to adjust the numerical part of the reverse learning rate dynamically according to the formula (i.e., ), where t is the current training batch, T is the optimal training time, is the attenuation coefficient, , where H is the lower bound of (a very small value close to 0), and N is the total number of training steps.

[0069] Specifically, a sample classification sub-module, which is used to identify the training data of this batch at the optimal defense time and each subsequent training batch, input the query of each piece of data into the toxicity sample identifier, if it is classified as toxicity, then label this piece of data as a suspicious sample, if it is classified as non-toxic, then label this piece of data as a non-suspicious sample.

[0070] The joint optimization sub-module is used to multiply the loss value of each suspicious sample by the reverse learning rate (negative value), and calculate the overall average loss value of the training data in the current batch in combination with the loss values of non-suspicious samples, calculate the gradient according to the overall average loss value, optimize the model by gradient descent, and update the model parameters.

[0071] Next, the system for defending against backdoor attacks on dense language retrieval models provided in this application will be described. The system for defending against backdoor attacks on dense language retrieval models described below can be correspondingly referred to the method for defending against backdoor attacks on dense language retrieval models described above.

[0072] It can be understood that the detailed functional implementation of each of the above units / modules can be referred to the introduction in the foregoing method embodiments, and will not be elaborated here.

[0073] It should be understood that the above system is used to execute the method in the above embodiments. For the corresponding program modules in the system, their implementation principles and technical effects are similar to those described in the above method. The working process of this system can refer to the corresponding process in the above method, and will not be elaborated here. [[ID=Eleven]]

[0074] Based on the method in the above embodiments, an electronic device is provided in an embodiment of this application. Figure 6 It is a schematic structural diagram of the electronic device provided in an embodiment of this application. As Figure 6 shown, the electronic device may include: a processor (Processor) 810, a communication interface (Communications Interface) 820, a memory (Memory) 830, and a communication bus 840. Among them, the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logical instructions in the memory 830 to execute the method in the above embodiments.

[0075] In addition, when the logical instructions in the above memory 830 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of this application.

[0076] Based on the method in the above embodiments, an embodiment of the present application provides a computer-readable storage medium storing a computer program. When the computer program runs on a processor, the processor is caused to execute the method in the above embodiments.

[0077] Based on the method in the above embodiments, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor is caused to execute the method in the above embodiments.

[0078] It can be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0079] The method steps in the embodiments of the present application may be implemented in a hardware manner or by a processor executing software instructions. The software instructions may be composed of corresponding software modules. The software modules may be stored in a random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), registers, hard disk, removable hard disk, CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be located in an ASIC.

[0080] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

[0081] It can be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0082] Those skilled in the art can easily understand that the above are only the preferred embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for resisting backdoor attacks on dense language retrieval models, characterized in that, Including: Calculating the learning loss values of training samples, screening out the training samples with the highest and lowest loss values to form two surrogate datasets; During the training process, continuously update the two surrogate datasets to ensure that they always contain the training samples with the largest and smallest loss values; the training samples include a query text and at least one answer text; Calculating the gradients of the two surrogate datasets with respect to the model parameters, and predicting the optimal defense timing based on the gradient cosine similarity; At the optimal defense timing, mark the queries in the surrogate dataset with the highest loss value as toxic samples, mark the queries in the surrogate dataset with the lowest loss value as clean samples, and use the marked queries to train a toxic sample identifier; after the optimal defense timing, each time the surrogate dataset is updated, incrementally train the toxic sample identifier using the surrogate dataset, and use the toxic sample identifier to mark the suspicious samples in the training samples; Perform gradient reverse optimization on the suspicious samples, and use normal gradient descent optimization on the non-suspicious samples.

2. The method for resisting backdoor attacks on dense language retrieval models according to claim 1, wherein The continuous update of the two surrogate datasets includes: For each training batch, calculate the learning loss values of the current batch of training data and the surrogate datasets, sort the loss values, and select the top K data with the highest loss values as the updated surrogate dataset with the largest loss value; select the last K data with the lowest loss values as the updated surrogate dataset with the smallest loss value; In the first training batch, directly calculate the learning loss values of the current batch of data, sort the loss values, and select the top K data with the highest loss values as the surrogate dataset with the largest loss value; select the last K data with the lowest loss values as the surrogate dataset with the smallest loss value.

3. The method for resisting backdoor attacks on dense language retrieval models according to claim 2, wherein The size of the surrogate dataset is 25% - 100% of the data in each training batch.

4. The method for resisting backdoor attacks on dense language retrieval models according to claim 1, characterized in that The predicting the optimal defense timing based on the gradient cosine similarity includes: Calculating the gradient of the average loss of the surrogate dataset with the largest loss value with respect to the model parameters, and at the same time calculating the gradient of the average loss of the surrogate dataset with the smallest loss value with respect to the model parameters, and calculating the cosine similarity of the two gradients; Smooth the gradient cosine similarity obtained for each training batch using a smoothing algorithm. When the smoothed gradient cosine similarity is lower than a preset threshold , it is considered that this batch is the optimal defense timing, where .

5. The method for resisting backdoor attacks on dense language retrieval models according to claim 1, wherein The training of the toxic sample identifier includes: At the optimal defense timing, extract and label the queries in each piece of data in the surrogate dataset with the largest loss value as toxic samples; extract and label the queries in each piece of data in the surrogate dataset with the smallest loss value as clean samples; use the labeled queries to train a binary classification model, and this binary classification model is the toxic sample identifier; In each subsequent training batch, incrementally train the toxic sample identifier using the updated surrogate dataset, that is, update the model parameters using the mini-batch gradient descent method on the basis of the original model parameters.

6. The method for resisting backdoor attacks on dense language retrieval models according to claim 1, wherein, The performing gradient reverse optimization on the suspicious samples and using normal gradient descent optimization on the non-suspicious samples includes: Input the queries of the data in each training batch into the toxic sample identifier, the samples classified as toxic samples by the toxic sample identifier are the suspicious samples, and the samples classified as clean samples by the toxic sample identifier are the non-suspicious samples; Input the suspicious sample into the dense language retrieval model to obtain the learning loss value, multiply the learning loss value of the suspicious sample by the reverse learning rate to obtain the updated loss value; input the non-suspicious sample into the dense language retrieval model to obtain the learning loss value, and the reverse learning rate is negative; Calculate the average loss value based on the updated loss value of the suspicious sample and the loss value of the non-suspicious sample, calculate the gradient of this loss value with respect to the parameters of the dense language retrieval model, and then perform gradient descent to update the parameters of the dense language retrieval model.

7. The method for resisting backdoor attacks on dense language retrieval models according to claim 6, characterized in that, The step of multiplying the learning loss value of the suspicious sample by the reverse learning rate to obtain the updated loss value includes: The numerical part of the inverse learning rate Decay according to the formula where t is the current training step, T is the optimal training time, is the decay coefficient, , where H is the lower bound of, and N is the total number of training steps.

8. A system for defending against backdoor attacks on dense language retrieval models, characterized in that, including: A data screening and dynamic update module for calculating the learning loss values of training samples, screening out the training samples with the highest and lowest loss values to form two proxy data sets; During the training process, continuously update the two proxy data sets to ensure that they always contain the training samples with the largest and smallest loss values; the training samples include a query text and at least one answer text; An optimal defense timing prediction module for calculating the gradients of the two proxy data sets with respect to the model parameters and predicting the optimal defense timing based on the gradient cosine similarity; A toxic sample identifier training module for, at the optimal defense timing, marking the queries in the proxy data set with the highest loss value as toxic samples, marking the queries in the proxy data set with the lowest loss value as clean samples, and using the marked queries to train the toxic sample identifier; after the optimal defense timing, each time the proxy data set is updated, incrementally train the toxic sample identifier using the proxy data set, and use the toxic sample identifier to mark the suspicious samples in the training samples; A joint gradient optimization module for performing gradient reverse optimization on the suspicious samples and normal gradient descent optimization on the non-suspicious samples.

9. An electronic device, characterized in that, including: At least one memory for storing computer programs; At least one processor for executing the programs stored in the memory, and when the programs stored in the memory are executed, the processor is used to execute the method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program runs on the processor, the processor is caused to execute the method according to any one of claims 1-7.

Images