A method and system for resisting backdoor attacks on dense language retrieval models

Abstract

The application belongs to the technical field of artificial intelligence model security, and specifically discloses a method and system for resisting backdoor attack of a dense language retrieval model, which comprises the following steps: calculating the learning loss value of a training sample, screening out the training samples with the highest and lowest loss values, forming two proxy data sets and continuously updating the two proxy data sets; calculating the gradient of the two proxy data sets on the model parameters, and predicting the best defense opportunity; at the best defense opportunity, marking the query of the proxy data set with the highest loss value as a toxic sample, marking the query of the proxy data set with the lowest loss value as a clean sample, and training a toxic sample identifier; thereafter, each time the proxy data set is updated, the toxic sample identifier is incrementally trained using the proxy data set, and suspicious samples in the training samples are marked; the suspicious samples are subjected to gradient reverse optimization, and the non-suspicious samples are subjected to normal gradient descent optimization. Through the application, the ability of the dense language retrieval model to resist backdoor attack can be significantly improved.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence model security technology, and more specifically, relates to a method and system for resisting backdoor attacks on dense language retrieval models. Background Technology

[0002] In recent years, with the rapid development of natural language processing technology, dense retrieval models have gradually replaced traditional sparse retrieval models and have been widely used in information retrieval, question-answering systems, and dialogue systems. In particular, retrieval-enhanced generation techniques, by combining large-scale pre-trained language models with retrieval models, have significantly improved the accuracy of information retrieval and the reliability of generated content. However, as the application scope of dense retrieval models expands, the security risks they face have also become increasingly prominent, with backdoor attacks posing a particularly significant threat.

[0003] Backdoor attacks are a type of attack that maliciously alters the behavior of a model. Attackers implant backdoors during model training, causing the model to behave abnormally when faced with specific trigger samples, while maintaining high performance on normal samples. Although numerous studies have shown that deep neural networks are vulnerable to backdoor attacks in classification tasks, recent research has found that dense retrieval models also face this threat. Specifically, attackers can implant backdoors into retrieval models, causing them to return pre-set malicious content when processing specific queries, thereby compromising the credibility and security of search results. Such attacks can lead to serious consequences such as the spread of misinformation, malicious manipulation of user behavior, and data breaches, posing significant security risks to practical applications.

[0004] Currently, various methods exist for defending against backdoor attacks on classification models, such as anomaly detection, model depoisoning, and training process intervention. However, these methods are difficult to apply directly to dense retrieval models, mainly because backdoor attacks on retrieval models present unique challenges: 1) Convergence differences: Backdoor tasks typically converge slower than normal tasks, making traditional detection methods based on training dynamic analysis difficult to apply directly. 2) Feature space mixing: Poisonous samples and clean samples are highly mixed in the feature space, making it difficult for anomaly detection-based defense methods to effectively distinguish them.

[0005] Therefore, developing an effective defense method and system against backdoor attacks on dense language retrieval models to improve the robustness and security of the models against malicious queries has become an urgent technical problem to be solved. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this application aims to provide a method and system for resisting backdoor attacks on dense language retrieval models. It seeks to solve the threat of backdoor attacks being implanted into dense language retrieval models when fine-tuning them using publicly available poisoned datasets, thereby improving the robustness and security of dense language retrieval models against malicious queries.

[0007] To achieve the above objectives, in a first aspect, this application provides a method for resisting backdoor attacks on dense language retrieval models, comprising:

[0008] Calculate the learning loss value of the training samples, and select the training samples with the highest and lowest loss values ​​to form two proxy datasets; during the training process, continuously update the two proxy datasets to ensure that they always contain the training samples with the largest and smallest loss values; the training samples include a query text and at least one answer text.

[0009] Calculate the gradient of the two proxy datasets with respect to the model parameters, and predict the optimal defense timing based on the gradient cosine similarity.

[0010] At the optimal defense time, the queries of the proxy dataset with the highest loss value are marked as toxic samples, and the queries of the proxy dataset with the lowest loss value are marked as clean samples. The toxic sample labeler is trained using the labeled queries. After the optimal defense time, the toxic sample labeler is incrementally trained using the proxy dataset each time the proxy dataset is updated, and the toxic sample labeler is used to mark suspicious samples in the training samples.

[0011] For suspicious samples, gradient inversion optimization is performed, while for non-suspicious samples, normal gradient descent optimization is used.

[0012] In one possible implementation, two proxy datasets are continuously updated, including:

[0013] For each training batch, calculate the learning loss value of the current batch of training data and the learning loss value of the proxy dataset, sort the loss values, select the first K data with the highest loss value as the proxy dataset with the highest updated loss value, and select the last K data with the lowest loss value as the proxy dataset with the lowest updated loss value.

[0014] In the first training batch, the learning loss value of the current batch of data is directly calculated, the loss values ​​are sorted, and the first K data with the highest loss value are selected as the proxy dataset with the largest loss value; the last K data with the lowest loss value are selected as the proxy dataset with the smallest loss value.

[0015] In one possible implementation, the size of the proxy dataset is 25% to 100% of the data in each training batch.

[0016] One possible implementation involves predicting the optimal defense timing based on gradient cosine similarity, including:

[0017] Calculate the gradient of the average loss of the proxy dataset with the largest loss value with respect to the model parameters, and simultaneously calculate the gradient of the average loss of the proxy dataset with the smallest loss value with respect to the model parameters. Calculate the cosine similarity between the two gradients.

[0018] A smoothing algorithm is used to smooth the gradient cosine similarity obtained for each training batch. When the smoothed gradient cosine similarity is lower than a preset threshold, the result is considered smoothed. This means that this batch is considered the optimal time for defense, among which .

[0019] In one possible implementation, training the toxic sample labeler includes:

[0020] At the optimal defense time, extract the query of each data in the proxy dataset with the largest loss value and label it as a toxic sample; extract the query of each data in the proxy dataset with the smallest loss value and label it as a clean sample; use the labeled queries to train a binary classification model, which is the toxic sample labeler.

[0021] In each subsequent training batch, the toxic sample labeler is incrementally trained using the updated proxy dataset, that is, the model parameters are updated using mini-batch gradient descent based on the original model parameters.

[0022] In one possible implementation, gradient backpropagation is performed on suspicious samples, while normal gradient descent optimization is used on non-suspicious samples, including:

[0023] Input the query of each training batch of data into the toxic sample identifier. The samples that are classified as toxic samples by the toxic sample identifier are suspicious samples, and the samples that are classified as clean samples by the toxic sample identifier are non-suspicious samples.

[0024] Suspicious samples are input into the dense language retrieval model to obtain learning loss values. The learning loss values ​​of suspicious samples are multiplied by the inverse learning rate to obtain updated loss values. Non-suspicious samples are input into the dense language retrieval model to obtain learning loss values, and the inverse learning rate is negative.

[0025] The average loss value is calculated based on the updated loss value of suspicious samples and the loss value of non-suspicious samples. The gradient of this loss value with respect to the parameters of the dense language retrieval model is then calculated, and gradient descent is performed to update the parameters of the dense language retrieval model.

[0026] In one possible implementation, the learning loss value of the suspicious sample is multiplied by the back-learning rate to obtain the updated loss value, including:

[0027] The numerical part of the inverse learning rate According to the formula The training time is decayed, where t is the current training step number and T is the optimal training time. The attenuation coefficient is... , where H is The lower bound of N is the total number of training steps.

[0028] Secondly, this application provides a system for resisting backdoor attacks on dense language retrieval models, comprising:

[0029] The data filtering and dynamic update module is used to calculate the learning loss value of the training samples, filter out the training samples with the highest and lowest loss values, and form two proxy datasets. During the training process, the two proxy datasets are continuously updated to ensure that they always contain the training samples with the largest and smallest loss values. The training samples include a query text and at least one answer text.

[0030] The optimal defense timing prediction module is used to calculate the gradient of the two proxy datasets with respect to the model parameters and predict the optimal defense timing based on the gradient cosine similarity.

[0031] The toxic sample labeler training module is used to label the queries of the proxy dataset with the highest loss value as toxic samples and the queries of the proxy dataset with the lowest loss value as clean samples at the optimal defense time, and to train the toxic sample labeler using the labeled queries; after the optimal defense time, the toxic sample labeler is incrementally trained using the proxy dataset each time the proxy dataset is updated, and the toxic sample labeler is used to label suspicious samples in the training samples.

[0032] The joint gradient optimization module is used to perform gradient inverse optimization on suspicious samples and normal gradient descent optimization on non-suspicious samples.

[0033] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.

[0034] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0035] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art:

[0036] (1) This application is specifically designed to defend against backdoor attacks on dense language retrieval models, rather than backdoor attacks on traditional classification tasks. It fills the technical gap in backdoor attack defense methods for dense language retrieval models and provides a brand-new solution for this field.

[0037] (2) This application dynamically selects and constructs a proxy dataset from the toxic training set, predicts the optimal defense timing based on the proxy dataset, and trains the toxic sample labeler at the optimal timing. Simultaneously, the toxic sample labeler is incrementally trained by continuously updating the proxy dataset. The entire process does not rely on any additional data, has broad applicability, and can effectively defend against various backdoor attack methods.

[0038] (3) This application combines gradient backpropagation and normal gradient descent strategies to maximize the model's retrieval performance on normal samples while defending against backdoor attacks. Specifically, gradient backpropagation is implemented on suspicious samples to weaken the impact of backdoor attacks, while normal gradient descent is used on non-suspicious samples to ensure the model's ability to learn from normal data. This dual optimization mechanism significantly enhances the robustness and practicality of the model. Attached Figure Description

[0039] Figure 1 This is one of the flowcharts illustrating the method for resisting backdoor attacks on dense language retrieval models provided in this application embodiment;

[0040] Figure 2 This is the second flowchart illustrating the method for resisting backdoor attacks on dense language retrieval models provided in this application embodiment;

[0041] Figure 3 This is the third flowchart illustrating the method for resisting backdoor attacks on dense language retrieval models provided in this application embodiment;

[0042] Figure 4 This is one of the structural diagrams of a system for resisting backdoor attacks on dense language retrieval models provided in this application embodiment;

[0043] Figure 5 This is the second schematic diagram of the system for resisting backdoor attacks on dense language retrieval models provided in the embodiments of this application;

[0044] Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0046] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0047] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0048] The embodiments of this application are described below with reference to the accompanying drawings.

[0049] Reference Figure 1 This application provides a method for resisting backdoor attacks on dense language retrieval models, including:

[0050] S101. Calculate the learning loss value of the training samples, and select the training samples with the highest and lowest loss values ​​to form two proxy datasets. During the training process, continuously update the two proxy datasets to ensure that they always contain the training samples with the highest and lowest loss values.

[0051] S102. Calculate the gradients of the two proxy datasets with respect to the model parameters, and predict the optimal defense timing based on their gradient cosine similarity.

[0052] S103. At the optimal defense time, the query of the proxy dataset with the highest loss value is marked as a poisonous sample, and the query of the proxy dataset with the lowest loss value is marked as a clean sample. A poisonous sample labeler is trained using the labeled queries. After the optimal defense time, the poisonous sample labeler is incrementally trained using the updated labeled queries each time the proxy dataset is updated.

[0053] S104. After the optimal defense time, use a toxic sample identifier to mark suspicious samples in the training samples, perform gradient backpropagation on suspicious samples, and use normal gradient descent optimization on non-suspicious samples.

[0054] First, in this embodiment of the application, in step S101, the learning loss value of the training data in that batch and the learning loss value of the proxy dataset are calculated in each training batch. The loss values ​​are sorted, and the first K data with the highest loss value are selected as the proxy dataset with the highest updated loss value; the last K data with the lowest loss value are selected as the proxy dataset with the lowest updated loss value. In the first training batch, the learning loss value of the data in that batch is directly calculated, the loss values ​​are sorted, and the first K data with the highest loss value are selected as the proxy dataset with the highest loss value; the last K data with the lowest loss value are selected as the proxy dataset with the lowest loss value.

[0055] The optimal defense timing is predicted using S102. The gradient of the average loss of the proxy dataset with the largest loss value with respect to the model parameters is calculated, along with the gradient of the average loss of the proxy dataset with the smallest loss value. The cosine similarity between these two gradients is then calculated. A smoothing algorithm is used to smooth the cosine similarity of the gradients obtained for each training batch. When the smoothed cosine similarity is below a preset threshold... ,in This means that this batch is considered the best time for defense.

[0056] The "optimal defense timing" refers to the time when the model has fully learned the semantic features shared by all samples and is about to begin learning the unique features of each sample, including the toxic features contained in toxic samples.

[0057] Experiments show that during the training of dense language retrieval models, the average loss value of toxic samples is generally higher than that of clean samples. However, the average loss value alone cannot accurately distinguish between toxic and clean samples.

[0058] Conversely, in the "optimal defense timing" phase, the toxicity characteristics have not yet been fully captured by the model, and the proportion of toxic samples in the proxy dataset with the highest loss value is the highest at this time. Therefore, the set of samples with the highest loss in this phase is used as the proxy set for toxic samples, while the set of samples with the lowest loss value is used as the proxy set for clean samples.

[0059] The smoothed gradient cosine similarity is lower than a preset threshold. This means that the gradient directions of toxic samples and clean samples tend to be orthogonal, and the model has shifted from learning common semantic features to learning individual difference features. This is the optimal time to activate the defense mechanism.

[0060] By gradually obtaining a toxic sample labeler with high recall and low false positives through S103, at the optimal defense time, the query of each data in the proxy dataset with the largest loss value is extracted and labeled as a toxic sample; the query of each data in the proxy dataset with the smallest loss value is extracted and labeled as a clean sample; a binary classification model is trained using the labeled queries, and this binary classification model is the toxic sample labeler; in each subsequent training batch, the toxic sample labeler is incrementally trained using the updated proxy dataset, that is, the model parameters are updated using mini-batch gradient descent based on the original model parameters.

[0061] In dense language retrieval tasks, each training sample typically consists of a query and one or more answers. The query is a natural language question or information request posed by the user, and the dense language retrieval model is used to retrieve the most relevant answers or documents from the corpus. In constructing the toxic sample identifier, the focus is on these query texts.

[0062] During the "optimal defense timing" phase, the toxicity features have not yet been fully captured by the model, and the proportion of toxic samples in the proxy dataset with the highest loss value is the highest at this time. Therefore, the toxic samples needed to train the toxic sample labeler are constructed using the query from the sample set with the highest loss value in this phase.

[0063] Experiments show that the toxicity sample identifier is prone to overfitting to toxicity features, thus solving the problem of high recall and low false alarm.

[0064] Joint optimization of gradient back-learning for suspicious samples and normal gradient descent for non-suspicious samples is performed using S104. The query data for each training batch is input into a toxic sample identifier. Samples classified as toxic by the toxic sample identifier are considered suspicious samples, while those classified as clean samples are considered non-suspicious samples. Suspicious samples are input into the dense language retrieval model to obtain a learning loss value. This learning loss value is multiplied by the back-learning rate (negative value) to obtain an updated loss value. Non-suspicious samples are also input into the dense language retrieval model to obtain a learning loss value. The average loss value is calculated based on the updated loss value for suspicious samples and the loss value for non-suspicious samples. The gradient of this loss value with respect to the parameters of the dense language retrieval model is calculated, and then gradient descent is performed to update the parameters of the dense language retrieval model.

[0065] The numerical part of the reverse learning rate According to the formula The training time is decayed, where t is the current training step number and T is the optimal training time. The attenuation coefficient is... , where H is The lower bound of is (a very small value close to 0), and N is the total number of training steps.

[0066] It is worth noting that as the gradient joint optimization of suspicious and non-suspicious samples proceeds, the proportion of toxic samples in the sample set with the largest loss value will increase. This is because the updating of the proxy dataset, the incremental training of the toxic sample labeler, and the gradient joint optimization complement each other, forming a virtuous cycle.

[0067] Reference Figure 2 The method for resisting backdoor attacks on dense language retrieval models provided in this application includes the following steps:

[0068] (1) Proxy dataset selection and update steps: Calculate the learning loss value of the training samples, select the training samples with the highest and lowest loss values ​​respectively to form two proxy datasets. During the training process, continuously update the two proxy datasets to ensure that they always contain the training samples with the highest and lowest loss values.

[0069] (2) Optimal defense timing detection steps: Calculate the gradient of the two proxy datasets with respect to the model parameters, and predict the optimal defense timing based on their gradient cosine similarity;

[0070] (3) Toxic sample labeler training steps: At the best defense time, the query of the proxy dataset with the highest loss value is labeled as a toxic sample, and the query of the proxy dataset with the lowest loss value is labeled as a clean sample. A toxic sample labeler is trained using the labeled queries. After the best defense time, the toxic sample labeler is incrementally trained using the updated labeled queries each time the proxy dataset is updated.

[0071] (4) Gradient joint optimization steps: After the optimal defense time, use the toxic sample identifier to mark suspicious samples in the training samples, perform gradient back optimization on suspicious samples, and use normal gradient descent optimization on non-suspicious samples.

[0072] This embodiment first obtains the proxy datasets with the maximum and minimum losses, respectively. The proxy datasets are updated in each training batch, and the gradient of the average loss of the two datasets with respect to the parameters of the dense language retrieval model is calculated. The cosine similarity of the two gradients is calculated and smoothed. When the smoothed cosine similarity is close to 0 (a small threshold is set), it is the optimal defense timing. At the optimal defense timing, queries from the proxy dataset with the maximum loss are marked as toxic, and queries from the proxy dataset with the minimum loss are marked as non-toxic. A binary classification model, the toxic sample labeler, is trained using the labeled queries. In subsequent training batches that update the proxy dataset, the toxic sample labeler is incrementally trained using the updated queries. After the optimal defense timing, the data inserted into each training batch is input into the toxic sample labeler. If the data is toxic, it is considered a suspicious sample; otherwise, it is considered a non-suspicious sample. The dense language retrieval model performs gradient backpropagation on suspicious samples to mitigate the impact of backdoor attacks, while simultaneously using normal gradient descent optimization on non-suspicious samples to ensure the model's ability to learn from normal data. This dual optimization mechanism significantly enhances the robustness and practicality of the dense language retrieval model.

[0073] Reference Figure 3 , Figure 3 The process of generating the initial proxy dataset is demonstrated.

[0074] Generated based on the initial proxy dataset, used to generate the proxy dataset in the first training batch.

[0075] Specifically, the training data of the first batch is forward-propagated through the initial dense language retrieval model to obtain the loss value of each data point. The data points are then sorted according to their loss values ​​(from high to low or from low to high). The first K data points with the highest loss values ​​are selected as the proxy dataset with the largest updated loss value, and the last K data points with the lowest loss values ​​are selected as the proxy dataset with the smallest updated loss value. At the same time, the model performs backpropagation to update the parameters.

[0076] The filtering and updating of the proxy dataset includes:

[0077] Calculate the learning loss value of the training samples, select the training samples with the highest and lowest loss values ​​to form two proxy datasets; during the training process, continuously update the two proxy datasets to ensure that they always contain the training samples with the largest and smallest loss values.

[0078] Specifically, the specific steps for filtering and updating the proxy dataset include:

[0079] For each training batch, calculate the learning loss value of the training data and the learning loss value of the proxy dataset. Sort the loss values ​​and select the first K data with the highest loss values ​​as the proxy dataset with the largest updated loss value; select the last K data with the lowest loss values ​​as the proxy dataset with the smallest updated loss value.

[0080] In the first training batch, the learning loss value of the batch of data is directly calculated, the loss values ​​are sorted, and the first K data with the highest loss value are selected as the proxy dataset with the largest loss value; the last K data with the lowest loss value are selected as the proxy dataset with the smallest loss value.

[0081] The size of each proxy dataset is 25%-100% of the current training batch data, with the specific proportion selected based on the actual application scenario and effect optimization requirements.

[0082] The optimal defense timing detection includes:

[0083] Calculate the gradients of the two proxy datasets with respect to the model parameters, and predict the optimal defense timing based on their gradient cosine similarity.

[0084] Specifically, the steps for predicting the optimal defense timing based on its gradient cosine similarity include:

[0085] Calculate the gradient of the average loss of the proxy dataset with the largest loss value with respect to the model parameters, and simultaneously calculate the gradient of the average loss of the proxy dataset with the smallest loss value with respect to the model parameters. Calculate the cosine similarity between the two gradients.

[0086] A smoothing algorithm is used to smooth the gradient cosine similarity obtained for each training batch. When the smoothed gradient cosine similarity is lower than a preset threshold, the result is considered smoothed. ,in This means that this batch is considered the best time for defense (T).

[0087] The training of the toxicity sample identifier includes:

[0088] At the optimal defense point, the proxy dataset with the highest loss value is labeled as a toxic sample, and the proxy dataset with the lowest loss value is labeled as a clean sample. A toxic sample labeler is then trained using the labeled datasets. After the optimal defense point, the toxic sample labeler is incrementally trained using the proxy datasets each time they are updated.

[0089] Specifically, the steps for training the toxic sample identifier using the proxy dataset include:

[0090] At the optimal defense time, extract the query of each data in the proxy dataset with the largest loss value and label it as a toxic sample; extract the query of each data in the proxy dataset with the smallest loss value and label it as a clean sample; use the labeled queries to train a binary classification model, which is the toxic sample labeler.

[0091] In each subsequent training batch, the toxic sample labeler is incrementally trained using the updated proxy dataset, that is, the model parameters are updated using mini-batch gradient descent based on the original model parameters.

[0092] The joint gradient optimization includes:

[0093] Use a toxic sample identifier to mark suspicious samples in the training samples; perform gradient backpropagation on suspicious samples, and use normal gradient descent optimization on non-suspicious samples.

[0094] Specifically, the steps for performing gradient inversion optimization on suspicious samples and normal gradient descent optimization on non-suspicious samples include:

[0095] Input the query of each training batch of data into the toxic sample identifier. The samples that are classified as toxic samples by the toxic sample identifier are suspicious samples, and the samples that are classified as clean samples by the toxic sample identifier are non-suspicious samples.

[0096] Suspicious samples are input into the dense language retrieval model to obtain learning loss values. The learning loss values ​​of suspicious samples are multiplied by the inverse learning rate (negative value) to obtain updated loss values. Non-suspicious samples are input into the dense language retrieval model to obtain learning loss values.

[0097] The average loss value is calculated based on the updated loss value of suspicious samples and the loss value of non-suspicious samples. The gradient of this loss value with respect to the parameters of the dense language retrieval model is then calculated, and gradient descent is performed to update the parameters of the dense language retrieval model.

[0098] Optionally, the reverse learning rate includes:

[0099] The numerical part of the inverse learning rate According to the formula The training time is decayed, where t is the current training step number and T is the optimal training time. The attenuation coefficient is... where H is The lower bound of is (a very small value close to 0), and N is the total number of training steps.

[0100] The system for resisting backdoor attacks on dense language retrieval models provided in this application is described below. The system for resisting backdoor attacks on dense language retrieval models described below can be referred to in correspondence with the method for resisting backdoor attacks on dense language retrieval models described above.

[0101] Reference Figure 4 This application provides a system for automatically generating and labeling vulnerability programs, comprising:

[0102] The data filtering and dynamic update module S410 is used to calculate the learning loss value of the training samples, filter out the training samples with the highest and lowest loss values ​​to form two proxy datasets, and continuously update the two proxy datasets during the training process to ensure that they always contain the training samples with the highest and lowest loss values.

[0103] The optimal defense timing prediction module S420 is used to calculate the gradient of the model parameters between two proxy datasets and predict the optimal defense timing based on their gradient cosine similarity.

[0104] The toxic sample labeler training module S430 is used to label the query of the proxy dataset with the highest loss value as a toxic sample and the query of the proxy dataset with the lowest loss value as a clean sample at the optimal defense time, and to train the toxic sample labeler using the labeled queries; after the optimal defense time, the toxic sample labeler is incrementally trained every time the proxy dataset is updated.

[0105] The joint gradient optimization module S440 is used to mark suspicious samples in the training samples with a toxic sample identifier after the optimal defense time, perform gradient back optimization on suspicious samples, and use normal gradient descent optimization on non-suspicious samples.

[0106] Reference Figure 5 The data filtering and dynamic update module includes:

[0107] The loss calculation and sorting submodule is used to calculate the learning loss value of the current batch and the proxy dataset in each training batch, and then sort the loss values.

[0108] The proxy dataset sample selection submodule is used to select the top K data with the largest loss value to update the proxy dataset, and the bottom K data with the smallest loss value to update the proxy dataset.

[0109] In the first training batch, since there is no historical data, the submodule directly generates the initial proxy dataset by sorting the loss values ​​of the current batch data.

[0110] In the data filtering and dynamic update module, the size of each proxy dataset is 25% to 100% of the current training batch data, and the specific ratio is selected according to the actual application scenario and effect optimization requirements.

[0111] Specifically, the loss calculation and ranking submodule is used to calculate the learning loss value of the training data and agent data of each training batch through a dense language retrieval model, and then sort them using a ranking algorithm (from high to low or from low to high).

[0112] The proxy dataset sample selection submodule is used to select the top K data with the largest loss value to update the latest proxy dataset with the largest loss value, and select the bottom K data with the smallest loss value to update the latest proxy dataset, based on the training data and the loss value of the proxy dataset in each training batch.

[0113] Since the proxy dataset has not yet been generated in the first training batch, the proxy dataset is selected directly based on the loss value of the data in the first training batch.

[0114] The size of the proxy dataset is fixed in each training batch, ranging from 25% to 100% of the current training batch data. The specific percentage is selected based on the actual application scenario and effect optimization requirements.

[0115] Furthermore, the optimal defense timing prediction module includes:

[0116] The gradient calculation submodule is used to calculate the gradient of the average loss with respect to the model parameters for the two proxy datasets respectively.

[0117] The cosine similarity analysis submodule is used to calculate the cosine similarity between two gradients and to smooth the similarity using a smoothing algorithm.

[0118] The threshold judgment submodule is used to determine the best defense opportunity for the current batch when the smoothed cosine similarity is lower than the preset threshold.

[0119] Specifically, the gradient calculation submodule is used to calculate the average loss value of the two proxy datasets for each training batch before the optimal defense timing, and then calculate the gradient of the average loss value with respect to all parameters of the dense language retrieval model.

[0120] The cosine similarity analysis submodule is used to calculate the cosine similarity of the gradients obtained from the two proxy datasets for each training batch before the optimal defense timing, and to smooth the obtained cosine similarity through a smoothing algorithm to obtain the trend of cosine similarity as training progresses.

[0121] The threshold judgment submodule is used to determine the best defense timing. It sets a threshold close to 0. When the smoothed cosine similarity first falls below this threshold, the training batch is the best time to defend.

[0122] The toxicity sample identifier training module includes:

[0123] The sample labeling submodule is used to extract queries from the proxy dataset with the largest loss value and label them as toxic samples during the optimal defense time and subsequent training, and to extract queries from the proxy dataset with the smallest loss value and label them as clean samples.

[0124] The classification model training submodule is used to train a binary classification model as a toxic sample identifier based on labeled queries at the optimal defense time.

[0125] The incremental training submodule is used to continue training the toxic sample labeler using labeled queries (based on the updated proxy dataset) after the optimal defense time.

[0126] Specifically, the sample labeling submodule is used to prepare training data for the toxic sample labeler, extract the queries from the agent dataset with the largest loss value at the best defense time and in each subsequent training batch and label them as toxic samples, and extract the queries from the dataset of the agent with the smallest loss value and label them as clean samples.

[0127] The classification model training submodule is used to train a binary classification model as a toxic sample identifier based on labeled queries at the optimal defense time.

[0128] The incremental training submodule is used to continue training on the toxic sample labelers obtained in the previous training batch, using updated labeled queries (based on the updated proxy dataset) in each training batch after the optimal defense time, and to update the classification model parameters.

[0129] Furthermore, the joint gradient optimization module includes:

[0130] The sample classification submodule is used to identify the data of each training batch using the toxic sample identifier at the optimal defense time and thereafter. The query for each data is input into the toxic sample identifier. If it is classified as toxic, the data is a suspicious sample; otherwise, it is a non-suspicious sample.

[0131] The joint optimization submodule is used to multiply the loss value of suspicious samples by the reverse learning rate (negative value), and combine it with the normal loss value of non-suspicious samples to calculate the total loss gradient and update the model parameters.

[0132] The learning rate adjustment submodule is used to adjust the learning rate according to the formula. Dynamically adjust the digital part of the back learning rate (i.e.) ), where t is the current training batch and T is the optimal training time. The attenuation coefficient is... , where H is The lower bound of is (a very small value close to 0), and N is the total number of training steps.

[0133] Specifically, the sample classification submodule is used to identify the training data of the batch at the optimal defense time and in each subsequent training batch. The query of each data is input into the toxic sample identifier. If it is classified as toxic, the data is marked as a suspicious sample. If it is classified as non-toxic, the data is marked as a non-suspicious sample.

[0134] The joint optimization submodule is used to multiply the loss value of each suspicious sample by the inverse learning rate (negative value), and combine it with the loss value of non-suspicious samples to calculate the overall average loss value of the current batch of training data. Based on the overall average loss value, the gradient is calculated, the model is optimized by gradient descent, and the model parameters are updated.

[0135] The system for resisting backdoor attacks on dense language retrieval models provided in this application is described below. The system for resisting backdoor attacks on dense language retrieval models described below can be referred to in correspondence with the method for resisting backdoor attacks on dense language retrieval models described above.

[0136] It is understood that the detailed functional implementation of each of the above units / modules can be found in the description in the aforementioned method embodiments, and will not be repeated here.

[0137] It should be understood that the above system is used to execute the methods in the above embodiments. The corresponding program modules in the system are similar in implementation principle and technical effect to those described in the above methods. The working process of the system can be referred to the corresponding process in the above methods, and will not be repeated here.

[0138] Based on the methods in the above embodiments, this application provides an electronic device. Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application, such as... Figure 6 As shown, the electronic device may include a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute the methods in the above embodiments.

[0139] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0140] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0141] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0142] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0143] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0144] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0145] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0146] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for resisting backdoor attacks on dense language retrieval models, characterized in that, include: Calculate the learning loss value of the training samples, select the training samples with the highest and lowest loss values, and form two proxy datasets; During training, the two proxy datasets are continuously updated to ensure that the training samples with the maximum and minimum loss values ​​are always included; the training samples include a query text and at least one answer text. Calculate the gradient of the two proxy datasets with respect to the model parameters, and predict the optimal defense timing based on the gradient cosine similarity. At the optimal defense time, the queries of the proxy dataset with the highest loss value are marked as toxic samples, and the queries of the proxy dataset with the lowest loss value are marked as clean samples. A binary classification model is trained using the marked queries as a toxic sample labeler. After the optimal defense time, the toxic sample labeler is incrementally trained using the proxy dataset each time the proxy dataset is updated, and suspicious samples in the training samples are marked using the toxic sample labeler. For suspicious samples, gradient inversion optimization is performed, while for non-suspicious samples, normal gradient descent optimization is used. The continuous updating of the two proxy datasets includes: For each training batch, calculate the learning loss value of the current batch of training data and the learning loss value of the proxy dataset, sort the loss values, select the first K data with the highest loss value as the proxy dataset with the highest updated loss value, and select the last K data with the lowest loss value as the proxy dataset with the lowest updated loss value. In the first training batch, the learning loss value of the current batch of data is directly calculated, the loss values ​​are sorted, and the first K data with the highest loss value are selected as the proxy dataset with the largest loss value; the last K data with the lowest loss value are selected as the proxy dataset with the smallest loss value. The method of predicting the optimal defense timing based on gradient cosine similarity includes: Calculate the gradient of the average loss of the proxy dataset with the largest loss value with respect to the model parameters, and simultaneously calculate the gradient of the average loss of the proxy dataset with the smallest loss value with respect to the model parameters. Calculate the cosine similarity between the two gradients. A smoothing algorithm is used to smooth the gradient cosine similarity obtained for each training batch. When the smoothed gradient cosine similarity is lower than a preset threshold, the result is considered smoothed. This means that this batch is considered the optimal time for defense, among which .

2. The method for resisting backdoor attacks on dense language retrieval models according to claim 1, characterized in that, The size of the proxy dataset is 25% to 100% of the data in each training batch.

3. The method for resisting backdoor attacks on dense language retrieval models according to claim 1, characterized in that, The training toxicity sample identifier includes: At the optimal defense time, extract the query of each data in the proxy dataset with the largest loss value and label it as a toxic sample; extract the query of each data in the proxy dataset with the smallest loss value and label it as a clean sample; use the labeled queries to train a binary classification model, which is the toxic sample labeler. In each subsequent training batch, the toxic sample labeler is incrementally trained using the updated proxy dataset, that is, the model parameters are updated using mini-batch gradient descent based on the original model parameters.

4. The method for resisting backdoor attacks on dense language retrieval models according to claim 1, characterized in that, The process of performing gradient backpropagation on suspicious samples and normal gradient descent optimization on non-suspicious samples includes: Input the query of each training batch of data into the toxic sample identifier. The samples that are classified as toxic samples by the toxic sample identifier are suspicious samples, and the samples that are classified as clean samples by the toxic sample identifier are non-suspicious samples. Suspicious samples are input into the dense language retrieval model to obtain learning loss values. The learning loss values ​​of suspicious samples are multiplied by the inverse learning rate to obtain updated loss values. Non-suspicious samples are input into the dense language retrieval model to obtain learning loss values, and the inverse learning rate is negative. The average loss value is calculated based on the updated loss value of suspicious samples and the loss value of non-suspicious samples. The gradient of this loss value with respect to the parameters of the dense language retrieval model is then calculated, and gradient descent is performed to update the parameters of the dense language retrieval model.

5. The method for resisting backdoor attacks on dense language retrieval models according to claim 4, characterized in that, The step of multiplying the learning loss value of the suspicious sample by the back learning rate to obtain the updated loss value includes: The numerical part of the inverse learning rate According to the formula The training time is decayed, where t is the current training step number and T is the optimal training time. The attenuation coefficient is... where H is The lower bound of N is the total number of training steps.

6. A system for resisting backdoor attacks on dense language retrieval models, characterized in that, include: The data filtering and dynamic update module is used to calculate the learning loss value of the training samples, filter out the training samples with the highest and lowest loss values, and form two proxy datasets. During training, the two proxy datasets are continuously updated to ensure that the training samples with the maximum and minimum loss values ​​are always included; the training samples include a query text and at least one answer text. The optimal defense timing prediction module is used to calculate the gradient of the two proxy datasets with respect to the model parameters and predict the optimal defense timing based on the gradient cosine similarity. The toxic sample labeler training module is used to label the query of the proxy dataset with the highest loss value as a toxic sample and the query of the proxy dataset with the lowest loss value as a clean sample at the optimal defense time. It then trains a binary classification model using the labeled queries as the toxic sample labeler. After the optimal defense time, the toxic sample labeler is incrementally trained using the proxy dataset each time the proxy dataset is updated, and the toxic sample labeler is used to label suspicious samples in the training samples. The joint gradient optimization module is used to perform gradient inverse optimization on suspicious samples and normal gradient descent optimization on non-suspicious samples. The continuous updating of the two proxy datasets includes: For each training batch, calculate the learning loss value of the current batch of training data and the learning loss value of the proxy dataset, sort the loss values, select the first K data with the highest loss value as the proxy dataset with the highest updated loss value, and select the last K data with the lowest loss value as the proxy dataset with the lowest updated loss value. In the first training batch, the learning loss value of the current batch of data is directly calculated, the loss values ​​are sorted, and the first K data with the highest loss value are selected as the proxy dataset with the largest loss value; the last K data with the lowest loss value are selected as the proxy dataset with the smallest loss value. The method of predicting the optimal defense timing based on gradient cosine similarity includes: Calculate the gradient of the average loss of the proxy dataset with the largest loss value with respect to the model parameters, and simultaneously calculate the gradient of the average loss of the proxy dataset with the smallest loss value with respect to the model parameters. Calculate the cosine similarity between the two gradients. A smoothing algorithm is used to smooth the gradient cosine similarity obtained for each training batch. When the smoothed gradient cosine similarity is lower than a preset threshold, the result is considered smoothed. This means that this batch is considered the optimal time for defense, among which .

7. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is run on the processor, it causes the processor to perform the method as described in any one of claims 1-5.

Images