BDDR backdoor detection and data restoration method and system oriented to large model

Abstract

The invention discloses a BDDR backdoor detection and data recovery method and system oriented to a large model, and belongs to the field of backdoor defense. Comprising the following steps: constructing a knowledge distillation architecture under federal learning, including an edge server and a plurality of clients, and obtaining distillation data; inputting the distillation data into a randomly initialized model for training, recording the loss change of each batch of data, and screening out abnormal batches to form a backdoor data set; the edge server initializes two independent models, respectively uses a distillation data set and a backdoor data set for training, and guides learning of backdoor features; using probability distribution to calculate and correct a backdoor label, generating a clean data set by adding noise, finely adjusting a large model, detecting residual backdoor feature intensity, and adjusting probability distribution calculation parameters to further weaken backdoor features according to the residual backdoor feature intensity so as to obtain a final repaired data set; while the generalization ability of the large model is improved, backdoor attacks can be effectively identified and defended, the data privacy of the client is protected, and the model security is ensured.

Description

Technical Field

[0001] This invention belongs to the field of backdoor defense, specifically relating to a BDDR backdoor detection and data repair method and system for large models. Background Technology

[0002] With the rapid development of deep learning technology, Large Language Models (LLMs) have demonstrated unprecedented capabilities in the field of natural language processing. However, security concerns regarding LLMs are becoming increasingly prominent, especially when fine-tuning using private domain data. This challenge is exacerbated by the depletion of public domain data, as existing datasets are insufficient to meet the needs of large models for specific domain applications. In specialized fields such as medical diagnosis and financial risk control, models require optimization by incorporating private domain data, which contains rich domain-specific information and can significantly improve model performance.

[0003] Fine-tuning LLMs using private domain data faces two major challenges: privacy protection—private data contains sensitive information, and direct sharing violates privacy regulations; and security threats—malicious actors may inject backdoor samples to manipulate model behavior through data poisoning attacks. Existing solutions have limitations: federated learning may leak sensitive information and faces the challenge of non-independent identically distributed (non-IID) data distribution; differential privacy can degrade model performance; and knowledge distillation lacks security mechanisms. Backdoor defense methods such as ABL may interfere with normal training; ABM ​​requires pre-embedding controllable backdoors; and traditional methods assume access to clean validation data, which is difficult to satisfy in distributed scenarios.

[0004] The core challenge lies in the fact that existing research treats privacy protection and backdoor defense as separate issues, lacking a unified framework. This significantly reduces model performance while ensuring security. Traditional fine-tuning methods suffer from the problem of neurons simultaneously handling benign and backdoor tasks, leading to reduced sample utility when backdoor removal is implemented. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a BDDR backdoor detection and data repair method and system for large models, thereby solving the problems in the prior art.

[0006] The objective of this invention can be achieved through the following technical solutions: A method for BDDR backdoor detection and data repair for large models includes the following steps: Construct a knowledge distillation architecture under federated learning, including an edge server S and multiple clients, and acquire distillation data to build a distillation dataset; Distillation data is input into a randomly initialized model for training. The loss changes of each batch of data are recorded. The edge server S filters out abnormal batches based on the loss changes and extracts the data from them to form a backdoor dataset. Edge server S initializes two independent models, which are trained using the distillation dataset and the backdoor dataset respectively, to guide the learning of backdoor features. Then, the backdoor labels are corrected by calculating the probability distribution. The backdoor feature intensity is reduced by adding noise, generating a clean dataset and fine-tuning the large model. The intensity of residual backdoor features is detected, and the probability distribution calculation parameters are adjusted accordingly to further weaken the backdoor features, resulting in the final repaired dataset.

[0007] Furthermore, the steps to construct the distillation dataset are as follows: S11, edge server S collects data from N clients through federated learning. Model trained on local data and synthesize a global model. ; S12, edge server S will implement the global model Send to the client Client After performing knowledge distillation using local data, the distilled data will be... The data is sent back to edge server S, which integrates the distillation data returned by all clients to obtain the distillation dataset. .

[0008] Furthermore, the process of synthesizing the global model includes: the edge server S receives the model parameters uploaded by all clients and generates a global model according to a weighted average synthesis strategy. : in, It is a client The model parameters include the weights and biases obtained from training on this client. It is a client Weights, based on the client Data volume and total data volume Allocation, ensure .

[0009] Furthermore, the steps for screening abnormal batches are as follows: For the loss value of the i-th batch after the t-th round of training The rate of decrease in loss is defined as: Define threshold function As a basis for judgment, when If so, it is determined to be an abnormal batch; in, This represents the average rate of decrease in loss across all batches. The standard deviation of the rate of decrease of the loss; For sensitivity parameters; This represents the total number of batches.

[0010] Furthermore, the steps to obtain the final repaired dataset include: S31, edge server S randomly initializes two independent models, namely model... and model Using distillation datasets Training Model Simultaneously using the backdoor dataset Training Model And guide the model by modifying the loss function. From the model Learn the characteristics of backdoors; S32, Edge server S will distill the dataset Simultaneously input the model and Make predictions and obtain the correct classification results of the original dataset based on the probability distribution calculation formula to correct the backdoor data labels; S33, edge server S further cleans the data by adding noise to weaken the backdoor feature strength and assess the residual backdoor feature strength; S34, the edge server S uses the cleaned dataset to fine-tune the large model LLM and detect the strength of residual backdoor features; if the preset attack success rate and classification accuracy targets are not achieved, the hyperparameters in the probability distribution calculation formula are adjusted according to the feedback of LLM to further weaken the backdoor features and obtain the final repaired dataset.

[0011] Furthermore, in S31, The modified loss function is: in, For loss calculation function, For the model For input The output prediction, For backdoor model For input The output prediction.

[0012] Furthermore, the probability distribution calculation formula is as follows: in, The final output feature distribution, For the model The output characteristics, For the model The output characteristics, For control model The hyperparameter of overall shear strength To control the hyperparameter of the threshold between clean activation and backdoor activation, For the model The average activation value, To correct the activation function of the linear unit, negative values ​​are set to 0; For the model For the sample No. Output of 3D features; is the dimension of the feature vector.

[0013] A BDDR backdoor detection and data repair system for large models includes: Data acquisition module: Constructs a knowledge distillation architecture under federated learning, including an edge server S and multiple clients, and acquires distilled data to build a distillation dataset; Backdoor data extraction module: Distilled data is input into a randomly initialized model for training, and the loss changes of each batch of data are recorded. The edge server S filters out abnormal batches based on the loss changes and extracts the data from them to form a backdoor dataset. Data Repair Module: Edge server S initializes two independent models, which are trained using the distillation dataset and the backdoor dataset respectively, to guide the learning of backdoor features. Then, the backdoor labels are corrected by calculating the probability distribution. The backdoor feature intensity is reduced by adding noise, generating a clean dataset and fine-tuning the large model. The intensity of residual backdoor features is detected, and the probability distribution calculation parameters are adjusted accordingly to further weaken the backdoor features, resulting in the final repaired dataset.

[0014] A computer storage medium storing a readable program, which, when executed, instructs a computing device to perform a BDDR backdoor detection and data repair method for large models as described above.

[0015] An electronic device, characterized in that it comprises: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction that causes the processor to perform an operation corresponding to the above-described BDDR backdoor detection and data repair method for large models.

[0016] The beneficial effects of this invention are: 1. By constructing a knowledge distillation architecture under federated learning, collaborative training in a distributed environment is achieved, protecting the data privacy of each client. Simultaneously, through global model synthesis and knowledge distillation techniques, the model's generalization ability and training efficiency are improved. Compared to traditional centralized training, this reduces data transmission costs and the risk of privacy leaks.

[0017] 2. This invention establishes a dynamic threshold judgment mechanism by monitoring the rate of loss decrease in each batch of data during training, which can accurately identify abnormal batches containing backdoor samples. This method is characterized by strong adaptability and a low false alarm rate, and can effectively distinguish between normal training fluctuations and abnormal loss changes caused by malicious backdoor attacks.

[0018] 3. This invention initializes two independent models to learn normal features and backdoor features respectively, and designs a specialized loss function to guide the backdoor model's learning, achieving accurate capture and separation of backdoor features. This collaborative learning mechanism can effectively distinguish between normal activation and malicious activation, providing a reliable feature foundation for subsequent data repair.

[0019] 4. This invention utilizes probability distribution calculation formulas to correct erroneous labels in backdoor data. Through difference analysis of model output features, it can accurately identify and correct tampered label information. This method, based on the statistical properties of feature distribution, has strong robustness and adaptability.

[0020] 5. This invention adds control parameters. and The adaptive noise can selectively weaken the strength of backdoor features while maintaining the integrity of normal features. This technique uses the ReLU activation function for feature selection, ensuring that only aberrant activations are suppressed, thus avoiding the unintended damage to normal features.

[0021] 6. This invention achieves continuous optimization of data repair results by detecting the intensity of residual backdoor features and dynamically adjusting hyperparameters. This iterative mechanism ensures the quality and stability of the repaired dataset, and can continuously improve the repair strategy based on feedback from a large model, avoiding the potential incompleteness of a one-time repair.

[0022] 7. This invention provides a complete technology chain from detection to repair, forming a systematic backdoor protection solution. Compared with the single detection or protection methods in the prior art, this invention can simultaneously solve key problems such as insufficient backdoor detection accuracy, limited data repair effect, and excessive computational overhead, realizing the organic combination of detection and repair, and providing reliable protection for the secure deployment of large models. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is the overall system architecture diagram of the present invention; Figure 2 This is a diagram of the BDDR backdoor defense module architecture of the present invention; Figure 3 This is a flowchart of the overall method of the present invention; Figure 4 This is a flowchart of the BDDR backdoor defense module of the present invention. Detailed Implementation

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] Example 1 like Figure 1 and 3 As shown, a BDDR (Backdoor Detection and Data Restoration) method for large models includes the following steps: S1 constructs a knowledge distillation architecture under federated learning, including edge server S and multiple clients, and acquires distillation data to build a distillation dataset; The process of acquiring all distillation data includes: S11, edge server S collects data from N clients through federated learning. Model trained on local data and synthesize a global model. ; S12, Server S will display the global model Send to the client Client After performing knowledge distillation using local data, the distilled data will be... The data is sent back to server S, which then integrates all the distillation data returned by the clients to obtain the distillation dataset. ; In S11, the global model is synthesized. The steps are as follows: S111, edge server S and N clients Establish a federated learning framework to coordinate model training and parameter sharing among clients, ensuring data privacy and the efficiency of distributed training; S112, per client The model is trained independently based on the local dataset. To preserve local data characteristics; S113, Edge server S receives model parameters uploaded by all clients and generates a global model according to the following weighted averaging synthesis strategy. : in, It is the number of clients. It is a client The model parameters include the weights and biases obtained from training on this client. It is a client Weights, based on the client Data volume and total data volume Allocation, ensure .

[0027] In S12, a distillation data dataset is constructed. The steps are as follows: S121, the edge server S will synthesize the global model. Distribute to all clients This enables clients to perform knowledge distillation on local data more efficiently, extracting key features of the local data, improving the diversity of all data, and preventing the leakage of local privacy information by outputting complete images. S122, the purpose of distillation is to compress the dataset while making it... Even if the compressed dataset performs comparably to the original dataset, this is mainly achieved through the following two methods (i.e., forward propagation and backward propagation): in, and These are compressed data samples before and after optimization, respectively. and These are the model parameters before and after optimization, respectively; The learning rate controls the step size of gradient descent; For loss function, This is the gradient of the loss function with respect to the distillation data.

[0028] To achieve the desired distillation effect, this method optimizes the loss function. The feature distance between compressed samples and original data is measured by the difference in features within the complex space: in, To compress the complex space characteristic functions of the sample set, The complex space feature function of the original dataset, yes The conjugate of complex numbers, yes The conjugate of the complex number is given by the following formula: in, This represents a multi-level feature mapping of the compressed sample set to the global model. This represents a multi-level feature mapping of the original dataset to the global model. It is a client Compressed sample set, It is a client The original private dataset, It is the imaginary unit.

[0029] Among them, each edge client Local private dataset For each class of samples, several samples are randomly selected according to the class balance principle and then randomly concatenated to obtain the initial compressed sample set. Each edge client Utilizing the global model provided by the edge server extract Multi-level feature mapping is obtained and Multi-level feature mapping By calculating the differences in features of complex space , Let be the probability distribution function.

[0030] loss function To ensure the similarity between the compressed sample and the original data feature space, and to further reduce the difference between the two, this method also introduces a moment matching loss: And maintain the loss of entropy: The former addresses the mean and variance to ensure the low-order statistical properties of the data; the latter addresses the distribution complexity to ensure the high-order statistical properties of the data. Ultimately, both approaches ensure that the compressed sample maintains consistency in statistical distribution with the original data, accurately reflecting its essential characteristics. The mean function, variance function, and Shannon entropy function are as follows: in, Let be the probability distribution function.

[0031] S123, Client Data after distillation The data is uploaded to the edge server S to form distilled data, which provides a foundation for subsequent backdoor detection and defense, as well as fine-tuning of the large model.

[0032] S124, the edge server S integrates all distillation data returned by clients to form a complete distillation dataset. .

[0033] S2, input the distillation data into the randomly initialized model for training, record the loss change of each batch of data, and the edge server S filters out the abnormal batches based on the loss change and extracts the data from them to form the backdoor dataset; The steps to obtain the backdoor dataset are as follows: S21, Input the distillation data into the randomly initialized model according to a fixed batch size for training, and record the loss change of each batch of data; S22, Server S filters out abnormal batches based on the characteristic that the loss rate of backdoor data decreases significantly faster than that of clean data, and extracts the data from them to form a backdoor dataset. ; In S21, the steps for recording the loss changes for each batch of data include: S211, In order to distinguish backdoor data from clean data as much as possible, the edge server S randomly initializes a shallow neural network model. The model is highly sensitive to backdoor data; the edge server S inputs distillation data into the training in batches. ; The model The parameters are initialized according to the following formula: in, Represents the model on server S The initial parameters, This indicates that the mean is 0 and the variance is . The normal distribution It is an identity matrix.

[0034] Inputting distillation data in batches involves processing the distillation dataset. Divided into One batch: The size of each batch is: in, For the first Data batches, For the first Batch data sample size.

[0035] S212, the edge server S records the loss and its change curve of each batch of data during the model's training process, serving as a basis for subsequent identification of backdoor data; let... , Then the recorded loss can be expressed as: in, It is a batch index. This is the total number of batches. It refers to training rounds. It is the total number of rounds. This is the loss value for batch b in round e. For batch size, For the loss function, For the model For the sample The output, It is the set of all losses.

[0036] In S22, abnormal batches are filtered and backdoor datasets are extracted. The steps include: S221, the edge server S analyzes the loss change curves of each batch of data to identify abnormal batches whose loss decreases significantly faster than that of normal data.

[0037] For the loss value of the i-th batch after the t-th round of training The rate of decrease in loss is defined as: To facilitate the determination of backdoor data batches, a threshold function is defined. As a basis for judgment, when If so, it is determined to be a backdoor data batch. The feature strengths of the backdoor dataset need to be manually adjusted. Specifically: in, The average rate of decline of losses across all batches is used as the benchmark for the normal rate of decline. The standard deviation of the rate of decrease of loss measures the dispersion of the rate of decrease for each batch. This is a sensitivity parameter, typically taking a value of [value missing]. Control the stringency of anomaly detection; This represents the total number of batches.

[0038] S222, Edge server S extracts data from the abnormal batch to form a backdoor dataset. ; S3, edge server S initializes two independent models, which are trained using the distillation dataset and the backdoor dataset respectively, to guide the learning of backdoor features; then the backdoor labels are corrected by calculating the probability distribution, the backdoor feature intensity is reduced by adding noise, a clean dataset is generated and the large model is fine-tuned, the residual backdoor feature intensity is detected, and the probability distribution calculation parameters are adjusted accordingly to further weaken the backdoor features, and finally the repaired dataset is obtained. like Figure 2 and Figure 4 As shown in Figure S3, the specific steps for obtaining the repaired dataset are as follows: S31, edge server S randomly initializes two independent models, namely model... and model Using distillation datasets Training Model Simultaneously using the backdoor dataset Training Model And guide the model by modifying the loss function. From the model Learn the characteristics of backdoors; S32, Edge server S will distill the dataset Simultaneously input the model and Make predictions and obtain the correct classification results of the original dataset based on the probability distribution calculation formula to correct the backdoor data labels; S33, edge server S further cleans the data by adding noise to weaken the backdoor feature strength and assess the residual backdoor feature strength; S34, edge server S uses the cleaned dataset to fine-tune the large model LLM and detects the strength of residual backdoor features; if the preset attack success rate and classification accuracy targets are not met, the hyperparameters in the probability distribution calculation formula are adjusted based on the feedback from the LLM. and This is done to further weaken the backdoor features and obtain the final repaired dataset.

[0039] In S31, the process of training the model and learning backdoor features includes: S311, Model Using distillation datasets The model is trained to learn the characteristics of normal data. Using only backdoor datasets And by modifying the loss function, it can be adapted from the model. Extract backdoor features from them; S312, while edge server S modifies the model The loss function guides the model. From the model Learn backdoor features; the specific loss function is as follows: in, For loss calculation function, For the model For input The output prediction, For backdoor model For input The output prediction.

[0040] In S32, the steps for correcting the backdoor data tag include: S321, Edge server S will distill the dataset Simultaneously input the model and model Obtain their output features respectively and ; S322, the edge server S corrects the backdoor data label based on the following probability distribution calculation formula: in, The final output feature distribution, For the model The output characteristics, For the model The output characteristics, For control model The hyperparameter of overall shear strength To control the hyperparameter of the threshold between clean activation and backdoor activation, For the model The average activation value, To correct the activation function of the linear unit, negative values ​​are set to 0; For the model For the sample No. Output of 3D features; is the dimension of the feature vector.

[0041] This step may be repeated multiple times to weaken the backdoor, depending on the measurements taken in steps S33 and S34. With large models The value and the threshold set by the operator. and .if or Then modify the hyperparameters according to the following formula. , Step S32 further weakens the backdoor: in, This indicates the attack success rate threshold. This represents the classification accuracy threshold. This is the current round number. , , , This is a hyperparameter used to control the adjustment ratio.

[0042] S323, the edge server S uses the above formula to offset the backdoor feature distribution, retains clean data features, and finally outputs the correct prediction result. Based on this result, it corrects the backdoor data label, providing correct label information for subsequent data purification. First, let... , Then use The activation function will output Convert to probability distribution: in, For the sample The probability distribution belonging to the j-th class, then using... The function selects the category with the highest probability as the final label: Finally, the data labels were modified to... .

[0043] After correcting all labels, you will get the dataset with corrected labels. .

[0044] In S33, the steps for purifying data and assessing the strength of residual backdoor features include: S331, the edge server S weakens the backdoor feature strength by adding noise or perturbation, making it closer to normal data characteristics, thereby ultimately achieving data purification. First, let... , Then, for all data samples Add Gaussian noise : S332, edge server S tests the cleaned dataset. The accuracy (CA, a measure of the proportion of correct predictions made by the model on clean data) and attack success rate (ASR, a measure of the effectiveness of a backdoor attack) were used to evaluate the strength of its residual backdoor features; a clean test set was prepared first. and backdoor test set Then use Training a small-scale test model with random initialization Then input the clean test set respectively. and backdoor test set Accuracy (CA) and attack success rate (ASR) were measured: in, The target label for backdoor attacks. This is an indicator function; it is 1 if the condition is true, and 0 otherwise.

[0045] S34, the steps of fine-tuning the large model and detecting the residual backdoor feature strength to obtain the repaired dataset include: S341, server S uses the cleaned dataset to fine-tune a large model to further weaken residual backdoor features and verify the strength of residual backdoor features in the dataset. First, it uses... Fine-tuning the large LLM model, its loss function is: in , , It's a mission loss. It is a regularization term used to suppress backdoor features. This is a regularization weight hyperparameter used to balance the task loss and the regularization term.

[0046] S342, next prepare the backdoor test set The attack success rate was measured by inputting LLM. : S343, Edge Server S Decision Obtained from S332 Does it meet the requirements? or If the conditions are met, adjust the parameters and continue to step S32; otherwise, continue to the next step.

[0047] S344, Edge server S outputs the final repaired dataset. .

[0048] Based on a similar inventive concept, embodiments of the present invention also provide a computer storage medium storing a readable program, which, when run by a processor, can execute the aforementioned BDDR backdoor detection and data repair method for large models.

[0049] Based on a similar inventive concept, this invention provides an electronic device, including: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other through the communication bus; The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the above-described BDDR backdoor detection and data repair method for large models.

[0050] Based on a similar inventive concept, embodiments of the present invention also provide a computer program product, including computer instructions, which instruct a computing device to perform the operations corresponding to the above-described BDDR backdoor detection and data repair method for large models.

[0051] Example 2 In this embodiment, the practicality and effectiveness of the present invention are verified through specific experiments; 1. Experimental setup Dataset and Attack Configuration: This study evaluated various backdoor attack methods based on the CIFAR-10 dataset (32×32 color images containing 10 categories): BadNets (using 1×1 white pixels as triggers), Blend (mixed-mode attack), SIG (sine wave attack), and Clean Label (CL, clean label attack), with the backdoor sample ratio set to 1%.

[0052] Federated Distillation Setup: A federated learning environment with 10 clients was constructed, employing a non-independent identically distributed (non-IID) data distribution. Two clients were designated as malicious actors. Distillation Dataset It contains approximately 5,000 samples.

[0053] Baseline methods: State-of-the-art defense methods were selected as the baseline for comparison: ABL, Fine-Pruning (FP), Meta-Cleaner (MCR), and Neural Attention Distillation (NAD). In the large language model experiments, the Janus pro 7B model was used with LoRA fine-tuning (rank set to 16, learning rate 2e-4).

[0054] Parameter settings: Detection threshold Target threshold , hyperparameters , Training configuration: batch size of 8, number of training rounds of 5, and number of pre-training iterations of 100.

[0055] Evaluation metrics: Clean accuracy (CA) was used to evaluate the model's performance on normal samples; attack success rate (ASR) was used to evaluate the effectiveness of backdoor attacks. All experiments were conducted on an NVIDIA RTX 4090 using PyTorch 2.5.1, with each experiment repeated 5 times.

[0056] 2. Experimental Results and Analysis Table 1 presents the comprehensive evaluation results under different attack scenarios on the CIFAR-10 dataset: Table 1 shows the comprehensive evaluation results under different attack scenarios. As can be seen from Table 1: Defense Effectiveness and Performance Maintenance: BDDR achieved near-perfect defense against all attack types, with an ASR (Average Score) controlled between 0.02% and 0.13%, effectively eliminating backdoor threats. In contrast, existing methods have significant vulnerabilities: ABL suffers an ASR as high as 56.42% against Blend attacks, while FP, MCR, and NAD have failure rates reaching 99% in multiple test scenarios.

[0057] While striving to maintain normal data processing performance while enhancing defense capabilities, security and defense stability are prioritized in certain situations. BDDR maintains competitive clean accuracy, with an average improvement of 7.89% compared to FP, 12.31% compared to MCR, and 9.54% compared to NAD across all attack scenarios.

[0058] Cross-platform applicability verification: Both the CIFAR-10 distillation experiment and the large language model experiment verified the broad applicability of BDDR. In the federated distillation scenario, BDDR achieved a CA of 33.8% and an ASR of 0.13%, significantly outperforming ABL (CA of 15.6% and ASR of 35.08%). The large language model fine-tuning experiment based on Janus Pro 7B demonstrated good scalability, with an ASR of 0.08% and a CA of 75%, proving the effectiveness of this method in scenarios where traditional defense architectures are not applicable.

[0059] Robustness Analysis and Core Findings: BDDR demonstrates stable defensive performance against a wide range of attacks, from traditional trigger attacks (BadNets, Blend) to advanced stealth attacks (SIG, CL). This broad applicability validates the core assumption of this invention: datasets contaminated by backdoors still contain recoverable and useful information. Through precise feature separation techniques, BDDR's "detect-repair" strategy not only eliminates the impact of backdoors but also maintains more stable defensive consistency compared to traditional "detect-discard" methods.

[0060] Example 3 A BDDR backdoor detection and data repair system for large models, specifically including: Data acquisition module: Constructs a knowledge distillation architecture under federated learning, including an edge server S and multiple clients, and acquires distilled data to build a distillation dataset; Backdoor data extraction module: Distilled data is input into a randomly initialized model for training, and the loss changes of each batch of data are recorded. The edge server S filters out abnormal batches based on the loss changes and extracts the data from them to form a backdoor dataset. Data Repair Module: Edge server S initializes two independent models, which are trained using the distillation dataset and the backdoor dataset respectively, to guide the learning of backdoor features. Then, the backdoor labels are corrected by calculating the probability distribution. The backdoor feature intensity is reduced by adding noise, generating a clean dataset and fine-tuning the large model. The intensity of residual backdoor features is detected, and the probability distribution calculation parameters are adjusted accordingly to further weaken the backdoor features, resulting in the final repaired dataset.

[0061] The methods of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses the code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for performing the methods shown herein.

[0062] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention.

Claims

1. A method for BDDR backdoor detection and data repair for large models, characterized in that, Includes the following steps: Construct a knowledge distillation architecture under federated learning, including an edge server S and multiple clients, and acquire distillation data to build a distillation dataset; Distillation data is input into a randomly initialized model for training. The loss changes of each batch of data are recorded. The edge server S filters out abnormal batches based on the loss changes and extracts the data from them to form a backdoor dataset. Edge server S initializes two independent models, which are trained using the distillation dataset and the backdoor dataset respectively, and guided to learn backdoor features. Then, the backdoor labels are corrected by calculating the probability distribution. The backdoor feature intensity is reduced by adding noise, a clean dataset is generated, and the large model is fine-tuned to detect the residual backdoor feature intensity. Based on this, the probability distribution calculation parameters are adjusted to further weaken the backdoor features, resulting in the final repaired dataset.

2. The BDDR backdoor detection and data repair method for large models according to claim 1, characterized in that, The steps to construct the distillation dataset are as follows: S11, edge server S collects data from N clients through federated learning. Model trained on local data and synthesize a global model. ; S12, edge server S will implement the global model Send to the client Client After performing knowledge distillation using local data, the distilled data will be... The data is sent back to edge server S, which integrates the distillation data returned by all clients to obtain the distillation dataset. .

3. The BDDR backdoor detection and data repair method for large models according to claim 2, characterized in that, The process of synthesizing the global model includes: edge server S receiving model parameters uploaded by all clients and generating a global model according to a weighted average synthesis strategy. : in, It is a client The model parameters include the weights and biases obtained from training on this client. It is a client Weights, based on the client Data volume and total data volume Allocation, ensure .

4. The BDDR backdoor detection and data repair method for large models according to claim 1, characterized in that, The steps for filtering out abnormal batches are as follows: For the loss value of the i-th batch after the t-th round of training The rate of decrease in loss is defined as: Define threshold function As a basis for judgment, when If so, it is determined to be an abnormal batch; in, This represents the average rate of decrease in loss across all batches. The standard deviation of the rate of decrease of the loss; For sensitivity parameters; This represents the total number of batches.

5. The BDDR backdoor detection and data repair method for large models according to claim 2, characterized in that, The steps to obtain the final repaired dataset include: S31, edge server S randomly initializes two independent models, namely model... and model Using distillation datasets Training Model Simultaneously using the backdoor dataset Training Model And guide the model by modifying the loss function. From the model Learn the characteristics of backdoors; S32, Edge server S will distill the dataset Simultaneously input the model and Make predictions and obtain the correct classification results of the original dataset based on the probability distribution calculation formula to correct the backdoor data labels; S33, edge server S further cleans the data by adding noise to weaken the backdoor feature strength and assess the residual backdoor feature strength; S34, the edge server S uses the cleaned dataset to fine-tune the large model LLM and detect the strength of residual backdoor features; if the preset attack success rate and classification accuracy targets are not achieved, the hyperparameters in the probability distribution calculation formula are adjusted according to the feedback of LLM to further weaken the backdoor features and obtain the final repaired dataset.

6. The BDDR backdoor detection and data repair method for large models according to claim 5, characterized in that, In S31, The modified loss function is: in, For loss calculation function, For the model For input The output prediction, For backdoor model For input The output prediction.

7. The BDDR backdoor detection and data repair method for large models according to claim 5, characterized in that, The formula for calculating the probability distribution is: in, The final output feature distribution, For the model The output characteristics, For the model The output characteristics, For control model The hyperparameter of overall shear strength To control the hyperparameter of the threshold between clean activation and backdoor activation, For the model The average activation value, To correct the activation function of the linear unit, negative values ​​are set to 0; For the model For the sample No. Output of 3D features; is the dimension of the feature vector.

8. A BDDR backdoor detection and data repair system for large models, characterized in that, include: Data acquisition module: Constructs a knowledge distillation architecture under federated learning, including edge server S and multiple clients, and acquires distilled data to build a distillation dataset; Backdoor data extraction module: Distilled data is input into a randomly initialized model for training, and the loss changes of each batch of data are recorded. The edge server S filters out abnormal batches based on the loss changes and extracts the data from them to form a backdoor dataset. Data Repair Module: Edge server S initializes two independent models, which are trained using the distillation dataset and the backdoor dataset respectively, and guided to learn backdoor features; Then, the backdoor labels are corrected by calculating the probability distribution. The backdoor feature intensity is reduced by adding noise, a clean dataset is generated, and the large model is fine-tuned to detect the residual backdoor feature intensity. Based on this, the probability distribution calculation parameters are adjusted to further weaken the backdoor features, resulting in the final repaired dataset.

9. A computer storage medium storing a readable program, characterized in that, When the program runs, it can instruct the computing device to perform a BDDR backdoor detection and data repair method for large models as described in any one of claims 1-7.

10. An electronic device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the BDDR backdoor detection and data repair method for large models as described in any one of claims 1-7.

Images