Large model backdoor attack detection and automatic tracing method

Abstract

The invention provides a large model backdoor attack detection and automatic tracing method, and aims to solve the problems of poor accuracy, high false alarm rate and difficulty in hidden backdoor recognition in large-scale pre-training model backdoor attack detection in the prior art. The method comprises the following steps: firstly, extracting parameter distribution characteristics and reasoning behavior characteristics of a target large model, and identifying potential abnormal clusters by utilizing clustering analysis; then, whether the model is affected by backdoor attacks or not is verified by analyzing deviation output by the model and generating an adversarial sample, and through a feature inversion backtracking technology, a mapping relation between input and output is tracked, and backdoor trigger features are positioned; and finally, identifying a backdoor attack source and a propagation path thereof through traceability analysis, and eliminating the backdoor influence by correcting abnormal parameters and retraining. The method has the advantages of high efficiency, low false alarm, automatic traceability and the like, the safety of a large-scale model can be effectively improved, and the credibility and reliability of the model in practical application are ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a large model backdoor attack detection and automatic tracing method. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, especially large-scale pre-training models (such as GPT series, BERT, etc.), more and more fields begin to rely on these models to perform complex tasks, such as natural language processing, computer vision and recommendation systems, etc. However, as the size of the model continues to increase, the potential security risks behind it are increasingly apparent. In particular, backdoor attacks have become an important threat to the security of large models. Attackers can embed specific triggers in the training process of the model, causing the model to exhibit abnormal behavior when encountering specific inputs, thereby endangering the security and credibility of the system. Traditional detection methods mainly focus on the analysis of the output behavior of the model, but due to the complexity and black-box nature of the model, existing detection techniques are difficult to effectively identify the presence of backdoor attacks.

[0003] Current technical means mainly rely on monitoring the inference behavior of the model, comparing the differences between normal output and abnormal output to detect potential backdoor attacks. In addition, some methods analyze the changes in model parameters during training, using clustering analysis and other techniques to discover possible abnormalities hidden in model parameters. However, these techniques still face many challenges, such as high false positive rate, inability to accurately identify complex backdoor attack patterns, and poor applicability to large-scale models. In particular, in practical applications, backdoor triggers are usually very subtle and difficult to directly discover through regular test sets and behavior analysis.

[0004] In order to solve the above problems existing in the prior art, the present application proposes an automatic backdoor detection and tracing method based on feature extraction and adversarial sample generation. By combining parameter analysis, inference behavior monitoring and adversarial sample generation of the model, the present application can efficiently identify and locate backdoor attacks in large models, and trace the attack source and propagation path through tracing technology. This method not only improves the accuracy of backdoor detection, but also greatly enhances the reliability and security of the system in practical applications. SUMMARY

[0005] The application is an automatic backdoor detection and tracing method based on feature extraction and adversarial sample generation, which solves the problems of poor accuracy, high false positive rate and inability to effectively identify hidden backdoors in the detection of large model backdoor attacks. The method can efficiently identify backdoor attacks in the model by extracting the parameter distribution features of the model, analyzing the abnormality of the reasoning behavior, and combining the adversarial sample generation technology. At the same time, through feature inversion and tracing analysis, the attack source and propagation path are traced back, greatly improving the security and credibility of large-scale pre-training models.

[0006] The method provided by the application comprises the following steps:

[0007] A large model backdoor attack detection and automatic tracing method comprises the following steps:

[0008] Step 1: Feature extraction

[0009] Extract the model parameters and reasoning behavior features of the target large model to obtain the model parameter distribution and the reasoning behavior feature set. The probability density function of the model parameter distribution is characterized by the parameter mean And variance Reflect the overall distribution characteristics of the parameters.

[0010] Step 2: Parameter clustering analysis

[0011] Based on the model parameter distribution obtained in step 1 , use the feature clustering algorithm to perform unsupervised clustering on the model parameters, identify abnormal clusters by minimizing the intra-cluster variance, and determine the potential backdoor attack target parameter set.

[0012] Step 3: Reasoning behavior analysis

[0013] Based on the abnormal cluster identified in step 2, analyze the mapping relationship between the model input and output, and judge whether the model reasoning behavior is abnormal by calculating the Euclidean distance deviation Between normal output And attack output .

[0014] Step 4: Generate adversarial samples

[0015] Based on the reasoning behavior anomaly detected in step 3, generate adversarial samples By minimizing the loss function , wherein the perturbation direction of the adversarial sample Is guided by the gradient of the loss function on the input , used to trigger potential backdoors.

[0016] Step 5: Feature inversion and backtracking

[0017] Adversarial samples generated by step 4 , calculate the gradient between input and output by back propagation algorithm , trace back the input-output mapping relationship, locate the backdoor trigger feature T;

[0018] Step 6: Traceability analysis

[0019] According to the backdoor trigger feature T identified in step 5, combined with the gradient update rule in the model training process , analyze historical training data, identify the source of backdoor attack and its propagation path;

[0020] Step 7: Backdoor correction and retraining

[0021] Based on the abnormal cluster parameters identified in step 2 and the traceability results of step 6, correct the abnormal parameters of the model, and eliminate the influence of backdoor through regularization loss function , and retrain the model.

[0022] Further, the model parameters θ in step 1 include the weights and bias terms of the neural network, and the inference behavior feature is represented by the response of the model to the input data x ; Through statistical analysis, the parameter mean And variance .

[0023] Further, the probability density function of the model parameter distribution in step 1 is:

[0024]

[0025] Where, represents the mean of the i-th parameter in the model parameter θ, reflecting the average value of the parameter in all samples; is the variance of the i-th parameter, indicating the range and dispersion of the parameter; is the specific value of the i-th parameter, reflecting the specific parameter configuration of the model in the current state.

[0026] Further, step 2 adopts K-means clustering algorithm, based on the parameter distribution in step 1, the model parameter space is divided; by calculating the intra-cluster variance , the cluster whose intra-cluster variance exceeds the abnormal detection threshold τ is determined as an abnormal cluster.

[0027] Further, the Euclidean distance deviation in step 3 is:

[0028] ;

[0029] in, This represents the Euclidean distance deviation between the output after an attack and the normal output, reflecting the magnitude of the change in model behavior. and These are the model's outputs under normal input and attacked input, respectively.

[0030] If the Euclidean distance deviation exceeds the threshold δ, the model is judged to be behaving abnormally.

[0031] Furthermore, the formula for generating adversarial examples in step 4 is:

[0032]

[0033] in, ϵ represents the generated adversarial example; x is the original input; ϵ is the step size that controls the perturbation intensity. The gradient of the loss function with respect to the input x is used to guide the direction of the perturbation; Output the target.

[0034] Furthermore, step 5 calculates the gradient between the input and output using the backpropagation algorithm. :

[0035] ;

[0036] in, The gradient between the input and output represents the effect of changes in the input on the model's output. and These are the partial derivatives of the loss function with respect to the input and output, respectively. This is the derivative of the output with respect to the input, used to derive the relationship between the input and the output.

[0037] Furthermore, step 6 analyzes the gradient update rules during the process:

[0038]

[0039] in, Let be the parameter at step t, representing the state of the model at a certain moment during training; η is the learning parameter.

[0040] The habit rate is used to control the step size of parameter updates; The gradient of the loss function with respect to the parameters determines the direction of parameter updates.

[0041] Furthermore, in step 7, the anomaly parameter correction targets the parameters in the anomaly clusters identified in step 2. Backdoor features are suppressed through a regularization term, and the corrected loss function is:

[0042]

[0043] wherein, is the corrected model parameter; λ is the regularization coefficient, which controls the model complexity; is the output of the corrected model.

[0044] Further, it also includes:

[0045] Step 8: Verify model security

[0046] Using standard test set Verify the model corrected in step 7, calculate the accuracy, precision, recall rate indicators to ensure that the model has no backdoor residue and the performance meets the standard.

[0047] The present application has the following advantages:

[0048] 1. Efficient detection: By combining feature extraction and adversarial sample generation technology, efficient backdoor attack detection is achieved, which can accurately identify hidden attack behavior.

[0049] 2. Low false positive rate: Using statistical features of model parameter distribution and reasoning behavior analysis, the false positive rate is reduced to ensure the accuracy of the detection result.

[0050] 3. Automatic tracing: Through feature inversion technology and tracing algorithm, the source and propagation path of backdoor attack can be automatically located, further enhancing the security protection capability.

[0051] 4. Adapt to large-scale model: This method is suitable for large-scale pre-training model, which can effectively play a role in complex network environment, solving the adaptability problem of traditional method to large-scale model.

[0052] 5. Enhance credibility: By correcting the model anomalies caused by backdoor attacks, the security and accuracy of the model after correction are ensured, and the credibility of the system is improved. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is the flowchart of the large model backdoor attack detection and automatic tracing method of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0055] Figure 1A flowchart of the large model backdoor attack detection and automatic tracing method of the application is shown, and the large model backdoor attack detection and automatic tracing method comprises

[0056] Step 1: Feature extraction

[0057] The parameters and inference behavior features of the model are extracted from the target large model. It is assumed that the target large model has 100,000 parameters, and the training data contains 50,000 samples. Taking a deep neural network (DNN) as an example, the model input x is a 32-dimensional vector, and the output is a 10-class prediction result. The inference function of the target large model is:

[0058]

[0059] where the dimension of the model parameter θ is , and the bias term b is a 10-dimensional vector.

[0060] Through statistical analysis of the model parameters, the parameter distribution obtained from the training process is:

[0061]

[0062] where and are the mean and variance of the model parameters, and the specific values are obtained by training.

[0063] Calculation example: Assuming that the mean of the 5th parameter is and the variance is , the probability density of the parameter is:

[0064]

[0065] Step 2: Parameter clustering analysis

[0066] 1. Clustering process

[0067] The K-means algorithm is used to cluster the model parameters, and it is assumed that there are 1000 parameters in the model and the clustering number is 10. The mean and variance of each cluster will be calculated. Assuming that the mean of a certain cluster is and the variance is .

[0068] 2. Judgment process beyond the preset threshold

[0069] The variance threshold is set to . If the variance of a cluster exceeds the threshold, the cluster is determined to be an abnormal cluster. For a cluster with a variance of , the judgment process is as follows:

[0070] ;

[0071] The specific determination results are shown in Table 1:

[0072] Table 1

[0073]

[0074] Step 3: Reasoning behavior analysis

[0075] In the clustering analysis of step 2, abnormal clusters have been identified (e.g. cluster number 1 and cluster number 4). We will use this abnormal cluster information in the reasoning behavior analysis.

[0076] Normal and attack output

[0077] Normal output:

[0078]

[0079] Attack output:

[0080]

[0081] Deviation calculation:

[0082]

[0083] Since , it can be determined that the model behavior is abnormal.

[0084] The model abnormality determination results are shown in Table 2:

[0085] Table 2

[0086]

[0087] Step 4: Generate adversarial samples

[0088] Generation process

[0089] The original input is , the target output is , and the step size is . The loss function is mean square error:

[0090]

[0091] Update the input by gradient descent method:

[0092]

[0093] The generated adversarial sample is:

[0094]

[0095] The generated adversarial sample is shown in Table 3:

[0096] Table 3

[0097]

[0098] Step 5: Feature Backpropagation

[0099] Backpropagation Calculation

[0100] Assume the input , the output , the gradient is calculated by the backpropagation algorithm:

[0101]

[0102] The calculated gradient .

[0103] The gradient calculation results are shown in Table 4:

[0104] Table 4

[0105]

[0106] Step 6: Trace Analysis

[0107] Gradient Update and Attack Source Traceback

[0108] The parameter update formula during training is:

[0109]

[0110] The gradient , and the trace analysis deduces that the attack source is in the first layer.

[0111] The trace analysis results are shown in Table 5:

[0112] Table 5

[0113]

[0114] Step 7: Backdoor Correction and Retraining

[0115] Correction Process

[0116] The abnormal parameters are corrected by regularization, and the model is retrained. Assume that the corrected loss function is:

[0117]

[0118] is the regularization coefficient.

[0119] The correction results are shown in Table 5:

[0120] Table 5

[0121]

[0122] Step 8: Model verification and performance evaluation

[0123] 1. Verification process

[0124] After completing the correction and retraining of the model, model verification is performed to evaluate its defense capabilities. During the verification process, the model performance before and after the attack is compared using a standard dataset. Assuming the verification dataset size is 10,000 samples, including normal samples and samples containing backdoor attacks. The verification accuracy is calculated by the following formula:

[0125]

[0126] The accuracy of the model before the attack is 80%, and after the correction, the model accuracy is improved to 92%, which proves that the correction effect is significant.

[0127] 2. Evaluation indicators

[0128] In the evaluation process, in addition to accuracy, the following evaluation indicators are used to comprehensively evaluate the performance of the model:

[0129] Precision:

[0130]

[0131] The calculated precision is 0.90.

[0132] Recall:

[0133]

[0134] The calculated recall is 0.88.

[0135] F1 Score:

[0136]

[0137] The calculated F1 score is 0.89.

[0138] The calculation results of each evaluation indicator are shown in Table 6:

[0139] Table 6

[0140]

[0141] According to the experimental results, the present application has the following innovations:

[0142] 1. Multi-dimensional feature extraction and anomaly detection: Existing technologies usually focus on anomaly analysis of model input data or output data, which has strong limitations. The present invention detects the model comprehensively by extracting features in multiple dimensions such as model parameters, reasoning behavior, adversarial samples, and gradient information. Especially through model parameter clustering analysis and reasoning behavior analysis, it can effectively capture potential backdoor attacks, whether the attack is through direct data input or not, improving the coverage and accuracy of detection.

[0143] 2. Abnormal cluster clustering and dynamic variance judgment mechanism: Traditional methods rely on simple threshold judgment, which is difficult to cope with complex models and data changes, and has poor adaptability to different models. The present invention introduces the K-means clustering algorithm, combined with dynamic variance judgment, through detailed analysis of the mean and variance of each parameter cluster, it can effectively identify abnormal clusters and make dynamic judgments based on preset thresholds. This way makes the detection accuracy and flexibility of the present invention greatly improved in different scenarios.

[0144] 3. Bias calculation in reasoning behavior analysis: Existing methods mostly focus on static data analysis, ignoring the abnormal behavior of the model in the reasoning phase. The present invention compares the reasoning output of the model under normal and attack input, and uses the bias calculation method to accurately capture the tiny abnormalities in the model reasoning phase. By calculating the Euclidean distance between the normal output and the attack output, it can effectively identify potential attack behavior, especially when the attack is more concealed, it can accurately reveal the abnormal behavior of the model.

[0145] 4. Combination of adversarial sample generation and feature inversion: The existing adversarial sample generation method in the prior art is mostly used to enhance the robustness of the model, but its detection and tracing function for backdoor attacks is limited. The present invention combines adversarial sample generation and gradient backpropagation technology, not only can generate adversarial samples that can trigger model vulnerabilities, but also can trace the attack source through adversarial sample inversion, extract the trigger features of backdoor attacks, and realize comprehensive tracing from input to output, improving the accuracy and effectiveness of backdoor attack detection.

[0146] 5. Gradient update tracing and dynamic repair strategy: The existing tracing method in the prior art often relies on static gradient information, and the repair process is mostly global retraining, which has high computational cost. The present invention introduces gradient update tracing based on the training process, which can accurately track the attack source and its propagation path after detecting the backdoor attack. Further, combined with regularization correction and local retraining strategy, not only the abnormal parameters are corrected, but also the efficiency and accuracy of the repair process are optimized, avoiding overfitting and resource waste.

[0147] 6. Closed-loop defense mechanism and continuous optimization: Compared with the single existing defense method, the backdoor attack repair method is more primitive. The present application provides a closed-loop defense mechanism, which realizes the automation of the whole process from detection, tracing to repair and verification. Through adaptive defense strategy and dynamic updating of sample library, the present application can cope with unknown attacks, continuously improve the security of the model, and reduce the risk of long-term dependence on manual intervention.

[0148] 7. Efficient fusion of correction and retraining: The model correction in the prior art usually involves large-scale retraining, which is high in cost and long in cycle. The present application combines regularization technology with efficient local retraining mechanism to minimize the computational cost and training cycle while correcting the model. Experimental results show that the repaired model not only maintains high accuracy, but also significantly improves the attack defense capability, and the repair process is more efficient and accurate.

[0149] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed by the present application can be easily thought of by any person skilled in the art, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A large model backdoor attack detection and automatic tracing method, characterized in that, Comprising the following steps: Step 1: Feature extraction Extract the model parameters and inference behavior characteristics of the target large model to obtain a model parameter distribution and an inference behavior characteristic set, wherein a probability density function of the model parameter distribution is represented by a parameter mean and a variance , reflecting the overall distribution characteristics of the parameters; Step 2: Parameter clustering analysis Based on the model parameter distribution obtained in step 1 Unsupervised clustering of model parameters using feature clustering algorithm, identifying abnormal clusters by minimizing intra-cluster variance, determining potential backdoor attack target parameter set; Step 3: Inference behavior analysis Based on the abnormal cluster identified in step 2, analyze the mapping relationship between the model input and output, and determine whether the model reasoning behavior is abnormal by calculating the Euclidean distance deviation of the normal output after the attack . ​ Step 4: Adversarial sample generation Based on the detected reasoning behavior anomaly of step 3, the adversarial sample is generated by minimizing the loss function The adversarial sample is generated by minimizing the loss function The adversarial sample is generated by minimizing the loss function The perturbation direction of the adversarial sample is guided by the gradient of the loss function on the input to trigger the potential backdoor Step 5: Feature backtracking The adversarial sample generated by step 4 , calculate the gradient between input and output by back propagation algorithm , trace back the input-output mapping relationship, locate the backdoor trigger feature T; Step 6: Traceability analysis According to the backdoor trigger feature T identified in step 5, combined with the gradient update rule in the model training process , analyze historical training data, identify the source of the backdoor attack and its propagation path; Step 7: Backdoor correction and retraining Based on the abnormal cluster parameters identified in step 2 and the traceability results in step 6, correct the model abnormal parameters, and regularize the loss function Eliminate backdoor effects and retrain the model.

2. The method of claim 1, wherein, The model parameters θ in step 1 include the weights and bias terms of the neural network, and the inferred behavior features are the responses of the model to the input data x representations; the parameter means and variances are obtained by statistical analysis .​ 3. The method of claim 1, wherein, Model parameter distribution in step 1 The probability density function of is: ； wherein, represents the mean of the i-th parameter in the model parameter θ, reflecting the average value of the parameter in all samples; is the variance of the i-th parameter, indicating the range and degree of dispersion of the parameter; is the specific value of the i-th parameter, reflecting the specific parameter configuration of the model in the current state.

4. The method of claim 1, wherein, Step 2 uses a K-means clustering algorithm, based on the parameter distribution of Step 1 Partition the model parameter space; by computing the within-cluster variance The within-cluster variance Clusters that exceed the anomaly detection threshold τ are identified as anomalous clusters.

5. The method of claim 1, wherein, Euclidean distance deviation in step 3 : ； wherein, represents the Euclidean distance deviation between the output after attack and the normal output, reflecting the change range of the model behavior; and are the output results of the model under normal input and attacked input, respectively. If the Euclidean distance deviation exceeds the threshold δ, it is determined that the model behavior is abnormal.

6. The method of claim 1, wherein, The adversarial sample generation formula in step 4 is: ； wherein, is the generated adversarial sample; x is the original input; and ϵ is the step size that controls the intensity of the perturbation; is the gradient of the loss function with respect to the input x, which is used to guide the perturbation direction; is the target output.

7. The method of claim 1, wherein, Step 5 computes the gradient between the input and output by a backpropagation algorithm : ； where, is the gradient between input and output, representing the effect of input change on model output; and are the partial derivatives of the loss function with respect to input and output, respectively; is the derivative of output with respect to input, used to derive the relationship between input and output.

8. The method of claim 1, wherein, Step 6 analyzes the gradient update rule in the process: ； wherein, is the parameter of the t-th step, representing the state of the model at a certain moment during the training process; η is the learning rate. a learning rate, used to control the step size of the parameter update; the gradient of the loss function with respect to the parameters determines the direction of the update of the parameters.

9. The method of claim 1, wherein, In step 7, the abnormal parameter correction is for the parameters in the abnormal cluster identified in step 2, and the backdoor feature is suppressed by a regularization term. The corrected loss function is: ； wherein, are the corrected model parameters; λ is a regularization coefficient, which controls the model complexity; is the output of the corrected model.

10. The method of claim 1, wherein, Also includes: Step 8: Verify model security Using standard test sets The model modified in step 7 is verified by calculating the accuracy, precision, and recall indicators to ensure that the model has no backdoor residues and meets the performance standards.

Images