A batch normalization mask pruning defense method based on channel contribution degree

Abstract

This invention discloses a batch normalization mask pruning defense method based on channel contribution, belonging to the field of artificial intelligence security and deep learning model protection technology. This method addresses the problems of insufficient robustness of existing defense technologies under poisoning attacks and difficulty in effectively suppressing backdoors with a small number of clean samples. It generates pseudo-poisoned samples through trigger inversion, performs masking operations on each batch normalization layer channel of the model based on clean and pseudo-poisoned samples, and calculates the channel contribution score. Based on the score, low-contribution channels are subjected to batch normalization masking and zeroed out to achieve structured pruning. Subsequently, the pruned model is fine-tuned to restore performance, and iterative optimization is used to stabilize the model performance. This invention can significantly reduce the success rate of backdoor attacks while basically maintaining the model's normal classification accuracy, improving the security and reliability of the model in practical applications such as electromagnetic signal recognition.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence security and deep learning model protection technology, and particularly relates to a batch normalized mask pruning defense method based on channel contribution. Background Technology

[0002] In recent years, the continuous advancement of artificial intelligence technology has driven the widespread application of deep neural networks in various fields such as image recognition, speech understanding, and signal processing. Particularly in the field of radio signal modulation classification, deep learning methods, with their end-to-end modeling approach and powerful automatic feature learning capabilities, can effectively improve the accuracy and robustness of modulation pattern recognition. Therefore, they have gradually replaced traditional manual feature methods, becoming the mainstream technology in this field. However, with the widespread application of deep learning in mission-critical tasks, the risk of backdoor attacks on models is constantly increasing. This means that attackers can implant specific triggers into the training data, causing the model to perform well on normal samples but execute pre-set malicious behaviors on samples containing the triggers. This seriously threatens the security and reliability of the model in practical applications.

[0003] To address this threat, the academic community has proposed various defense strategies to improve the security and robustness of models. For example, fine-tuning methods weaken the backdoor effect by pruning neurons that contribute little to normal predictions and fine-tuning the model. However, these methods rely on the assumption that poisoned neurons are dormant during normal sample inference, an assumption that may not be robust in complex real-world models. Input cleansing defense techniques utilize GradCAM to locate regions affected by triggers and employ generative adversarial networks for repair. However, these methods are computationally expensive, and the repair effectiveness depends on the accuracy of the localization. Neural Cleanse defends by inversely synthesizing trigger patterns and combining pruning or retraining, but the inverse process is computationally complex and may not converge stably to the true trigger. Furthermore, knowledge distillation-based cleansing methods attempt to perturb and eliminate the weight distribution associated with backdoors through the distillation process. However, the effectiveness of this method decreases significantly when only a small number of reliable clean samples are available.

[0004] These existing methods are insufficient in terms of channel-level measurement and structured cleanup, making it difficult to achieve efficient and accurate backdoor defense with a small number of clean samples. Specifically, existing technologies fail to fully utilize the model's internal structure, such as batch normalization layers, for refined channel-level analysis, and lack a systematic evaluation of the relative contributions of channels in normal classification and backdoor activation. This results in either computationally expensive defense processes or difficulties in significantly reducing the attack success rate while maintaining the model's normal classification performance. Therefore, in practical applications with only a small number of trustworthy clean samples, there is an urgent need for an efficient defense method that can accurately locate and suppress backdoor-related channels while maintaining the model's original performance. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention proposes a batch normalized mask pruning defense method based on channel contribution, thereby resolving the issues present in the prior art.

[0006] Firstly, to achieve the above objectives, this invention provides a batch normalized mask pruning defense method based on channel contribution, comprising the following steps:

[0007] Pseudo-poisoning samples were generated using a small number of clean samples via trigger inversion technology.

[0008] Based on the clean samples and pseudo-poisoned samples, a masking operation is performed on the batch normalization layer of the model channel by channel. The changes in the classification accuracy of clean samples and the attack success rate of pseudo-poisoned samples before and after the mask is set to zero are measured to calculate the contribution score of each channel.

[0009] Based on the contribution score, a batch normalization mask is set to zero for low contribution channels to achieve structured pruning;

[0010] The pruned model was fine-tuned using clean samples to restore the recognition performance on the clean samples;

[0011] The pruning and fine-tuning process is performed iteratively until the model performance is stable.

[0012] Optionally, the process of generating the pseudo-poisoning sample includes:

[0013] The trigger is defined as consisting of a mask matrix and a trigger mode. The mask matrix takes values ​​between 0 and 1, and the trigger mode is initialized to a small-amplitude noise with randomness.

[0014] By using an alternating optimization strategy, when the mask matrix is ​​fixed, gradient descent is used to update the triggering mode and project it into the magnitude constraint range. When the triggering mode is fixed, gradient descent is used to update the mask matrix and a soft thresholding operation is applied to induce sparsity.

[0015] After optimization, the mask matrix is ​​binarized to obtain a binary mask.

[0016] The binary mask is combined with the triggering mode to form the final trigger, which is then superimposed on the clean sample to generate a pseudo-poisoned sample.

[0017] Optionally, the process of calculating the contribution score for each channel includes:

[0018] Record the baseline classification accuracy of the model on a clean sample set and the baseline attack success rate on a pseudo-poisoned sample set without blocking any channels;

[0019] For each channel of each batch normalization layer, temporarily set the mask of that channel to zero, and remeasure the classification accuracy and attack success rate on the same sample set;

[0020] Calculate the decrease in classification accuracy and the decrease in attack success rate after blocking the channel;

[0021] The channel contribution score is obtained by multiplying the decrease in classification accuracy by the first weighting coefficient and the decrease in attack success rate by the second weighting coefficient, and then subtracting the two.

[0022] Optionally, the process of performing structured pruning includes:

[0023] Collect contribution scores for all channels in all batch normalized layers;

[0024] The scores are globally sorted from lowest to highest.

[0025] Based on the preset pruning ratio, select multiple channels ranked at the top from the sort list, so that the proportion of the number of selected channels to the total number of channels is equal to the pruning ratio.

[0026] For each selected channel, its mask parameter in the batch normalization layer is permanently set to zero, thereby masking the channel during model forward propagation.

[0027] Optionally, the process of fine-tuning the pruned model includes:

[0028] Use a clean sample set as training data;

[0029] The loss function is defined as the cross-entropy loss between the model output and the true label;

[0030] During backpropagation, only the gradients of the unmasked model parameters are calculated and updated, while the gradients of the masked channel parameters are zero and are not updated.

[0031] The training process continues until the loss function converges or the preset number of iterations is reached.

[0032] Optionally, the iterative pruning and fine-tuning process includes:

[0033] After completing one round of pruning and fine-tuning, the classification accuracy of clean samples and the attack success rate of spoofed samples are calculated on the validation set.

[0034] Compare the current performance with the performance of the previous iteration. If the change in classification accuracy is less than a preset threshold and the attack success rate is lower than the target value, then terminate the process.

[0035] Otherwise, perform the contribution score calculation, pruning, and fine-tuning again with the same or more conservative pruning ratio.

[0036] Secondly, the present invention also provides a batch normalized mask pruning defense system based on channel contribution, used to implement a batch normalized mask pruning defense method based on channel contribution, the system comprising:

[0037] The sample generation module is used to generate pseudo-poisoning samples from a small number of clean samples using trigger inversion technology.

[0038] The contribution calculation module is used to perform a masking operation on the batch normalization layer of the model channel by channel based on the clean samples and the pseudo-poisoned samples, and measure the changes in the classification accuracy of clean samples and the attack success rate of pseudo-poisoned samples before and after the mask is set to zero, so as to calculate the contribution score of each channel.

[0039] The pruning module is used to perform batch normalization masking to zero operations on low contribution channels according to the contribution score, thereby realizing structured pruning;

[0040] The fine-tuning module is used to fine-tune the pruned model using clean samples to restore the recognition performance on the clean samples.

[0041] The iterative control module is used to iteratively execute the pruning and fine-tuning process until the model performance is stable.

[0042] Thirdly, the present invention also provides a computer terminal device, comprising:

[0043] One or more processors;

[0044] A memory, coupled to the processor, for storing one or more programs;

[0045] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the batch normalized mask pruning defense method based on channel contribution in the first aspect above.

[0046] Fourthly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the steps of the batch normalized mask pruning defense method based on channel contribution in the first aspect described above.

[0047] Fifthly, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the batch normalized mask pruning defense method based on channel contribution as described in the first aspect.

[0048] Compared with the prior art, the present invention has the following advantages and technical effects:

[0049] This invention provides a batch normalized mask pruning defense method based on channel contribution. This invention can effectively detect and resist backdoor attacks in the model, improving the robustness and security of the model. By running efficiently under conditions of a small number of reliable clean samples, this invention has high versatility and can be applied to various deep learning models and task scenarios. While maintaining the ability to identify normal samples, it significantly reduces the success rate of backdoor attacks, thereby achieving effective purification at the model level and improving the model's defense performance and stability in practical applications. Attached Figure Description

[0050] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0051] Figure 1 This is a flowchart of the BN mask pruning defense method based on channel contribution according to an embodiment of the present invention;

[0052] Figure 2 This is a block diagram of the BN mask pruning defense system based on channel contribution according to an embodiment of the present invention. Detailed Implementation

[0053] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0054] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0055] This invention proposes a poisoning defense method based on channel contribution evaluation and BN mask structured pruning. This method assigns a controllable mask to the channels of the model's normalization layer, and uses clean samples and pseudo-poisoned samples generated by trigger inversion to evaluate the relative contribution of each channel under clean classification performance and trigger activation in parallel, thereby constructing a channel contribution score and identifying key layers / key channels. Subsequently, it relies on BN masks to achieve fast and reversible structured pruning, and combines lightweight fine-tuning or feature alignment to restore or maintain the performance of clean samples.

[0056] Example 1

[0057] like Figure 1 As shown, this embodiment provides a batch normalized mask pruning defense method based on channel contribution, including:

[0058] Pseudo-poisoning samples were generated using a small number of clean samples via trigger inversion technology.

[0059] Based on the clean samples and pseudo-poisoned samples, a masking operation is performed on the batch normalization layer of the model channel by channel. The changes in the classification accuracy of clean samples and the attack success rate of pseudo-poisoned samples before and after the mask is set to zero are measured to calculate the contribution score of each channel.

[0060] Based on the contribution score, a batch normalization mask is set to zero for low contribution channels to achieve structured pruning;

[0061] The pruned model was fine-tuned using clean samples to restore the recognition performance on the clean samples;

[0062] The pruning and fine-tuning process is performed iteratively until the model performance is stable.

[0063] Specifically, the above process includes:

[0064] S1: A defense dataset is constructed based on a small number of known clean electromagnetic signal samples, and a trained poisoning model containing potential backdoors is loaded. Based on this, the target labels are inversely optimized using the trigger inversion method to obtain sparse trigger patterns that can activate the model's backdoors. These triggers are then superimposed onto clean samples to generate pseudo-poisoned samples for subsequent model evaluation and channel sensitivity analysis.

[0065] S2: After preparing clean samples and pseudo-poisoned samples, a masking operation is performed on each channel of the batch normalization layer (BN layer) of the model. The changes in classification accuracy of clean samples and attack success rate of pseudo-poisoned samples before and after the mask is set to zero are measured, and the channel contribution score is calculated. This score comprehensively represents the relative importance of the channel to normal recognition performance and backdoor activation, thereby identifying key layers and low-contribution channels that may carry poisoning features.

[0066] S3: Based on the global channel contribution ranking, perform Batch Normalization (BN) masking to zero for low-contribution channels to achieve fast structured pruning. After pruning, keep the mask fixed and only fine-tune the non-mask parameters to restore or maintain recognition performance on clean samples. Optionally, feature distribution constraints can be added to the optimization objective function to further enhance the model's ability to distinguish clean samples and weaken the influence of poisoned samples.

[0067] S4: Repeat the channel pruning and fine-tuning process until the model performance stabilizes. Finally, save the obtained defended model and evaluate the classification accuracy and attack success rate before and after defense on the test set. Experiments show that this method can significantly reduce the backdoor attack success rate while basically maintaining normal classification performance, thereby improving the robustness and reliability of the model in practical applications.

[0068] As one implementation method in this embodiment, the process of generating pseudo-poisoning samples includes:

[0069] The trigger is defined as consisting of a mask matrix and a trigger mode. The mask matrix takes values ​​between 0 and 1, and the trigger mode is initialized to a small-amplitude noise with randomness.

[0070] By using an alternating optimization strategy, when the mask matrix is ​​fixed, gradient descent is used to update the triggering mode and project it into the magnitude constraint range. When the triggering mode is fixed, gradient descent is used to update the mask matrix and a soft thresholding operation is applied to induce sparsity.

[0071] After optimization, the mask matrix is ​​binarized to obtain a binary mask.

[0072] The binary mask is combined with the triggering mode to form the final trigger, which is then superimposed on the clean sample to generate a pseudo-poisoned sample.

[0073] Furthermore, the process of step S1 is as follows:

[0074] First, a training set consisting of modulated signals is constructed. , which indicates No. One signal sample, This represents the label corresponding to the training sample. Let the total number of categories be denoted as the th. Class of , This represents the total number of samples. Each signal sample... Both are I / Q vectors, obtained from discrete complex signals after preprocessing, and are represented as follows:

[0075] (1)

[0076] in, and Let represent the real and imaginary parts of a discrete complex signal, respectively, and combine them to form a two-dimensional real-valued vector. Next, define the trigger as... The target category is and with the poisoning rate The proportion of trigger injections is used to generate a poisoned sample set, represented as:

[0077] (2)

[0078] (3)

[0079] in, This refers to the trigger injection function, used to perturb predefined triggers. Superimposed on the original signal sample This process generates corresponding poisoning samples. Its label was changed to the attack target category. This yields the poisoning sample set. ,in This indicates the number of poisoned samples in the training set; the poisoned samples are then merged with the remaining clean samples to form a mixed training set. ,Right now:

[0080] (4)

[0081] in This is the remaining clean sample set.

[0082] Furthermore, based on the mixed training set By minimizing the cross-entropy loss function For network parameters Training was conducted to obtain a poisoning model. The optimization objective is:

[0083] (5)

[0084] The specific form of the cross-entropy loss function is:

[0085] (6)

[0086] In the formula Indicates the first Does the _ sample belong to the _ ... kind, The representation model will Predicted as the first The probability of the class. Through the training process of equations (5)–(6), the poisoning model used for subsequent purification and analysis is obtained. .

[0087] Finally, to obtain poisoned samples for channel-level evaluation and pruning under the constraint of holding only a small number of reliable clean samples, the inversion / reconstruction of triggers is further implemented to generate pseudo-poisoned samples. The process is as follows: First, define the trigger injection function. The general form:

[0088] (7)

[0089] in The original input sample, The sample after adding the trigger; The trigger mode (consistent with the input dimension, its initial value is Gaussian random noise with a mean of 0 and a standard deviation of 1, multiplied by a scaling factor of 0.08 as the initial amplitude. This initialization method can ensure that the trigger has good diversity and randomness in the early stage of iteration, avoiding getting trapped in local optima. At the same time, by limiting the range of the initial perturbation with a small amplitude, it gradually converges into a sparse mode that can effectively activate the backdoor in subsequent iterations.) Let be a mask matrix with values ​​in the range [0,1]. In implementation, it can be relaxed to a continuous variable for optimization, and finally thresholded to obtain a binary mask. This form characterizes the mechanism by which the trigger covers the input: when... When the corresponding position is covered by the trigger, when The corresponding position remains unchanged.

[0090] Therefore, the trigger inversion can be formulated as a regularized optimization problem that balances misclassification capability and trigger simplicity:

[0091] (8)

[0092] in Cross-entropy loss is preferred. To induce sparsity in the mask, sparse regularization weights are used. For mask Norm. To ensure the controllability of the trigger amplitude, it is preferable to add amplitude constraints:

[0093] (9)

[0094] in Represents element-wise product. This is the upper limit of the amplitude.

[0095] To efficiently solve equations (8)–(9), an alternating optimization strategy is preferred: while maintaining the mask... Fixed-time update trigger mode In maintaining Update mask at fixed time During updates, mini-batch gradient descent is used, supplemented by projection and soft thresholding to satisfy constraints and induce sparsity. For example, in the first... Fixed in the next iteration renew And project:

[0096] (10)

[0097] Then fixed renew (Relax first, then soften threshold and project):

[0098]

[0099] (11)

[0100] in Indicates the use of a small batch. These are the step size, For soft thresholding operators, This indicates a dimension-wise projection to [0,1]. Preferably, after the iteration is complete, the projection is performed on... Thresholding yields a binary mask:

[0101] (12)

[0102] in The binarization threshold is used to ultimately invert the trigger. The trigger exhibits a sparse perturbation pattern in a local region of the input signal, the shape of which is determined by a sparse mask. Control the distribution location, by The specific value is determined, and the model backdoor can be stably activated after multiple iterations of optimization.

[0103] Furthermore, in the target category In determining the target, two strategies can be preferred: one is to pre-determine the attacker's target category. The first is known and directly used for inversion; the second is an automatic determination strategy (when...). When the value is unknown, the preferred method is to invert each candidate category set one by one and compare the inversion cost and mask sparsity, selecting the most likely category based on the criterion of "minimum cost or most sparsity mask". .

[0104] Finally, the inverted trigger is... Superimposed on trusted clean samples to form a pseudo-poisoned sample set:

[0105] (13)

[0106] The pseudo-poisoned samples, together with the original clean samples, serve as inputs for channel contribution scoring of the BN channel in S2 and for fast structured pruning and fine-tuning based on the BN mask in subsequent S3.

[0107] As one implementation method in this embodiment, the process of calculating the contribution score for each channel includes:

[0108] Record the baseline classification accuracy of the model on a clean sample set and the baseline attack success rate on a pseudo-poisoned sample set without blocking any channels;

[0109] For each channel of each batch normalization layer, temporarily set the mask of that channel to zero, and remeasure the classification accuracy and attack success rate on the same sample set;

[0110] Calculate the decrease in classification accuracy and the decrease in attack success rate after blocking the channel;

[0111] The channel contribution score is obtained by multiplying the decrease in classification accuracy by the first weighting coefficient and the decrease in attack success rate by the second weighting coefficient, and then subtracting the two.

[0112] Furthermore, the process of step S2 is as follows:

[0113] After completing the trigger inversion and pseudo-poisoning sample generation in S1, the channel contribution of each layer of the model is further quantified to evaluate the sensitivity of each layer to pseudo-poisoning samples and its degree of contamination. This score aims to characterize the differences in importance of different channels under normal classification and trigger activation, thereby providing a basis for subsequent BN mask pruning.

[0114] Specifically, firstly, without any channel masking, the baseline performance of the model on clean and pseudo-poisoned sample sets is recorded, including clean classification accuracy and attack success rate. Then, a mask is applied to each individual channel of each BN layer to mask that channel, and the performance metrics are recalculated. Comparing the baseline and masking results, the performance change of that channel under the two inputs can be obtained, defined as:

[0115] (14)

[0116] (15)

[0117] in, This represents the classification accuracy of the model on a clean sample set without any channel masking. Indicates that the first is blocked Layer The classification accuracy of the model on a clean sample set after each channel; Indicates channel The degree of contribution to the classification performance of clean samples. Similarly, This indicates the success rate of the model's attack on pseudo-poisoned samples without shielding. Indicates blocking the first Layer The attack success rate of the model on pseudo-poisoned samples after each channel; This indicates a channel. Dependence on trigger activation.

[0118] Based on this, the channel contribution score is defined as follows:

[0119] (16)

[0120] in, and These are weighting coefficients, used to balance the role of channels in maintaining normal performance and suppressing backdoor dependence. The larger the value, the more significant the contribution of the channel to the model's normal classification, and it should be retained first; a lower score value indicates that the channel may be related to poisoning behavior and can be used as a pruning candidate.

[0121] As one implementation method in this embodiment, the process of performing structured pruning includes:

[0122] Collect contribution scores for all channels in all batch normalized layers;

[0123] The scores are globally sorted from lowest to highest.

[0124] Based on the preset pruning ratio, select multiple channels ranked at the top from the sort list, so that the proportion of the number of selected channels to the total number of channels is equal to the pruning ratio.

[0125] For each selected channel, its mask parameter in the batch normalization layer is permanently set to zero, thereby masking the channel during model forward propagation.

[0126] As one implementation method in this embodiment, the process of fine-tuning the pruned model includes:

[0127] Use a clean sample set as training data;

[0128] The loss function is defined as the cross-entropy loss between the model output and the true label;

[0129] During backpropagation, only the gradients of the unmasked model parameters are calculated and updated, while the gradients of the masked channel parameters are zero and are not updated.

[0130] The training process continues until the loss function converges or the preset number of iterations is reached.

[0131] Furthermore, the process of step S3 is as follows:

[0132] In obtaining the channel contribution score described in step S2 Next, the channels of each layer in the entire model are sorted from low to high according to their scores, and then pruned according to a preset ratio. Select the channel with the lowest score as the pruning target. For the first... The first layer Each channel, if its score satisfies:

[0133] (17)

[0134] Then the channel will be masked, i.e., in the corresponding BN mask. Setting the value to zero indicates that the channel no longer participates in forward propagation. In the formula, The threshold is determined by a preset pruning ratio. The decision can also be made by dynamically adjusting the performance of the validation set.

[0135] The purification model obtained after pruning Above, further utilize clean sample sets The network is fine-tuned and trained with the optimization objective of minimizing the cross-entropy loss function:

[0136] (18)

[0137] in, The total number of categories, The one-hot encoding representing the actual label; Indicates the sample predicted by the purification model Belongs to the The probability of a class.

[0138] By fine-tuning the training on the pruned model, the classification accuracy lost due to channel masking can be recovered, and the feature responses associated with backdoors can be further weakened. Through this step, the model can significantly reduce the attack success rate while maintaining clean sample classification performance, thereby achieving effective purification at the model level.

[0139] As one implementation method in this embodiment, the iterative pruning and fine-tuning process includes:

[0140] After completing one round of pruning and fine-tuning, the classification accuracy of clean samples and the attack success rate of spoofed samples are calculated on the validation set.

[0141] Compare the current performance with the performance of the previous iteration. If the change in classification accuracy is less than a preset threshold and the attack success rate is lower than the target value, then terminate the process.

[0142] Otherwise, perform the contribution score calculation, pruning, and fine-tuning again with the same or more conservative pruning ratio.

[0143] Furthermore, in step S4, after completing the model purification and fine-tuning described in step S3, when the classification accuracy on the validation set tends to stabilize or reaches the preset convergence condition, training is terminated and the network parameters of the classification model at this time are saved. , referred to as the final defense model .

[0144] During the testing phase, clean sample sets were used respectively. With poisoning sample set Input the final defense model and calculate its classification accuracy (Clean Accuracy, CA) on normal samples and attack success rate (ASR) on poisoned samples. The metrics are defined as follows:

[0145] (19)

[0146] (20)

[0147] in, This is an indicator function; it takes the value 1 when the predicted category matches the true label, and 0 otherwise. This refers to the category of the attack target.

[0148] By comparing the CA and ASR indices before and after the defense, it can be verified that the proposed method can effectively maintain the model's normal recognition capabilities while significantly reducing the success rate of backdoor attacks, thereby improving the overall security and robustness of the model.

[0149] The working principle of this invention is:

[0150] The interference of poisoned samples on the model is eliminated by combining trigger inversion and channel pruning. First, pseudo-poisoned samples are generated by inversion using a small number of reliable clean samples, and potential attack methods are simulated by optimizing the target constraint trigger mode and sparse mask. Then, based on the contribution of channels to classification accuracy and attack success rate, channels highly correlated with backdoor behavior are screened and pruned to weaken or remove the influence of poisoned features. Finally, the purified model is further fine-tuned to significantly reduce the success rate of backdoor attacks while maintaining the ability to recognize normal samples.

[0151] Based on this, the present invention provides a batch normalized mask pruning defense method based on channel contribution. The beneficial effects of the present invention are:

[0152] This method can effectively detect and defend against backdoor attacks in the model, improving its robustness and security. Compared with existing methods, this method can still run efficiently under conditions of limited poisoned samples, demonstrating high versatility and applicability to various deep learning models and task scenarios.

[0153] Reference Figure 1 A Batch Normalization (BN) mask pruning defense method based on channel contribution includes the following steps:

[0154] S1: A defense dataset is constructed based on a small number of known clean electromagnetic signal samples, and a trained poisoning model containing potential backdoors is loaded. Based on this, the target labels are inversely optimized using the trigger inversion method to obtain sparse trigger patterns that can activate the model's backdoors. These triggers are then superimposed onto clean samples to generate pseudo-poisoned samples for subsequent model evaluation and channel sensitivity analysis.

[0155] In step S1, the specific operation process is as follows:

[0156] In this embodiment, using the given dataset RML2016.10a, the modulation classification model used in this invention is 1D-ResNet. The RML2016.10a dataset contains 11 modulation categories: 8BPSK, QPSK, 8PSK, 16QAM, 64QAM, BFSK, CPFSK, and PAM4, as well as 3 analog modulations: WB-FM, AM-SSB, and AM-DSB. In this dataset, complex numbers are used to represent signal data, with orthogonal signals composed of I-channel and Q-channel data, where I represents the in-phase component and Q represents the quadrature component, with a 90-degree phase difference. The dataset is divided into training, validation, and test sets in a 6:2:2 ratio. High signal-to-noise ratio (above 10 dB) data is selected from both datasets for experiments. The training set has a signal size of 44,000, and the test set has a size of 11,000, with the same number of modulation categories. The 1D-ResNet model's network architecture consists of 6 residual blocks and 2 fully connected layers. Each residual block contains 5 convolutional layers, 2 activation layers, and 1 max pooling layer. The difference between Block 1 and the other 5 residual blocks is that the convolutional layers in Block 1 use 2D convolutional kernels, while the convolutional layers in Blocks 2-6 use 1D convolutional kernels.

[0157] Then, trigger inversion is performed on a set of trustworthy clean samples to reconstruct the triggers that can activate the backdoor of the poisoned model. Specifically, the triggers are made up of a mask. With pattern Together, they represent an iterative optimization process that simultaneously satisfies the requirements of both misleading and compact triggers. This iteration is based on clean samples... Add it to the sample, calculate the classification loss of the model for the sample with the added trigger, and introduce it into the loss term. Sparse regularization or mask norm constraints are used to limit the spatial coverage of the triggers; simultaneously, an upper bound projection is applied to the trigger amplitude to ensure that the perturbation amplitude is controllable. To improve the robustness and reproducibility of the inversion, the inversion initialization uses random noise based on the standard normal distribution multiplied by a coefficient of 0.08 as the initial value of the trigger mode, and is solved by alternating optimization (first fixing the mask and updating the mode, then fixing the mode and updating the mask). In implementation, the mini-batch stochastic gradient method is used to update the objective function, and after each iteration, soft thresholding / projection is performed on the mask to induce sparsity and satisfy the interval constraints. Preferably, the total number of inversion iterations is set to 200; when the inverted triggers enable the model to achieve an attack success rate (ASR) of 90% or more on the validation set, the inversion is considered successful and the iteration is terminated; then the mask is binarized to obtain the final trigger position. The reconstructed triggers are then superimposed on a small number of reliable clean samples to generate pseudo-poisoned samples, which serve as inputs for subsequent channel sensitivity analysis and pruning purification.

[0158] S2: After preparing clean samples and pseudo-poisoned samples, a masking operation is performed on each channel of the batch normalization layer (BN layer) of the model. The changes in classification accuracy of clean samples and attack success rate of pseudo-poisoned samples before and after the mask is set to zero are measured, and the channel contribution score is calculated. This score comprehensively represents the relative importance of the channel to normal recognition performance and backdoor activation, thereby identifying key layers and low-contribution channels that may carry poisoning features.

[0159] In step S2, the specific operation process is as follows:

[0160] Reference Figure 1 After successfully identifying pseudo-poisoned samples, the channel contribution of each layer in the model is further quantified to assess the sensitivity of each layer to pseudo-poisoned samples and its degree of contamination. Specifically, quantification is achieved by applying masking intervention to each channel of the BN layer and observing performance changes: First, the baseline performance of the model under unmasked conditions is recorded (including the classification accuracy of clean samples and the attack success rate of pseudo-poisoned samples); then, a single channel of each BN layer is temporarily zeroed (i.e., the BN mask corresponding to that channel is set to 0), and the above performance indicators are recalculated on the same subset of data; comparing the difference before and after masking yields the influence of that channel on normal recognition and trigger activation. These two types of influence are combined according to predetermined weights to form a channel contribution score, resulting in a channel sensitivity mapping for the entire model. A higher score indicates a greater impact of that layer / channel on the model output when processing poisoned samples (more dependent on trigger-related features); a lower score indicates that the channel is more critical for maintaining normal discrimination. By normalizing and sorting the scores of all layers, key layers with higher levels of contamination can be identified, providing a direct basis for subsequent BN mask pruning and fine-tuning purification.

[0161] S3: Based on the global channel contribution ranking, perform Batch Normalization (BN) masking to zero for low-contribution channels to achieve fast structured pruning. After pruning, keep the mask fixed and only fine-tune the non-mask parameters to restore or maintain recognition performance on clean samples. Optionally, feature distribution constraints can be added to the optimization objective function to further enhance the model's ability to distinguish clean samples and weaken the influence of poisoned samples.

[0162] In step S3, the specific operation process is as follows:

[0163] Reference Figure 1 After quantifying channel contributions and obtaining a global ranking, the low-contribution channels with lower rankings are further subjected to BN mask zeroing operations to achieve rapid structured pruning. Specifically, channels to be pruned are selected according to a preset ratio or threshold, and their corresponding BN masks are set to 0 and kept fixed. Subsequently, only the parameters of the remaining unmasked channels are slightly fine-tuned to continuously reduce the attack success rate of pseudo-poisoned samples while maintaining or restoring the clean sample recognition performance. During the fine-tuning process, feature distribution constraints / alignment can be optionally introduced into the training target to enhance the model's consistency in judging clean samples and weaken trigger-related responses. Through the above "ranking-mask zeroing-fine-tuning" process, contaminated layer channels can be efficiently purified, resulting in a purified model that is significantly insensitive to backdoor attacks while maintaining or slightly fluctuating clean accuracy.

[0164] S4: Repeat the channel pruning and fine-tuning process until the model performance stabilizes. Finally, save the obtained defended model and evaluate the classification accuracy and attack success rate before and after defense on the test set. Experiments show that this method can significantly reduce the backdoor attack success rate while basically maintaining normal classification performance, thereby improving the robustness and reliability of the model in practical applications.

[0165] In step S4, the specific operation process is as follows:

[0166] Repeat the channel pruning and fine-tuning process until the model performance stabilizes, and save the resulting defended model. Specifically, after completing a mask zeroing based on global channel contribution and subsequent minor fine-tuning, monitor the clean classification accuracy and attack success rate of pseudo-poisoned samples on the validation set. If the clean accuracy fluctuates less than a preset threshold for several consecutive rounds, and the attack success rate has dropped to the target level or no longer decreases significantly, then the current round of pruning-fine-tuning is considered converged. If the convergence condition is not met, repeat the process described in S3 (i.e., recalculate the channel contribution, select the next batch of low-contribution channels to zero and fine-tune) with the same or more conservative pruning ratio until the stopping criterion is met.

[0167] As described above, this invention—a poisoning defense method based on channel contribution metric and BN mask pruning—is implemented under the RML2016.10a signal dataset. First, with only a small number of known clean samples, a trained poisoning model containing potential backdoors is loaded. Trigger inversion technology is used to inversely optimize the target labels, reconstructing sparse triggers capable of activating the model's backdoors, and superimposing them onto clean samples to generate pseudo-poisoned samples, thereby constructing a defense dataset for evaluation and cleanup. Subsequently, based on the performance of clean and pseudo-poisoned samples, a mask zeroing-recovery probe operation is performed channel-by-channel on each BN layer in the model. The changes in clean classification accuracy and attack success rate before and after masking are measured, and the channel contribution score is calculated. Based on this, key channels and key layers that are more sensitive to poisoned samples and easily carry backdoor features are identified. After obtaining the global channel contribution ranking, BN mask zeroing is performed on low-contribution channels to achieve rapid structured pruning. After pruning, the mask is kept fixed, and only minor adjustments are made to non-mask parameters to restore or maintain the recognition performance on clean samples. Optionally, feature distribution constraints are introduced into the fine-tuning optimization objective to strengthen intra-class aggregation of clean samples and weaken the response of pseudo-poisoned samples, thereby further improving the purification effect. During the testing phase, clean test samples are used to evaluate the model's classification accuracy, and poisoned test samples with triggers are used to evaluate the attack success rate (ASR), providing a comprehensive comparison of performance before and after defense. Unlike methods that rely on adversarial perturbations or rely solely on loss threshold screening, this invention, through a collaborative process of trigger inversion + channel-level contribution quantification + BN mask pruning, can accurately locate and suppress backdoor-related channels and levels without accessing the attacker's original training process, achieving efficient purification of the poisoned model. While maintaining or only slightly affecting the classification accuracy of normal signals, it can significantly reduce the success rate of backdoor attacks, improving the robustness and reliability of the model in practical electromagnetic signal recognition applications.

[0168] like Figure 2 As shown in the figure, this figure corresponds to the relevant content of the modular implementation of the system in the specific implementation method of the specification, showing the core components such as the sample generation module, contribution calculation module, pruning module, fine-tuning module and iterative control module and their connection relationship.

[0169] Example 2

[0170] In this embodiment, a computer terminal device is provided, including:

[0171] One or more processors;

[0172] A memory, coupled to the processor, for storing one or more programs;

[0173] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the above-described batch normalized mask pruning defense method based on channel contribution.

[0174] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-described batch normalized mask pruning defense method based on channel contribution.

[0175] In this embodiment, an electronic device is also provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the above-described batch normalized mask pruning defense method based on channel contribution.

[0176] In this embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the above-described batch normalized mask pruning defense method based on channel contribution.

[0177] The aforementioned program can run on a processor or be stored in memory (or a computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0178] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented by different modules for different steps.

[0179] This embodiment provides such an apparatus or system. The system, referred to as a batch normalized mask pruning defense system based on channel contribution, includes:

[0180] The sample generation module is used to generate pseudo-poisoning samples from a small number of clean samples using trigger inversion technology.

[0181] The contribution calculation module is used to perform a masking operation on the batch normalization layer of the model channel by channel based on the clean samples and the pseudo-poisoned samples, and measure the changes in the classification accuracy of clean samples and the attack success rate of pseudo-poisoned samples before and after the mask is set to zero, so as to calculate the contribution score of each channel.

[0182] The pruning module is used to perform batch normalization masking to zero operations on low contribution channels according to the contribution score, thereby realizing structured pruning;

[0183] The fine-tuning module is used to fine-tune the pruned model using clean samples to restore the recognition performance on the clean samples.

[0184] The iterative control module is used to iteratively execute the pruning and fine-tuning process until the model performance is stable.

[0185] As one implementation method in this embodiment, the sample generation module includes:

[0186] A trigger definition unit is used to define a trigger consisting of a mask matrix and a trigger mode. The mask matrix takes values ​​between 0 and 1, and the trigger mode is initialized to a small-amplitude noise with randomness.

[0187] An optimization unit is used to update the triggering mode with gradient descent when the mask matrix is ​​fixed and project it into the magnitude constraint range, and update the mask matrix with gradient descent when the triggering mode is fixed and apply a soft thresholding operation to induce sparsity.

[0188] The binarization unit is used to binarize the mask matrix after optimization to obtain a binary mask;

[0189] The overlay unit is used to combine the binary mask with the triggering mode to form the final trigger, which is then overlaid on the clean sample to generate a pseudo-poisoned sample.

[0190] As one implementation method in this embodiment, the contribution calculation module includes:

[0191] The baseline recording unit is used to record the baseline classification accuracy of the model on a clean sample set and the baseline attack success rate on a pseudo-poisoned sample set when no channels are blocked.

[0192] The masking unit is used to temporarily set the mask of each channel of each batch normalization layer to zero and remeasure the classification accuracy and attack success rate on the same sample set.

[0193] The change calculation unit is used to calculate the decrease in classification accuracy and the decrease in attack success rate after blocking the channel.

[0194] The scoring calculation unit is used to multiply the decrease in classification accuracy by a first weighting coefficient, multiply the decrease in attack success rate by a second weighting coefficient, and subtract the two to obtain the channel contribution score.

[0195] As one implementation method in this embodiment, the pruning module includes:

[0196] The scoring collection unit is used to collect the contribution scores of all channels in all batch normalized layers;

[0197] A sorting unit is used to globally sort the scores from low to high.

[0198] The channel selection unit is used to select multiple top-ranked channels from the sorting list according to a preset pruning ratio, so that the proportion of the number of selected channels to the total number of channels is equal to the pruning ratio.

[0199] The mask zeroing unit is used to permanently zero out the mask parameter of each selected channel in the batch normalization layer, thereby masking the channel during model forward propagation.

[0200] As one implementation method in this embodiment, the fine-tuning module includes:

[0201] The data loading unit is used to use a clean sample set as training data.

[0202] The loss definition unit is used to define the loss function as the cross-entropy loss between the model output and the true label.

[0203] The gradient calculation unit is used to calculate the gradients of the unmasked model parameters and update these parameters during backpropagation, while the gradients of the masked channel parameters are zero and are not updated.

[0204] The training unit is used to iterate the training process until the loss function converges or the preset number of iterations is reached.

[0205] As one implementation method in this embodiment, the iteration control module includes:

[0206] The performance evaluation unit is used to calculate the classification accuracy of clean samples and the attack success rate of fake poisoned samples on the validation set after completing one pruning and fine-tuning.

[0207] The comparison unit is used to compare the current performance with the performance of the previous iteration. If the change in classification accuracy is less than a preset threshold and the attack success rate is lower than the target value, the process is terminated.

[0208] The cyclic control unit is used to re-perform contribution score calculation, pruning, and fine-tuning with the same or more conservative pruning ratio.

[0209] The system or apparatus is used to implement the functions of the methods in the above embodiments. Each module in the system or apparatus corresponds to each step in the method, as has been described in the method and will not be repeated here.

[0210] The above implementation method solves the problem of batch normalized mask pruning defense based on channel contribution in related technologies, thereby ensuring that the problems existing in the prior art are resolved.

[0211] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A batch normalized mask pruning defense method based on channel contribution, characterized in that, The backdoor attack defense applied to the radio signal modulation identification model includes the following steps: A pseudo-poisoning sample is generated using a small number of clean samples through trigger inversion technology, wherein the clean samples include clean electromagnetic signal modulation samples, and the pseudo-poisoning sample includes pseudo-poisoning electromagnetic signal samples. The process of generating pseudo-poisoning samples includes: The trigger is defined as consisting of a mask matrix and a trigger mode. The mask matrix takes values ​​between 0 and 1, and the trigger mode is initialized to a small-amplitude noise with randomness. By using an alternating optimization strategy, when the mask matrix is ​​fixed, gradient descent is used to update the triggering mode and project it into the magnitude constraint range. When the triggering mode is fixed, gradient descent is used to update the mask matrix and a soft thresholding operation is applied to induce sparsity. After optimization, the mask matrix is ​​binarized to obtain a binary mask. The binary mask is combined with the triggering mode to form the final trigger, which is then superimposed on the clean sample to generate a pseudo-poisoned sample. Based on the clean samples and pseudo-poisoned samples, a masking operation is performed on the batch normalization layer of the model channel by channel. The changes in the classification accuracy of clean samples and the attack success rate of pseudo-poisoned samples before and after the mask is set to zero are measured to calculate the contribution score of each channel. The process of calculating the contribution score for each channel includes: Record the baseline classification accuracy of the model on a clean sample set and the baseline attack success rate on a pseudo-poisoned sample set without blocking any channels; For each channel of each batch normalization layer, temporarily set the mask of that channel to zero, and remeasure the classification accuracy and attack success rate on the same sample set; Calculate the decrease in classification accuracy and the decrease in attack success rate after blocking the channel; Multiply the decrease in classification accuracy by the first weighting coefficient, multiply the decrease in attack success rate by the second weighting coefficient, and subtract the two to obtain the channel contribution score. Based on the contribution score, a batch normalization mask is set to zero for low contribution channels to achieve structured pruning; The pruned model was fine-tuned using clean samples to restore the recognition performance on the clean samples; The pruning and fine-tuning process is performed iteratively until the model's performance on the electromagnetic signal modulation recognition task is stable.

2. The method according to claim 1, characterized in that, The process of performing structured pruning includes: Collect contribution scores for all channels in all batch normalized layers; The scores are globally sorted from lowest to highest. Based on the preset pruning ratio, select multiple channels ranked at the top from the sort list, so that the proportion of the number of selected channels to the total number of channels is equal to the pruning ratio. For each selected channel, its mask parameter in the batch normalization layer is permanently set to zero, thereby masking the channel during model forward propagation.

3. The method according to claim 1, characterized in that, The process of fine-tuning the pruned model includes: Use a clean sample set as training data; The loss function is defined as the cross-entropy loss between the model output and the true label; During backpropagation, only the gradients of the unmasked model parameters are calculated and updated, while the gradients of the masked channel parameters are zero and are not updated. The training process continues until the loss function converges or the preset number of iterations is reached.

4. The method according to claim 1, characterized in that, The iterative process of pruning and fine-tuning includes: After completing one round of pruning and fine-tuning, the classification accuracy of clean samples and the attack success rate of spoofed samples are calculated on the validation set. Compare the current performance with the performance of the previous iteration. If the change in classification accuracy is less than a preset threshold and the attack success rate is lower than the target value, then terminate the process. Otherwise, perform the contribution score calculation, pruning, and fine-tuning again with the same or more conservative pruning ratio.

5. A batch normalized mask pruning defense system based on channel contribution, characterized in that, The system for implementing the method according to any one of claims 1-4 comprises: The sample generation module is used to generate pseudo-poisoning samples from a small number of clean samples using trigger inversion technology. The contribution calculation module is used to perform a masking operation on the batch normalization layer of the model channel by channel based on the clean samples and the pseudo-poisoned samples, and measure the changes in the classification accuracy of clean samples and the attack success rate of pseudo-poisoned samples before and after the mask is set to zero, so as to calculate the contribution score of each channel. The pruning module is used to perform batch normalization masking to zero operations on low contribution channels according to the contribution score, thereby realizing structured pruning; The fine-tuning module is used to fine-tune the pruned model using clean samples to restore the recognition performance on the clean samples. The iterative control module is used to iteratively execute the pruning and fine-tuning process until the model performance is stable.

6. A computer terminal device, characterized in that, include: One or more processors; A memory, coupled to the processor, for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the steps of the method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1-4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1-4.

Images