A model backdoor attack countermeasure based on frequency domain feature fusion reconstruction

Abstract

The application discloses a model backdoor attack confrontation method based on frequency domain feature fusion reconstruction. The method filters the feature map in the frequency domain by using Fourier convolution on the feature map set of the student model to remove the backdoor attack mode injected in the time domain; semantic information is cascaded and fused from the deep feature map to the shallow feature map in sequence, so that the output of the whole student model becomes a whole, thereby increasing the semantic information that can be learned by the student model in the process of matching the feature map of the teacher model and weakening the attack backdoor based on local information that may exist; attention operation is used on the fused feature map to enhance the shallow semantic information density by using deep high-order semantic information between adjacent output feature map layers, thereby improving the learning ability of the student model and obtaining higher training precision. The application can learn and obtain a student model with high precision and capable of removing the time domain attack backdoor and the attack backdoor based on local information on the basis of a pre-trained model from an untrustworthy source.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of information security and relates to a model backdoor attack countermeasure method based on frequency domain feature fusion reconstruction. BACKGROUND

[0002] With the rapid development of deep learning and neural network models, artificial intelligence systems have begun to play an increasingly important role in daily life. For terminal users who are relatively short of computing resources and data sets, they need to download pre-trained models from the Internet, and then fine-tune these models according to their own needs (He T, Zhang Z, Zhang H, et al. Bag of tricks for image classification with convolutional neural networks [C] / / Proceedings of the IEEE / CVF conference on Computer Vision and Pattern Recognition (CVPR). 2019: 558-567.) or knowledge distillation (Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network [J]. arXiv preprint arXiv: 1503.02531, 2015.) in order to migrate these models to specific tasks. The artificial intelligence models obtained in this way not only can achieve good generalization performance, but also can obtain the model size required by the user, reduce the computing cost, and at the same time achieve good performance in specific tasks.

[0003] However, the pre-trained models published on the Internet are at risk of being implanted with malicious backdoor patterns. Attackers can make pre-trained models exhibit predefined behaviors such as performance degradation, purposeful damage, or erroneous decisions when encountering certain input features through specific training methods. Backdoor attacks on artificial intelligence models are highly targeted, and when the model application scenario contains a backdoor activation factor, it will cause unexpected errors to occur (Liu Y, Wen R, He X, et al. {ML-Doctor}: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models [C] / / 31st USENIX Security Symposium (USENIX Security 22). 2022: 4525-4542.). In addition to attackers directly publishing models with backdoors, some pre-trained models published on regular channels may also have their download links hijacked to spread malicious models to end users to achieve the purpose of attack. Therefore, artificial intelligence models downloaded from the Internet need to be processed as necessary to ensure safety before use.

[0004] Currently, applying an optimized knowledge distillation method to process victim models is one of the effective means to counter artificial intelligence model backdoor attacks (Kim J, Lee B K, Ro Y M. Distilling robust and non-robust features in adversarial examples by information bottleneck [J]. Advances in Neural Information Processing Systems, (NIPS) 2021, 34: 17148-17159.). The existing technical solutions mainly learn to match each feature pixel in the output, and take measures to reduce the possibility of backdoor propagation during the learning process. The advantage of these methods is that they can resist learning of all pixels for untrusted models, which helps to remove trigger patterns that work in the overall range. However, since the selected resistance pattern is effective in the full range of input features, these methods have poor protection effect for backdoor attacks that use small-range semantic information as triggers.

[0005] The attention mechanism is a global weighting information interaction mechanism, via which the information processed can obtain the correlation relationship within the information, give the correlation weight of the element within it and the remaining elements (Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need [J]. Advances in Neural Information Processing Systems (NIPS), 2017, 30.), and the mechanism can collect and enhance the semantic information of the deep learning model in the global range.

[0006] For the backdoor attack mode injected in the time domain, the common backdoor attack countermeasures can also be processed (Li Y, Lyu X, Koren N, et al. Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks [C] / / International Conference on Learning Representations. 2021), but there is a phenomenon of reducing the original performance of the model. SUMMARY

[0007] In view of the problems in the prior art, the purpose of the present application is to provide a model backdoor attack countermeasure method based on frequency domain feature fusion reconstruction.

[0008] For the neural network pre-training model backdoor attack injected in the time domain, the present application proposes to use Fourier convolution to filter the feature map in the frequency domain, filter the time domain attack trigger mode, so that the pre-training model can be safely learned; Fourier transform is a commonly used signal processing technology, which can convert time domain signals into frequency domain signals, so that the original signal can be filtered and processed in the frequency domain, that is, some frequency domain components can be selectively filtered out or retained in the frequency domain, and then Fourier inverse transform is performed to obtain the restored filtered time domain signal, so that some operations that are difficult to achieve in the time domain are achieved. The partial signal feature mode in the time domain is clearer when observed in the frequency domain, so the potential backdoor attack trigger mode can be processed in the frequency domain.

[0009] For the neural network pre-training model backdoor attack based on local information, the present application proposes a layer interconnection semantic information fusion method to fuse the feature map information of different levels output by the learning training model, form the overall semantic ability, and resist the neural network pre-training model backdoor attack based on local information.

[0010] In order to solve the problem of the accuracy reduction of the neural network model caused by the training mode resisting backdoor attacks, the application uses an inter-layer attention mechanism to cascade and correlate the output of different layers of a to-be-learned training model, and uses high-order semantic information from adjacent deep feature maps to weight and fuse the semantic information of shallow feature maps, so that the semantic information aggregation degree can be improved, and stronger model accuracy can be learned in the training process.

[0011] To achieve the above object, the application realizes the technical scheme as follows: a model backdoor attack countermeasure method based on frequency domain feature fusion reconstruction, comprising the following steps:

[0012] (1) selecting and initializing a large model corresponding to a superior application task of a target application task as a teacher model M tea , downloading and using a teacher model M tea stored by a public platform, covering the parameter weights of the teacher model M tea with the incompletely trusted pre-training weights, fixing the parameter weights of the teacher model M tea after the covering is completed, selecting and initializing a small model with the same output layer number as M tea or a model consistent with the teacher model M tea as a student model M stu ; for example, when the target application task is a dense small target detection task or a multi-instance small target segmentation computer vision downstream task, the superior application task is generally an image classification task, at this time, the large model can refer to a BASIC-L, ViT-e, etc. image classification neural network model with a large-scale dataset for training and a large number of parameters;

[0013] (2) inputting the target application task dataset into the teacher model M tea and the student model M stu respectively, obtaining feature maps on the intermediate layers at the same stage, generating a teacher feature map set Feats tea and a student feature map set Feats stu , and finally outputting a probability distribution Logits tea and Logits stu ;

[0014] (3) performing size alignment on a deeper layer student feature map Feats stu [l+1] to match a shallower layer feature map Feats stu [l], wherein l represents a feature stage, and after the size alignment of the two time domain feature maps, they are jointly input into a Fourier convolution layer;

[0015] (4) in the Fourier convolution layer, the feature map Feats stu[l+1] and the shallower layer of feature maps Feats stu [l] are processed into frequency domain respectively, and are marked as Fourier g [l] and Fourier l [l], where the more global high-order features are relative to the current l-th layer of features, and g is used to identify that it is a global feature, and l represents a local feature. The high-order and low-order information in the frequency domain is cross- weighted to complete the fusion of Feats stu [l+1] contains high-order semantic information injection fusion into Feats stu [l] information fusion, while using Feats stu [l] contains detailed semantic information to Feats stu [l+1] is supplemented, so that the information between the two layers of feature maps is fused in the frequency domain; in the cross-weighting calculation process, convolution filtering is performed in the frequency domain at the same time. The convolution filter is a filter with learnable parameters, which can be learned by Fourier g [l] and Fourier l [l] features are filtered to remove the abnormal state of the time-domain backdoor trigger mode in the frequency domain, avoiding M tea transmit the time-domain backdoor attack trigger mode, and the processed frequency domain features Fourier g [l] and Fourier l [l] are restored to the time domain using the inverse transform of the discrete Fourier transform, and the feature map Fourier global [l] and Fourier local [l] are two time-domain feature maps.

[0016] (5) The fused feature map Fourier global [l] and Fourier local [l] global semantic information attention weight calculation, where Fourier global [l] is used as the query vector Q and the key vector K, and Fourier local [l] is used as the value vector V to complete the self-attention operator calculation, and the output is the self-attention feature map Feats attn [l];

[0017] (6) From the deepest M stu feature map Feats stuStarting at [L], steps (3), (4), and (5) are executed in the shallow cascade. During the processing at the beginning of the [l-1]th level, the deeper feature maps Feats in the Fourier convolutional layer input are processed. stu [l] Fourier transforms generated by fusion via Fourier convolutional layers in the l-th stage of processing. local [l] features are replaced, meaning that in the processing at levels [l-1] and beyond, the input to the Fourier convolutional layer is the current layer's Feats. stu [l-1] and the local part of the output of a deeper Fourier convolutional layer local [l];

[0018] (7) For each i-th level of the final obtained Feats, sequentially... tea [l] and Feats attn [l] Perform channel size alignment using the same technique as in step (3), where l∈[1,L-1] and L is M. tea The output shows the total number of layers, which is achieved by aligning the number of channels and dimensions of the two feature maps using convolutional layers and interpolation functions. Then, the Kullback-Leible divergence cross-entropy loss function is used to calculate and sum the losses for the two feature maps corresponding to each layer, and finally, the Logits are calculated. stu Calculate the loss according to the objective function specific to the target application task;

[0019] (8) Calculate the total loss by weighted summation of the different losses and backfeed it to the training M. stu Convergence yields a student model that is highly accurate, secure, and free from potential backdoor attack trigger patterns.

[0020] As demonstrated by the training process of this method, it can transform the student model output to the frequency domain during knowledge distillation and transfer, thus addressing the hidden information in the publicly available M... tea The time-domain backdoor attack trigger patterns in the feature map output generated by the pre-trained weights are filtered in the frequency domain to remove the time-domain backdoor. For backdoor attacks on neural network pre-trained models based on local information, this method utilizes inter-layer cascaded semantic information fusion to fuse the feature map information of adjacent layers in the output of the model to be trained. Shallow detailed features are applied to refine and improve deep global features, and deep high-order semantic information is injected into shallow features to enhance expressive power, thus forming an overall semantic capability. Cascading feature layers sequentially connects the model output into a whole, using the cascaded fused overall state to evaluate the M... tea To learn and avoid M teaThe characteristic information is learned one-to-one, so the neural network pre-training model backdoor attack based on local information can be resisted; in summary, the method can implement safe migration processing on the teacher model of the non-safe domain, and obtain a student model network with high precision and without backdoor.

[0021] Further, the pre-training model downloading platform of the predetermined model in step (1) selects an open source or commercial pre-training model weight providing platform, but the training data source, training method and training parameter of the model weight are regarded as untrusted sources for subsequent processing; the student model M stu The model capability is equal to or weaker than the teacher model M tea .

[0022] Further, the target application task data set in step (2) is a student model M stu The data set to be applied by the user is organized by the user himself, so it is regarded as a trusted data set, and the scale and annotation accuracy are weaker than the data set used by the pre-training model;

[0023] The intermediate level of each stage of the model refers to the neural network model processing flow, which has a more obvious reusable paradigm, extracts the final output of each reusable paradigm as the output of a layer to obtain the complete performance of the reusable paradigm and reduce the total amount of feature maps to speed up the calculation;

[0024] The final output probability distribution Logits stu and Logits tea is the estimated value of the correct probability of each class before the neural network model outputs the final result.

[0025] Further, the feature map alignment operation in step (3) includes two operations: channel alignment and feature map scale alignment, the channel alignment operation is a convolution layer with a convolution kernel size of 1x1 and a step of 1 for channel processing of a deeper layer of feature maps Feats stu [l+1], which can improve semantic information by fully connecting the internal information of the layer and align the channel number to a shallower layer of feature maps;

[0026] The feature map scale alignment operation is to scale the deeper layer of feature maps Feats stu [l+1] using an interpolation function, and the interpolation function is determined by the selected interpolation mode.

[0027] Further, the Fourier convolution layer in step (4) is a deep learning operator that can split the input into local and global abstractions for local and global processing, which is optimized for application in the present application, and the deeper layer of pyramid feature map output Featsstu [l+1] as global abstraction, output Feats from shallower layer stu [l] as local abstraction, operate on;

[0028] The processing of the feature map into a frequency domain form refers to converting time domain features into a frequency domain form using a two-dimensional real number field discrete Fourier transform for the last two dimensions of features of the feature map;

[0029] The cross weighting calculation is an improved operation mode on a Fourier convolution operator, and the formula of the operation mode is as follows:

[0030] Y l = Y l→l + Y g→l = f l (X l ) + f g→l (X g ),

[0031] Y g = Y g→g + Y l→g = f g (X g ) + f l→g (X l )

[0032] Where Y represents output, X represents input, l and g represent local and global information, and the f function with a subscript is a processing function for local and global, respectively, and the left and right sides of the '→' symbol represent the information hierarchy to which the information before and after the execution of the processing belongs;

[0033] The f processing function is a frequency domain convolution filtering function, specifically a convolution kernel size of 1*1 and a convolution kernel number of a convolution layer of the number of feature channels processed, which can strongly activate the existing time domain backdoor trigger mode in the frequency domain. The function is composed of learnable training parameters, and the abnormality in the teacher model is found during the training process of the student model, and the frequency domain disturbance caused by the local feature is weakened through the global matching item of the final loss function, so that the purpose of resisting backdoor attacks is achieved;

[0034] The information fusion between the two levels is completed in the optimized Fourier convolution, and the fusion manner is to fuse the filtered global, local, local to global, and global to local frequency domain features according to the calculation formulas of Y l and Y g to obtain the frequency domain fused feature map Fourier g [l] and Fourier l [l];

[0035] The method for restoring the frequency domain features to the time domain is an inverse operation of two-dimensional real number domain discrete Fourier transform, which is applied to the last two dimensions of the frequency domain output, and can convert the output back to the time domain for subsequent operation.

[0036] Further, the attention mechanism in the step (5) is a mechanism for weighting and obtaining weights of multiple targets, and the formula is as follows:

[0037]

[0038] wherein Q and K are initialized using Fourier global [l], and V is initialized using Fourier local [l], d k is the number of channels, used to control the model scale.

[0039] In order to improve the operation speed, a multi-head attention mechanism is used to accelerate the calculation, and the multi-head attention can use the parallel computing capability of a graphics computing unit to calculate and obtain acceleration, and the formula is as follows:

[0040] MultiHeadAttention(Q,K,V)=Concat(head1,…,head h )W O

[0041]

[0042] wherein the projection weight is a parameter matrix

[0043] The global and the local are not only the global and the local information abstraction under the definition of the Fourier convolution layer, but also the deep and shallow layers on the feature map level. From the condensation degree of information, the deep feature map can be regarded as the global feature in comparison with the information contained in the pixel.

[0044] Further, the execution step in the step (6) is performed from the deep layer to the shallow layer in the L-order output feature map Feats stu set of the student model, and in the first level fusion, the input is the deep one layer feature map Feats stu [L] and the shallow one layer feature map Feats stu [L-1] are initialized, and the deeper one layer feature map in the input of the subsequent execution round [l-1] is replaced by Fourier local [i] obtained in the last round.

[0045] Further, the Kullback-Leible divergence mutual entropy objective function in the step (7) is a common objective function for fitting distribution, and the formula is as follows:

[0046]

[0047] Wherein, p and q are approximate probability distributions, which are Feats tea [l] and Feats attn respectively in the method scenario. tea L is the total number of layers of Feats l , and lambda stu is an optional parameter, which can set the proportion of the loss of different layer features in the total loss.

[0048] The Logits loss are the original target function, which is the target function applied to the application task required by the user, namely Custom l , which is defined and calculated by the user.

[0049] Further, the weighting parameter in the step (8) includes the layer weight parameter lambda loss defined and used in the step (7) and the loss alpha controlling the proportion of the distillation part and the original target loss, and the formula is as follows:

[0050] Total loss = alpha x Distill loss

[0051] The student model of the backpropagation training is a method for optimizing the model using the error backpropagation optimization algorithm, and the selected optimization function can be defined by the user.

[0052] The advantages of the present application are as follows:

[0053] 1. Using Fourier convolution to convert the model output to the frequency domain for processing, which can eliminate part of the backdoor trigger response mode in the time domain feature which is not obvious but can directly lead to the change in the frequency domain;

[0054] 2. Using self-attention mechanism to process the feature map from the Fourier convolution, which can perform self-attention fusion on the semantic information in the frequency domain, giving stronger boundary information, which helps to improve the model expression effect;

[0055] 3. Cascade processing of information of different levels can form an inverse pyramid structure, which can supplement the forward pyramid information of the original model output, and the reverse transmission process can also help the low-level model to obtain high-order semantic information, which helps to improve the model learning ability, thereby improving the model precision;

[0056] 4. Using the present application, a student model with high precision and capable of removing time domain attack backdoors and local information-based attack backdoors can be learned based on a pre-trained model from an untrusted source. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The present application is a method flowchart.

[0058] Figure 2 The present application is an improved Fourier convolution information flow diagram. DETAILED DESCRIPTION

[0059] The present application will be further described in detail below with reference to the accompanying drawings, which are provided only to explain the present application and are not intended to limit the scope of the present application.

[0060] The model backdoor attack countermeasure method based on frequency domain feature fusion reconstruction provided by the embodiments of the present application is as shown in Figure 1 .

[0061] The teacher model applied in the present embodiment is ResNet101, the student model is ResNet18, and the user dataset is a high-resolution image classification dataset consistent with the ImageNet format, including the following steps:

[0062] (1) Initialize the teacher model and the student model, and use the batched user dataset to perform inference to obtain the teacher feature map set Feats tea and the student feature map set Feats stu , and the final output probability distribution Logits tea and Logits stu . Specifically:

[0063] (1a) Initialize the ResNet101 model as the teacher model M tea , use the untrusted ResNet101 weight parameters downloaded from the open source platform to cover the parameters, initialize the ResNet18 as the student model M stu , load the ImageNet dataset and initialize the random gradient descent optimizer according to the training parameters;

[0064] (1b) input the dataset into M tea and M stu in turn, respectively calculate the outputs at different levels to obtain the feature map sets Feats stu and Feats tea , and the final output probability distributions Logits stu and Logits tea .

[0065] (2) Channel alignment of the deeper layer feature map Feats stu [l+1] to match the shallower layer feature map Feats stu [l] where l denotes the feature stage, which are jointly input into the Fourier convolution layer after two temporal feature map size alignment; the specific description is as follows:

[0066] (2a) A convolution layer with a 1x1 kernel size and a step of 1 is used to align the channels of the deeper layer output Feats stu [l+1] to the channel number of the shallower layer output Feats stu [l].

[0067] (2b) The bilinear interpolation mode interpolation function is used to scale the deeper layer output Feats stu [l+1] after channel alignment to obtain the same feature map size as Feats stu [l].

[0068] (3) Fourier convolution layer processing is performed to remove the temporal backdoor in the frequency domain, as follows:

[0069] (3a) In the Fourier convolution layer, the feature map Feats stu [l+1] from the deeper layer and the feature map Feats stu [l] from the shallower layer are processed into frequency domain forms, respectively, and are marked as Fourier g [l] and Fourier l [l], respectively, where the more global high-order features relative to the current l-th layer features, so g is used to identify the global features, and the current layer is used to identify more detailed information, and l is used to identify the local features.

[0070] (3b) Cross weighting calculation is performed on the high-level and low-level information in the frequency domain to complete the information fusion of the high-order semantic information contained in Feats stu [l+1] into Feats stu [l], while using the detailed semantic information contained in Feats stu [l] to supplement Feats stu [l+1], so as to fuse the information between the two layer feature maps in the frequency domain, where the cross weighting calculation rule is as shown in Figure 2 , and the formula is as follows:

[0071] Y l = Y l→l + Y g→l = f l (X l ) + fg→l (X g ),

[0072] Y g =Y g→g +Y l→g =f g (X g )+f l→g (X l )

[0073] where Y represents output, X represents input, and l and g represent local and global information, respectively, and the subscripted f function is a processing function for local and global, respectively, and the left and right sides of the '→' symbol represent the information hierarchy to which the information before and after processing belongs, respectively;

[0074] (3c) In the cross weighting calculation process, convolution filtering processing is performed in the frequency domain. The convolution filter is a filter with learnable parameters, which can be filtered by Fourier g [l] and Fourier l [l] features to remove the abnormal state of the time-domain backdoor trigger mode in the frequency domain, avoiding the subsequent learning process of M tea passing the time-domain backdoor attack trigger mode;

[0075] (3d) After processing, the frequency domain feature Fourier g [l] is restored to the time domain using the inverse transform of the discrete Fourier transform, and the transform scope is the last two dimensions of the feature map. The output is the feature map Fourier l [l] and Fourier global [l] after frequency domain semantic information fusion and convolution filtering. local [l] two time domain feature maps.

[0076] (4) Use the self-attention mechanism to calculate the global semantic information attention weight of the fused feature map Fourier global [l] and Fourier local [l], where Fourier global [l] with high-order global information is used as the query vector Q and the key vector K, and Fourier local [l] with low-order detailed information is used as the value vector V to complete the self-attention operator calculation, and the output is the attention feature map Feats attn [l]; The specific description is as follows:

[0077] (4a) Use the attention mechanism of the following formula to calculate Fourier global [l] and Fourierlocal [1] Process to obtain the interaction weight of local and global information:

[0078]

[0079] Where Q and K use Fourier global [1] Initialization, V uses Fourier local [1] Initialization, d k is the number of channels to control the model scale;

[0080] (4b) In order to improve the operation speed, the multi-head attention mechanism is used to speed up the calculation, and the multi-head attention can use the parallel computing ability of the graphics computing unit to calculate to obtain acceleration, and the formula is as follows:

[0081] MultiHeadAttention(Q,K,V)=Concat(head1,…,head h )W O

[0082]

[0083] Where the projection weight is the parameter matrix

[0084] (5) From the deepest M stu feature map Feats stu [L] Start, perform steps (2), (3), (4) to the shallow layer, wherein in the process of starting the i-1 level processing, the deeper layer feature map in the input of the Fourier convolution layer is replaced by the Fourier local [l] feature generated by the Fourier convolution layer fusion in the i level processing, that is, in the i-1 level and subsequent processing, the input of the Fourier convolution layer is the current layer Feats stu [l-1] and the local part of the Fourier local [l] in the output of the deeper Fourier convolution layer.

[0085] (6) Use the same technology as step (3) to align the channel size of each level Feats tea [l] and Feats attn [l] obtained finally, wherein l∈[1,L-1], that is, use the convolution layer and the interpolation function to align the size and channel number of the two feature maps, then use the Kullback-Leible divergence mutual entropy loss function to calculate the loss of the two feature maps corresponding to each level l in turn and sum, and calculate Logits stuThe loss is calculated according to the target function specific to the target application task; the specific description is as follows:

[0086] (6a) Kullback-Leible divergence mutual entropy target function is a target function for fitting different distributions, the formula is as follows, where all the λ l is set to 1:

[0087]

[0088] Where p and q are approximate probability distributions, respectively, in the context of the present method, i.e., Feats tea and Feats attn are feature maps, L is the total number of Feats tea layers, λ l is an optional parameter, which can set the proportion of loss of different layer features in the total loss;

[0089] (6b) Calculate the classification target function loss for Logits stu , the loss function used is the multi-class cross-entropy loss, the formula is as follows:

[0090]

[0091] Where M represents the number of categories, N is the total number of samples, y ic is a symbolic function (0 or 1), if the real category of sample i is equal to c, then y ic takes 1, otherwise y ic takes 0, p ic is the predicted probability that sample i belongs to category c.

[0092] (7) The total loss is obtained by weighting the sum of different losses and returned to the student model for convergence, the total loss calculation formula is as follows:

[0093] Total loss = 0.5 x Distill loss + Custom loss

[0094] After calculating the total loss, use the stochastic gradient descent algorithm to update the gradient and optimize M stu until the model loss converges, then a high-precision ResNet18 model with safe backdoor removal can be obtained.

[0095] While specific embodiments of the application have been disclosed in order to illustrate the application and to assist those skilled in the art in practicing the application, it is to be understood that various substitutions, modifications and changes can be made by those skilled in the art without departing from the spirit of the application and the scope of the appended claims. Accordingly, it is intended that the application not be limited, except by the scope of the claims.

Claims

1. A method for countering backdoor attacks based on frequency domain feature fusion and reconstruction, comprising the following steps: 1) Select and initialize a model corresponding to the parent application task of the target application task as the teacher model M. tea Obtain the teacher model M tea The pre-trained model weights for the teacher model M tea The parameter weights are covered; a model is selected and initialized as the student model M. stu The student model M stu The number of output levels and the teacher model M tea Consistent; 2) Input a sample from the target application task dataset into the teacher model M. tea And student model M stu Among them, the teacher model M was obtained. tea Feats is a set of feature maps composed of the feature maps output from each intermediate level. tea And teacher model M tea The probability distribution Logits of the last layer output tea The student model M is obtained. stu Feats is a set of feature maps composed of the feature maps output from each intermediate level. stu And obtain the student model M stu The probability distribution Logits of the last layer output stu ; 3) Starting from the layer above the deepest intermediate layer, construct the student model M. stu Feats, the feature map output from the (l+1)th intermediate layer stu [l+1] and the feature map output of the l-th intermediate layer (Feats) stu [l] After size alignment, they are all input into the Fourier convolutional layer; the Fourier convolutional layer processes the feature maps Feats stu [l+1], Feats stu [l] After conversion to the frequency domain, information fusion is performed to convert Feats stu [l+1] contains higher-order semantic information that is injected and fused into Feats. stu The global frequency domain feature map of the l-th intermediate layer is obtained from [l]. Fourier g [l], and the use of Feats stu The detailed semantic information contained in [l] is relevant to Feats stu [l+1] is used to supplement the local frequency domain feature map of the l-th intermediate layer. Fourier l [l], then to Fourier g [l] with Fourier l [l] Perform inverse transforms to restore the time domain, obtaining the global time-domain feature map of the l-th intermediate layer (Fourier). global [l] and the local temporal feature map of the l-th intermediate layer Fourier local [l]; then, using the self-attention mechanism, Fourier global [l]、Fourier local [l] Perform global semantic information attention weight calculation to obtain the self-attention feature map Feats. attn [l], and in the next level of processing the (l-1)th layer, the local temporal feature map Fourier from the output of the lth layer is utilized. local [l] Replace the feature map set Feats used as input to the Fourier convolutional layer for high-order semantic information. stu Feats, the feature map output from the (l-1)th intermediate layer stu [l-1]; Where l∈[1,L-1], and L is the number of intermediate layers; 4) For the feature map set Feats tea For each feature map, the feature map output from the l-th intermediate layer is used as a reference. tea [l] and the corresponding self-attention feature map Feats attn After alignment, the loss of the l-th intermediate layer is calculated using the Kullback-Leible divergence cross-entropy loss function. Then, the losses of each intermediate layer are summed to obtain the Distillation loss value. loss Based on the Logits corresponding to each sample in the target application task dataset. stu The classification loss value Custom is calculated based on the true label corresponding to the sample. loss ; 5) Based on the loss value Distill loss and loss value Custom loss The calculated total loss is Total loss Optimize the student model M stu ; 6) Iterate and repeat steps 2)-5) until the student model M is obtained. stu Convergence yields a secure model free of potential backdoors.

2. The method according to claim 1, characterized in that, Total loss =α×Distill loss +Custom loss α is a proportionality coefficient.

3. The method according to claim 1 or 2, characterized in that, The loss value Custom is calculated using the target application task loss function. loss .

4. The method according to claim 1, characterized in that, Using the self-attention mechanism, Fourier global [l] is used as the query vector Q and the keyword vector K, and Fourier local [l] is used as the value vector V for self-attention calculation, and the self-attention feature map Feats is output. attn [l].

5. The method according to claim 1, characterized in that, The target application task dataset is a reliable dataset, but it is weaker than the dataset used by the pre-trained model in terms of scale and annotation accuracy.

6. The method according to claim 1, characterized in that, The student model M stu In terms of model capability, it is equal to or weaker than the teacher model M. tea .

7. The method according to claim 1, characterized in that, Based on total loss loss The student model M was optimized using the backpropagation algorithm. stu Optimize.

8. A server, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing each step of the method of any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images