Adversarial training method based on image reconstruction and feature fusion

Abstract

The invention discloses an adversarial training method based on image reconstruction and feature fusion, and the method comprises the steps: obtaining an image sample containing a class label, generating an adversarial sample, and enabling a normal sample and the generated adversarial sample to form a training set X0; the method comprises the following steps: firstly, adding random noise to all samples in a training set X0 to obtain a training set X1, and inputting the training set X1 into a preprocessor; then, after the training set X1 is processed by an image reconstruction module in the preprocessor, the training set X1 and the training set X1 are input into a feature fusion module in the preprocessor; and finally, the output of the feature fusion module in the preprocessor is sent to a target classification model, joint training is carried out on the preprocessor and the target classification model, a loss function is optimized, and the trained target classification model is obtained, so that the target classification model keeps relatively high classification accuracy for normal samples, and meanwhile, the classification accuracy is improved. And the robustness, the consistency and the generalization ability of the model can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an adversarial training method based on image reconstruction and feature fusion. Background Art

[0002] The wide application of deep learning models covers many fields such as image classification, speech recognition, and natural language processing. In the field of smart healthcare, image classification is widely used in disease diagnosis and treatment. For example, deep learning models can be used to identify and classify various diseases from X-rays, CT scans, or MRI images. Self-driving cars developed by companies such as Google, Tesla, and Baidu have extensively applied deep learning technology, such as video-based perception and decision-making, speech recognition control, and multi-sensory and multi-modal fusion perception and decision-making. In the field of financial security, face recognition has also been widely used as a biometric authentication technology. These models learn complex feature representations and decision boundaries through training and optimization of large-scale data. However, studies have shown that deep learning models are prone to misjudgment or misclassification when faced with slightly modified inputs. These small perturbations can be intentionally made adversarial samples or they can be generated unintentionally. The application of deep learning models in key areas, such as autonomous driving and financial security, any wrong decision may bring huge risks. Therefore, the research on adversarial sample attack defense technology is of great significance.

[0003] Research shows that one of the reasons for the existence of adversarial samples is the discontinuity of input-output mapping caused by the high nonlinearity of deep learning models. In recent years, a large number of scholars have conducted in-depth research on the security issues of deep learning models from the perspective of adversarial defense. In terms of active defense against image adversarial sample attacks, defense distillation is a method of compressing the model while ensuring training accuracy. In addition, various adversarial sample detection and defense methods have been proposed: feature learning (principal component analysis, feature compression, etc.), distribution statistics (softmax distribution, kernel density and uncertainty estimation, etc.), input reconstruction, etc. From the perspective of image denoising, defense methods such as feature denoising, JPEG compression, total variance minimization, HGD denoiser, and Defense-GAN are proposed for image preprocessing and feature transformation.

[0004] Adversarial training is one of the effective active defense methods against adversarial attacks. By adding adversarial samples to the training dataset to form an incremental set, the robustness of the target model can be significantly improved. Typical adversarial training methods include PGD-based adversarial training, TRADES, etc. However, current adversarial training methods also have some limitations: on the one hand, excessive attention to adversarial samples may lead to a decrease in the classification performance of normal samples; on the other hand, the model is not adaptable enough when facing unknown types of attacks. Summary of the invention

[0005] The purpose of the present invention is to provide an adversarial training method based on image reconstruction and feature fusion, which can effectively improve the robustness (i.e., the ability to resist adversarial attacks), consistency and generalization ability (i.e., the ability to perform in the face of unknown attacks) of the target classification model while ensuring the accuracy of the original samples as much as possible when facing image adversarial sample attacks.

[0006] In order to achieve the above-mentioned object of the invention, the embodiment of the present invention provides the following technical solutions: An adversarial training method based on image reconstruction and feature fusion includes the following steps: Step S1, obtaining a normal sample, wherein the normal sample is an image sample containing a category label; generating an adversarial sample based on the obtained normal sample; and forming a training set X0 with the obtained normal sample and the generated adversarial sample; Step S2, first, add random noise to all samples in the training set X0 to obtain the training set X1, and input it into a preprocessor; the preprocessor includes an image reconstruction module and a feature fusion module; Step S3, then, after the training set X1 is processed by the image reconstruction module in the preprocessor, it is input into the feature fusion module in the preprocessor together with the training set X1; Step S4, finally, the output of the feature fusion module in the preprocessor is sent to the target classification model, the preprocessor and the target classification model are jointly trained, the loss function is optimized, and a trained target classification model is obtained.

[0007] In the adversarial training method based on image reconstruction and feature fusion, step S3 further includes: The image reconstruction module in the preprocessor includes wavelet transform, energy threshold estimation, filtering processing and wavelet inverse transform modules. First, the input data is transformed into a square through a transformation function, and a discrete wavelet transform is performed to decompose the signal into k orthogonal sub-bands, and the detail coefficients and scale coefficients in the horizontal, vertical and diagonal directions are extracted to form a multi-resolution spectrum representation. Then, the frequency band energy threshold is estimated, and the frequency band energy refers to the square sum of the frequency band coefficients. After the energy value is normalized, that is, the range is set to [0, 1], the adaptive filtering threshold Tau is defined as (1 / k)*exp(-Lambda*(Mu / Delta)), wherein Mu and Delta represent the mean and standard deviation of the frequency band energy respectively, and Lambda is a hyperparameter. According to the calculated adaptive filtering threshold Tau, the detail coefficient is thresholded; then, filtering processing is performed, and the transformed coefficient is smoothed by a Gaussian filter; finally, an inverse wavelet transform is performed to reconstruct the processed detail coefficients back to the original space to obtain a reconstructed image.

[0008] In the adversarial training method based on image reconstruction and feature fusion, step S3 further includes: The feature fusion module in the preprocessor adopts an improved denoising convolutional neural network model; the improved denoising convolutional neural network model introduces the NAFNet (Nonlinear Activation Free Network) module and the PSA (Pyramid Squeeze Attention) module on the basis of the denoising convolutional neural network model DenoiseCNN, and designs 4 channel structures for extracting and fusing features from different scales and directions; first, downsampling is performed through the Conv+Relu module to extract high-dimensional features of the image; then, the NAFNet module is used to achieve efficient feature extraction; then, features are extracted and fused from different scales and directions through 4 channel structures; then, adaptive feature fusion is performed through the PSA module to enhance the expression ability of the features; finally, the original size image is generated through bilinear interpolation upsampling and Conv+Sigmoid; The four channel structures for extracting and fusing features from different scales and directions, the first channel uses 1×1 convolution to adjust the channel and compress the features; the second channel uses two 3×3 convolutions to increase the receptive field and capture more local spatial features while maintaining a small computational overhead; the third channel uses a combination of 1×7 and 7×1 convolutions, that is, long convolution kernels in different directions, which helps to capture feature information in different directions, especially line or edge information; the fourth channel uses DW convolution (Depthwise Convolution) and PW convolution (Pointwise Convolution), DW convolution can separate the spatial and depth information of the channel, reduce computational complexity, and maintain feature richness; PW convolution uses a 1×1 convolution kernel for calculation for each spatial position of the input feature map, and its main function is to reduce dimensionality and increase nonlinearity; finally, the features of different scales of the four channels are fused.

[0009] In the adversarial training method based on image reconstruction and feature fusion, step S4 further includes: The loss function consists of two parts: preprocessor loss and classification loss; the preprocessor loss adopts mean square error loss, which consists of three parts: the similarity between the original input and the preprocessed sample, the similarity between the normal sample and the adversarial sample image after reconstruction, and the similarity between the normal sample and the adversarial sample after feature fusion; the classification loss adopts cross entropy loss.

[0010] Compared with the prior art, the present invention has the following advantages: (1) The feature fusion module in the preprocessor proposed in the present invention adopts an improved denoising convolutional neural network model. In order to introduce the NAFNet module and the PSA module on the basis of the denoising convolutional neural network model DenoiseCNN, four channel structures are designed to extract and fuse features from different scales and directions. Among them, the NAFNet module can capture rich feature information while ensuring computational efficiency through linear operations and a simplified channel attention mechanism, avoiding the training instability caused by traditional activation functions, making the improved denoising convolutional neural network more efficient and stable; the PSA module can learn attention weights with lower model complexity and effectively integrate local and global attention, which can improve the representation ability of the model when processing complex noise; the four channel structures designed by the present invention to extract and fuse features from different scales and directions can effectively denoise while ensuring rich feature information; (2) The adversarial training method based on image reconstruction and feature fusion proposed in the present invention, wherein the image reconstruction module can effectively filter out most of the disturbances introduced by the adversarial attack by setting an appropriate energy threshold, thereby improving the adversarial robustness of the model; the feature fusion module extracts robust features from the reconstructed image, fuses them with the original features to form a more comprehensive feature representation, and outputs effective robust features in combination with attention; and the preprocessor that integrates the image reconstruction and feature fusion modules is jointly trained with the target classification model, so that the target classification model can fully learn the robust features output by the preprocessor, so that the model can maintain a high classification accuracy for normal samples while effectively improving the robustness, consistency and generalization ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 It is a flow chart of the adversarial training method based on image reconstruction and feature fusion of the present invention.

[0013] Figure 2 It is a structural schematic diagram of the image reconstruction module in the preprocessor of the present invention.

[0014] Figure 3 It is a structural schematic diagram of the feature fusion module in the preprocessor of the present invention.

[0015] Figure 4Schematic diagram of the structure of the denoising convolutional neural network model DenoiseCNN.

[0016] Figure 5 It is a structural diagram of the NAFNet module in the feature fusion module of the present invention.

[0017] Figure 6 It is a structural schematic diagram of the PSA module in the feature fusion module of the present invention.

[0018] Figure 7 This is a robustness comparison test between the method of the present invention and other methods.

[0019] Figure 8 This is a test of the consistency and generalization performance of the method of the present invention. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0021] like Figure 1 As shown, this embodiment provides an adversarial training method based on image reconstruction and feature fusion, comprising the following steps: Step S1, obtaining a normal sample, wherein the normal sample is an image sample containing a category label; generating an adversarial sample based on the obtained normal sample; and forming a training set X0 with the obtained normal sample and the generated adversarial sample; In this embodiment, images containing category labels are obtained from the CIFAR-10 dataset as normal samples, of which the training set has 50,000 images and the test set has 10,000 images. Preferably, the ResNet18 model is used to perform an EoT-PGD (Expectation over Transformation, Projected Gradient Descent) single-step attack on the normal samples, with the maximum perturbation set set to 10 / 255, to generate adversarial samples.

[0022] Step S2, first, add random noise to all samples in the training set X0 to obtain the training set X1, and input it into a preprocessor; the preprocessor includes an image reconstruction module and a feature fusion module; In this embodiment, random noise of varying degrees is added to all samples in the training set X0 to generate multiple noisy variants, and the most suitable version is selected according to the standard deviation of the variants. Preferably, the variant with the smallest standard deviation is selected as the final noise-enhanced sample, thereby obtaining the training set X1.

[0023] Step S3, then, after the training set X1 is processed by the image reconstruction module in the preprocessor, it is input into the feature fusion module in the preprocessor together with the training set X1; The image reconstruction module in the preprocessor, such as Figure 2 As shown, it includes modules such as wavelet transform, energy threshold estimation, filtering processing and inverse wavelet transform. First, the input data is transformed into a square through the transformation function, and a discrete wavelet transform is performed to decompose the signal into k orthogonal sub-bands, and the detail coefficients and scale coefficients in the horizontal, vertical and diagonal directions are extracted to form a multi-resolution spectral representation. Then, the band energy threshold is estimated, and the band energy refers to the sum of the squares of the band coefficients. After the energy value is normalized (the range is set to [0, 1]), the adaptive filtering threshold Tau is defined as (1 / k)*exp(-Lambda*(Mu / Delta)), where Mu and Delta represent the mean and standard deviation of the band energy, respectively, and Lambda is a hyperparameter. According to the calculated adaptive filtering threshold Tau, the detail coefficient is thresholded. Next, filtering processing is performed, and the Gaussian filter smoothing coefficient is used for the transformed coefficient to reduce the influence of high-frequency noise. Finally, an inverse wavelet transform is performed to reconstruct the processed detail coefficients back to the original space to obtain a reconstructed image; The feature fusion module in the preprocessor, such as Figure 3 As shown in the figure, an improved denoising convolutional neural network model is used to implement the denoising convolutional neural network model DenoiseCNN (as shown in Figure 4 As shown in Figure 2, NAFNet (NonlinearActivation Free Network) module is introduced based on Figure 5 As shown) and PSA (Pyramid Squeeze Attention) modules (as shown Figure 6 As shown in Figure 2, four channel structures are designed to extract and fuse features from different scales and directions. Denoising convolutional neural network model DenoiseCNN, as shown in Figure 2 Figure 4As shown, it usually relies on simple Conv (convolutional layer), Relu (activation function), BN (batch normalization layer), and stacks multiple convolution blocks on this basis to enhance the feature extraction capability. The improved denoising convolutional neural network model proposed in this embodiment first performs downsampling through the Conv+Relu module to extract high-dimensional features of the image. Then, the NAFNet module is used to achieve efficient feature extraction. Next, features are extracted and fused from different scales and directions through a 4-channel structure. After that, adaptive feature fusion is performed through the PSA module to enhance the expression capability of the features. Finally, the original size image is generated through bilinear interpolation upsampling and Conv+Sigmoid; The NAFNet module, such as Figure 5 As shown in the figure, through linear operations and simplified channel attention mechanism (SCA), while ensuring computational efficiency, it is possible to capture rich feature information, avoid the training instability caused by traditional activation functions, and make the improved denoising convolutional neural network more efficient and stable; The four channel structures for extracting and fusing features from different scales and directions are as follows: Figure 3 As shown, the first channel uses 1×1 convolution to adjust the channel and compress the features. The second channel uses two 3×3 convolutions to increase the receptive field and capture more local spatial features while maintaining a small computational overhead. The third channel uses a combination of 1×7 and 7×1 convolutions, that is, long convolution kernels in different directions, which helps to capture feature information in different directions, especially line or edge information. The fourth channel uses DW convolution (Depthwise Convolution) and PW convolution (Pointwise Convolution). DW convolution can separate the spatial and depth information of the channel, reduce computational complexity, and maintain the richness of features; PW convolution uses a 1×1 convolution kernel for calculation of each spatial position of the input feature map, and its main function is to reduce dimensionality and increase nonlinearity. Finally, the features of different scales of the four channels are fused to output a more diverse and hierarchical feature representation. The channel structure designed in this embodiment can ensure feature information while denoising; The PSA module, such as Figure 6As shown, the channel is first divided into multiple, preferably 4 in this embodiment, and multi-scale feature extraction is performed on the spatial information on each channel feature map. Features at different levels are extracted through multi-scale convolution, so that the model can pay attention to global features and process local details. Then, the channel attention of feature maps of different scales is extracted through the SE (Squeeze-and-Excitation) module to obtain the channel attention vector at each different scale. Next, Softmax is used to recalibrate the multi-scale channel attention vector to obtain the new attention weight after multi-scale channel interaction. Finally, the recalibrated weights and the corresponding feature maps are element-wise multiplied, and a feature map after multi-scale feature information attention weighting is output, so that the PSA module can learn the attention weights with lower model complexity, and effectively integrate local and global attention, which can improve the representation ability of the model when dealing with complex noise.

[0024] Step S4, finally, the output of the feature fusion module in the preprocessor is sent to the target classification model, the preprocessor and the target classification model are jointly trained, the loss function is optimized, and a trained target classification model is obtained; The loss function consists of two parts: preprocessor loss and classification loss; the preprocessor loss adopts mean square error loss, which consists of three parts: the similarity between the original input and the preprocessed sample, the similarity between the normal sample and the adversarial sample image after reconstruction, and the similarity between the normal sample and the adversarial sample after feature fusion; the classification loss adopts cross entropy loss.

[0025] Regarding the deployment and application of trained target classification models, such as Figure 1 As shown, in this embodiment, random noise is first added to the sample to be classified F0 to obtain the sample to be classified F1, and the sample to be classified F1 is input into the trained preprocessor; then, after image reconstruction, the sample to be classified F1 is input into the feature fusion module together; finally, the sample to be classified F1 is input into the trained target classification model for classification.

[0026] In order to verify the effectiveness of the method proposed in this embodiment, a series of tests were carried out in this embodiment. The SGD optimizer was used in adversarial training, the learning rate was set to 0.001, the momentum parameter was 0.9, and the weight decay coefficient was 0.0001.

[0027] (1) Comparative test. This test evaluates the performance of the method proposed in this embodiment in terms of classification accuracy of normal samples and adversarial samples, and compares it with adversarial defense methods such as TRADES, ME-NET and SABRE. All tests use the CIFAR-10 dataset, and the attack methods are FGSM, PGD-20 (20 iterations) and PGD-100 (100 iterations). The maximum perturbation amplitude is 8 / 255. All defense methods are based on the same ResNet18 model architecture and follow the same adversarial restrictions. The test results show that Figure 7 As shown in the figure, the classification accuracy of the method proposed in this embodiment is significantly higher than that of other methods when facing adversarial attacks such as FGSM, PGD-20 and PGD-100, especially when facing high-intensity adversarial attacks of PGD-100, reaching 97.41%, indicating that the method proposed in this embodiment can significantly improve the robustness of the model. At the same time, when the method proposed in this embodiment is tested for classification of normal samples, its accuracy reaches 92.09%, indicating that the method proposed in this embodiment can not only significantly improve the robustness of the model, but also maintain a high classification accuracy for normal samples.

[0028] (2) Ablation test. This test quantitatively evaluates the contribution of each component of the feature fusion module to improving the performance of the target classification model. Specifically, Figure 3 After the NAFNet and PSA modules in the preprocessor are removed respectively, their impact on the performance of the target classification model is evaluated. The test results show that the complete feature fusion module that integrates the NAFNet and PSA modules performs outstandingly in improving the robustness of the target classification model. In addition, this embodiment also analyzes the impact of the image reconstruction and feature fusion modules in the preprocessor on the performance of the target classification model. Specifically, the image reconstruction and feature fusion modules in the preprocessor are removed respectively, and their impact on the performance of the target classification model is evaluated. The test results show that the image reconstruction module significantly improves the adversarial robustness by removing the high-frequency components of the disturbance, but its effect is limited when facing high-intensity attacks. The feature fusion module shows higher adversarial performance by extracting robust features, but there is a slight loss in the classification performance of normal samples. The complete preprocessor that integrates the image reconstruction and feature fusion modules maintains the highest accuracy in terms of both high-intensity attacks and the classification performance of normal samples.

[0029] (3) Consistency and generalization performance test. The above is only part of the test results of this embodiment, not all of them. In fact, on this basis, this embodiment also carried out consistency and generalization performance tests based on different image classification models, facing different attack methods, and different data sets. The results show that Figure 8As shown, the classification accuracy of the method proposed in this embodiment on the CIFAR-10 and MNIST datasets, and in the face of adversarial attacks such as FGSM, PGD-7, PGD-20, PGD-100, PGD-1000, EoTPGD-7, EoTPGD-20, EoTPGD-100 and CW is significantly higher than that of other methods, and the accuracy remains stable, indicating that the method proposed in this embodiment can effectively improve the robustness of the target classification model and has good consistency and generalization ability.

[0030] The specific embodiments described above further describe the purpose, technical solutions and beneficial effects of the present invention in detail. Those skilled in the art may make various modifications to the above contents without departing from the spirit and scope of the present invention as defined in the claims. Therefore, the scope of the present invention is not limited to the above description, but is determined by the scope of the claims.

Claims

1. An adversarial training method based on image reconstruction and feature fusion, characterized in that: include: Step S1, obtaining a normal sample, wherein the normal sample is an image sample containing a category label; generating an adversarial sample based on the obtained normal sample; and forming a training set X0 with the obtained normal sample and the generated adversarial sample; Step S2, first, add random noise to all samples in the training set X0 to obtain the training set X1, and input it into a preprocessor; the preprocessor includes an image reconstruction module and a feature fusion module; Step S3, then, after the training set X1 is processed by the image reconstruction module in the preprocessor, it is input into the feature fusion module in the preprocessor together with the training set X1; Step S4, finally, the output of the feature fusion module in the preprocessor is sent to the target classification model, the preprocessor and the target classification model are jointly trained, the loss function is optimized, and a trained target classification model is obtained.

2. The adversarial training method based on image reconstruction and feature fusion according to claim 1, characterized in that: The step S3 comprises: The image reconstruction module in the preprocessor includes wavelet transform, energy threshold estimation, filtering processing and wavelet inverse transform modules. First, the input data is transformed into a square through a transformation function, and a discrete wavelet transform is performed to decompose the signal into k orthogonal sub-bands, and the detail coefficients and scale coefficients in the horizontal, vertical and diagonal directions are extracted to form a multi-resolution spectrum representation. Then, the frequency band energy threshold is estimated, and the frequency band energy refers to the square sum of the frequency band coefficients. After the energy value is normalized, that is, the range is set to [0, 1], the adaptive filtering threshold Tau is defined as (1 / k)*exp(-Lambda*(Mu / Delta)), wherein Mu and Delta represent the mean and standard deviation of the frequency band energy respectively, and Lambda is a hyperparameter. According to the calculated adaptive filtering threshold Tau, the detail coefficient is thresholded; then, filtering processing is performed, and the transformed coefficient is smoothed by a Gaussian filter; finally, an inverse wavelet transform is performed to reconstruct the processed detail coefficients back to the original space to obtain a reconstructed image.

3. The adversarial training method based on image reconstruction and feature fusion according to claim 1, characterized in that: The step S3 comprises: The feature fusion module in the preprocessor adopts an improved denoising convolutional neural network model; the improved denoising convolutional neural network model introduces the NAFNet module and the PSA module on the basis of the denoising convolutional neural network model DenoiseCNN, and designs 4 channel structures for extracting and fusing features from different scales and directions; first, downsampling is performed through the Conv+Relu module to extract high-dimensional features of the image; then, the NAFNet module is used to achieve efficient feature extraction; then, features are extracted and fused from different scales and directions through 4 channel structures; then, adaptive feature fusion is performed through the PSA module to enhance the expression ability of the features; finally, the original size image is generated through bilinear interpolation upsampling and Conv+Sigmoid; The four channel structures for extracting and fusing features from different scales and directions, the first channel adopts 1×1 convolution to adjust the channel and compress the features; the second channel adopts two 3×3 convolutions, which can increase the receptive field and capture more local spatial features while maintaining a small computational overhead; the third channel adopts a combination of 1×7 and 7×1 convolutions, that is, long convolution kernels in different directions, which helps to capture feature information in different directions, especially line or edge information; the fourth channel adopts DW convolution and PW convolution, DW convolution can separate the spatial and depth information of the channel, reduce the computational complexity, and maintain the richness of the features; PW convolution uses a 1×1 convolution kernel for calculation for each spatial position of the input feature map, and its main function is to reduce dimensionality and increase nonlinearity; finally, the features of different scales of the four channels are fused.

4. The adversarial training method based on image reconstruction and feature fusion according to claim 1, characterized in that: The step S4 comprises: The loss function consists of two parts: preprocessor loss and classification loss; the preprocessor loss adopts mean square error loss, which consists of three parts: the similarity between the original input and the preprocessed sample, the similarity between the normal sample and the adversarial sample image after reconstruction, and the similarity between the normal sample and the adversarial sample after feature fusion; the classification loss adopts cross entropy loss.

Images