Adversarial sample defense method based on image preprocessing

Abstract

The present invention belongs to the field of artificial intelligence security technology, and discloses an adversarial sample defense method based on image preprocessing, including step 1: constructing a defense model AGD; the defense model AGD has a total of 17 layers of networks, composed of three major modules, among which the first 12 layers of networks are dilated convolution modules, the middle four layers of networks are feature enhancement modules, and the last layer of networks is an attention module; step 2: constructing an image reconstruction model; designing an image reconstruction model based on super-resolution, and reconstructing samples processed by the AGD defense model; step 3: constructing a defense framework SR-AGD; combining the characteristics of the AGD defense model and the image reconstruction model, constructing the defense framework SR-AGD. The image preprocessing-based defense technology SR-AGD proposed in the present invention effectively solves the defects of incomplete removal of adversarial disturbances and affecting the classification of normal samples in existing similar defense methods, and plays a good role in defending against adversarial attacks.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence security technology, and in particular relates to an adversarial sample defense method based on image preprocessing. Background Art

[0002] With the deepening of adversarial sample research, researchers have proposed many defense methods for emerging adversarial attacks. Although these defense methods have played a defensive role to a certain extent, there are still many problems. For example, the gradient masking method cannot defend against adversarial attacks that do not require model gradient information; the adversarial training defense method has high training costs, it is difficult to balance the accuracy and robustness of the model, and the adaptability is poor. Image preprocessing, as an efficient defense method, not only has good defense effects, but also has compatibility. It can be combined with other types of defense methods to further improve the defense capabilities of the model. Therefore, we start from the perspective of image preprocessing and remove the adversarial perturbations on the adversarial samples to make the adversarial samples reclassified correctly.

[0003] There are also some problems in the existing similar defense methods. For example, when using filters to remove adversarial perturbations, only a small part of the perturbations can be removed; when removing adversarial perturbations on pixels, there is a phenomenon of residual perturbation amplification; although the HGD defense method has a better defense effect, the effect of removing adversarial perturbations still needs to be improved. It can be seen that adversarial perturbations are different from natural noise on images and are more difficult to extract and remove. Summary of the invention

[0004] The purpose of the present invention is to provide an adversarial sample defense method based on image preprocessing to solve the above-mentioned technical problems.

[0005] In order to improve the defense effect and enhance the ability to remove adversarial disturbances, the present invention is based on the attention mechanism and super-resolution defense technology SR-AGD: using the characteristics of the attention mechanism to propose a defense model AGD, enhance the model's ability to extract features of adversarial disturbances, and improve the effect of removing adversarial disturbances; using super-resolution to propose an image reconstruction model, reconstruct the image processed by the AGD model, and reconstruct the image details and structure. The specific technical solution is as follows:

[0006] A method for defending against adversarial samples based on image preprocessing comprises the following steps:

[0007] Step 1: Construct a defense model AGD; the defense model AGD has a total of 17 layers of networks, consisting of three major modules, among which the first 12 layers of networks are dilated convolution modules, the middle four layers of networks are feature enhancement modules, and the last layer of networks is an attention module;

[0008] Step 2: Construct an image reconstruction model; design an image reconstruction model based on super-resolution, reconstruct samples processed by the AGD defense model, and reduce the impact of AGD on normal samples;

[0009] Step 3: Construct the defense framework SR-AGD; Combining the characteristics of the AGD defense model and the image reconstruction model, construct the defense framework SR-AGD to achieve the defense effect against adversarial attacks.

[0010] Furthermore, in the dilated convolution module, step 1 changes the convolution layers of the 2nd, 5th, 9th and 12th layers from ordinary convolution to dilated convolution, so as to increase the receptive field to map more context information in the image, thereby facilitating the acquisition of more adversarial perturbation features. Formula (1) represents the process of the dilated convolution module, where x * represents the adversarial sample of the input, f DC Represents the role of the dilated convolution module, O DC represents the output of the dilated convolution module,

[0011] O DC =f DC (x * ). (1)

[0012] Furthermore, in the feature enhancement module, the step 1 combines the input adversarial sample features with the features output by the 16th layer network. Formula (2) represents the process of the feature enhancement module, where x * Represents the adversarial sample of the input, O DC represents the output of the dilated convolution module, f FE Represents the role achieved by the feature enhancement module, O FE represents the output of the dilated convolution module,

[0013] O FE =f FE (x * , O DC ). (2)

[0014] Furthermore, in step 1, a CBAM attention module is added to the last layer of the network to implement the attention mechanism; the CBAM attention mechanism mainly includes two parts, namely channel attention and spatial attention. The attention module uses the two dimensions of channel and space to obtain the weight coefficients of the intermediate feature maps in the AGD defense model using two independent attention mechanisms. Formula (3) represents the processing process of the attention module, where O FE represents the output of the feature enhancement module and is also the input of the attention module. AB Represents the role of the attention module. represents the output of the attention module, i.e., the residual image against the perturbation information,

[0015]

[0016] Furthermore, in the design of the loss function of the AGD defense model in step 1, the difference between the output results of the image before and after the disturbance removal in the high-level network of the target model is used as the loss function, as shown in formula (4), where: represents the image after AGD processing to remove the adversarial perturbation, x represents the original image, and F l represents the output result of the l-th layer network in the target model, ||·|| represents the L1 norm,

[0017]

[0018] Furthermore, the image reconstruction model of step 2 is composed of three modules, namely a feature extraction module, a multi-scale feature fusion module and a sub-pixel reconstruction module.

[0019] Furthermore, in the feature extraction module of step 2, the initial features of the image after AGD processing are extracted as the input of the subsequent module, and a 1×1 convolution is used to extract the initial features of the image after AGD defense model processing. The process is expressed by formula (5):

[0020]

[0021] in, represents the input low-resolution image, i.e., the image obtained after being processed by the AGD defense model, F E Represents a convolution layer pair with a convolution kernel size of 1×1 The feature extraction process, E Represents the feature map obtained after extracting the initial features.

[0022] Furthermore, in the multi-scale feature fusion module, the step 2 uses multiple convolution kernels of different sizes to perform convolution operations on the initial features to extract feature information of different dimensions, and uses feature cascade to achieve fusion of multi-scale features, thereby improving the richness of image features acquired by the model. The multi-scale feature fusion module increases the width of the network, can extract feature information of different dimensions, and make full use of this feature information. The process can be expressed by formula (6):

[0023] O M =F M (O E ), (6)

[0024] Among them, O E represents the initial features extracted by the feature extraction module, F Mrepresents the role of the multi-scale feature fusion module, O M Represents the new feature information obtained after multi-scale feature fusion. The multi-scale feature fusion module contains three convolution kernels of different scales, with sizes of 3×3, 5×5 and 7×7 respectively. Convolution kernels of different sizes are used to obtain feature information of different sizes of input images to improve the richness of features. At the same time, the residual fusion method is used to avoid the loss of feature information in network transmission, ensuring that the feature information extracted by each layer of the network can be fully utilized. In addition, after cascading and fusing multiple features extracted by convolution kernels of different scales, a convolution of size 1×1 is used for dimensionality reduction.

[0025] Furthermore, the sub-pixel reconstruction module of step 2 performs residual fusion on the high-level features obtained by the multi-scale feature fusion module and the low-level features in the low-resolution image before sub-pixel reconstruction. After that, the sub-pixel reconstruction module performs sub-pixel convolution operation on the fused features to reconstruct the high-resolution image. The module can be expressed by formula (7), where x′ represents the image reconstructed by the image reconstruction model,

[0026] x′=F S (O E , O M ), (7)

[0027] In the training process of the image reconstruction model, the pixel-level mean square error (MSE) is used as the loss function. The samples processed by the AGD defense model are input into the image reconstruction model. The output reconstructed image needs to be as similar as possible to the original image. Therefore, the training goal of the image reconstruction model is to minimize the difference between the reconstructed sample and the original image. The loss function can be expressed by formula (8):

[0028]

[0029] in, Represents a parameter set, represents the output of the image reconstruction model, ||·|| represents the L2 norm, and during the training process, the stochastic gradient descent method is used to optimize the loss function.

[0030] Furthermore, the step 3 directly inputs the adversarial sample into the SR-AGD defense framework, first uses the AGD defense model to remove the adversarial perturbation in the adversarial sample to obtain the processed sample, and then sends it to the image reconstruction model for processing. During the reconstruction process of the image reconstruction model, the residual adversarial perturbation is further removed, and finally the defense effect against adversarial attacks is achieved.

[0031] The adversarial sample defense method based on image preprocessing of the present invention has the following advantages: The image preprocessing-based defense technology SR-AGD proposed in the present invention effectively solves the defects of incomplete removal of adversarial disturbances and impact on normal sample classification in existing similar defense methods, and has a good effect of defending against adversarial attacks. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a network structure diagram of the defense model AGD of the present invention;

[0033] Figure 2 A schematic diagram of the network structure of the image reconstruction model of the present invention;

[0034] Figure 3 This is a defense flow chart of the SR-AGD defense framework of the present invention. DETAILED DESCRIPTION

[0035] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of an adversarial sample defense method based on image preprocessing of the present invention in conjunction with the accompanying drawings.

[0036] The image preprocessing-based defense method SR-AGD (Super-Resolution Attention-Guided Denoiser) proposed in this paper effectively solves the defects of existing similar defense methods, such as incomplete removal of adversarial perturbations and affecting the classification of normal samples, and plays a good role in defending against adversarial attacks. SR-AGD mainly consists of two parts, namely the attention-based defense model AGD and the super-resolution-based image reconstruction model. The technical details of these two parts will be introduced below.

[0037] 1. Building a defense model AGD

[0038] The defense model AGD (Attention-Guided Denoiser) has a total of 17 layers of networks, consisting of three modules. Among them, the first 12 layers of the network are dilated convolution modules, the middle four layers of the network are feature enhancement modules, and the last layer of the network is an attention module. The model structure is as follows Figure 1 shown.

[0039] In the dilated convolution module, the convolution layers of the 2nd, 5th, 9th and 12th layers are changed from ordinary convolution to dilated convolution to enlarge the receptive field to map more contextual information in the image, thus helping to obtain more adversarial perturbation features. Formula (1) represents the process of the dilated convolution module, where x * represents the adversarial sample of the input, f DC Represents the role of the dilated convolution module, O DCRepresents the output of the dilated convolution module.

[0040] O DC =f DC (x * ). (1)

[0041] In the feature enhancement module, by fusing the input adversarial sample features with the features output by the 16th layer network, the problem that as the number of network layers increases, the influence of shallow network features on the deep network gradually weakens. In this way, both global and local features are fully utilized, and the expansion convolution module is complementary to improve the ability of the defense model to remove adversarial perturbations. Formula (2) represents the process of the feature enhancement module, where x * Represents the adversarial sample of the input, O DC represents the output of the dilated convolution module, f FE Represents the role achieved by the feature enhancement module, O FE Represents the output of the dilated convolution module.

[0042] O FE =f FE (x * , O DC ). (2)

[0043] Since the adversarial perturbations on adversarial samples are carefully designed, in order to improve the model's ability to extract adversarial perturbations from complex backgrounds, an attention mechanism is introduced into the model. Since the CBAM attention module can be added to the neural network model arbitrarily and trained together with the model, we added this module to the last layer of the network to implement the attention mechanism. The CBAM attention mechanism mainly consists of two parts, namely channel attention and spatial attention. The attention module uses two independent attention mechanisms to obtain weight coefficients for the intermediate feature maps in the AGD defense model using the two dimensions of channel and space, thereby realizing adaptive feature refinement of the feature maps. At the same time, without increasing too much computational effort, the performance of the AGD defense model is significantly improved. Formula (3) represents the processing process of the attention module, where O FE represents the output of the feature enhancement module and is also the input of the attention module. AB Represents the role of the attention module. Represents the output of the attention module, i.e., the residual image against the perturbation information.

[0044]

[0045] The defense model AGD, which consists of three modules: dilated convolution module, feature enhancement module and attention module, makes full use of the characteristics of dilated convolution and attention mechanism, processes the adversarial perturbations in adversarial samples in a more detailed manner, extracts richer feature information of adversarial perturbations, helps to remove adversarial perturbations, and effectively improves the defense capability against adversarial attacks.

[0046] In the design of the loss function of the AGD defense model, the difference between the output results of the image before and after the perturbation removal in the high-level network of the target model is used as the loss function, as shown in formula (4). represents the image after AGD processing to remove the adversarial perturbation, x represents the original image, and F l represents the output result of the l-th layer network in the target model, and ||·|| represents the L1 norm.

[0047]

[0048] 2. Building an Image Reconstruction Model

[0049] Although the AGD defense model has achieved a certain defense effect, there is still a problem. Normal images that can be correctly classified by the target model will mislead the model into misclassification after being processed by AGD. From a visual perspective, the quality of the image is reduced. This shows that when AGD processes normal samples, there is a problem of destroying the normal image structure and reducing the classification accuracy of the model. Therefore, due to the excellent performance of super-resolution image reconstruction technology in image reconstruction tasks, we designed an image reconstruction model based on super-resolution to reconstruct samples processed by the AGD defense model to reduce the impact of AGD on normal samples.

[0050] At the same time, the image reconstruction model further removes residual adversarial disturbances when reconstructing adversarial samples processed by the AGD defense model.

[0051] The image reconstruction model mainly consists of three modules: feature extraction module, multi-scale feature fusion module and sub-pixel reconstruction module. Figure 2 shown.

[0052] (1) Feature extraction module: extracts the initial features of the image after AGD processing as the input of the subsequent modules. It mainly uses a 1×1 convolution to extract the initial features of the image after AGD defense model processing. The process is expressed by formula (5).

[0053]

[0054] in, represents the input low-resolution image, i.e., the image obtained after being processed by the AGD defense model, FE Represents a convolution layer pair with a convolution kernel size of 1×1 The feature extraction process, E Represents the feature map obtained after extracting the initial features.

[0055] (2) Multi-scale feature fusion module

[0056] In the multi-scale feature fusion module, multiple convolution kernels of different sizes are used to perform convolution operations on the initial features to extract feature information of different dimensions, and feature cascade is used to achieve the fusion of multi-scale features, thereby improving the richness of image features acquired by the model. The multi-scale feature fusion module increases the width of the network, can extract feature information of different dimensions, and make full use of this feature information. The process can be expressed by formula (6).

[0057] O M =F M (O E ), (6)

[0058] Among them, O E represents the initial features extracted by the feature extraction module, F M represents the role of the multi-scale feature fusion module, O M Represents the new feature information obtained after multi-scale feature fusion. The multi-scale feature fusion module contains three convolution kernels of different scales, and the sizes of the convolution kernels are 3×3, 5×5, and 7×7 respectively. Using convolution kernels of different sizes, feature information of different sizes of the input image can be obtained to improve the richness of the features. At the same time, the residual fusion method is used to avoid the loss of feature information in network transmission, ensuring that the feature information extracted by each layer of the network can be fully utilized. In addition, in order to avoid too many feature maps and increase the difficulty of model training, multiple features extracted by convolution kernels of different scales are cascaded and fused, and then a convolution of size 1×1 is used for dimensionality reduction.

[0059] (3) Sub-pixel reconstruction module. Since a large amount of information contained in low-resolution images can also be used for super-resolution reconstruction of images, before sub-pixel reconstruction, the high-level features obtained by the multi-scale feature fusion module are residually fused with the low-level features in the low-resolution image to improve the quality of image reconstruction. Afterwards, the sub-pixel reconstruction module performs sub-pixel convolution operations on the fused features to reconstruct a high-resolution image. This module can be expressed by formula (7), where x′ represents the image reconstructed by the image reconstruction model.

[0060] x′=F S (O E , O M ). (7)

[0061] In the training process of the image reconstruction model, the pixel-level mean square error (MSE) is used as the loss function. The samples processed by the AGD defense model are input into the image reconstruction model, and the output reconstructed image must be as similar as possible to the original image. Therefore, the training goal of the image reconstruction model is to minimize the difference between the reconstructed sample and the original image. The loss function can be expressed by formula (8).

[0062]

[0063] in, Represents a parameter set, represents the output of the image reconstruction model, and ||·|| represents the L2 norm. During the training process, the stochastic gradient descent method is used to optimize the loss function.

[0064] 3. Building a defense framework SR-AGD

[0065] We combined the characteristics of the AGD defense model and the image reconstruction model to construct the SR-AGD defense framework. The defense process against adversarial attacks is as follows: Figure 3 As shown in the figure, the adversarial sample is directly input into the SR-AGD defense framework. The AGD defense model is first used to remove the adversarial perturbation in the adversarial sample to obtain the processed sample, which is then sent to the image reconstruction model for processing. During the reconstruction process of the image reconstruction model, the residual adversarial perturbation is further removed, and finally the defense effect against adversarial attacks is achieved.

[0066] For normal samples, after being input into the SR-AGD defense framework, the structural details of the image will first be destroyed by the AGD defense model, so it is then input into the image reconstruction model to reconstruct the lost image details and edge information, thereby effectively alleviating the impact of the AGD defense model on normal samples. The SR-AGD defense framework successfully resists adversarial attacks and reduces the impact on normal samples through the joint processing of the AGD defense model and the image reconstruction model.

[0067] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A method for defending against adversarial samples based on image preprocessing, characterized in that: The steps include: Step 1: Construct a defense model AGD; the defense model AGD has a total of 17 layers of networks, consisting of three major modules, among which the first 12 layers of networks are dilated convolution modules, the middle four layers of networks are feature enhancement modules, and the last layer of networks is an attention module; In the step 1, in the dilated convolution module, the convolution layers of the 2nd, 5th, 9th and 12th layers are changed from ordinary convolution to dilated convolution, so as to enlarge the receptive field to map more context information in the image, thereby helping to obtain more adversarial perturbation features. Formula (1) represents the process of the dilated convolution module, where x * represents the adversarial sample of the input, f DC Represents the role of the dilated convolution module, O DC represents the output of the dilated convolution module, O DC =f DC (x * ) (1) In step 1, in the feature enhancement module, the input adversarial sample features are fused with the features output by the 16th layer network. Formula (2) represents the process of the feature enhancement module, where x * Represents the adversarial sample of the input, O DC represents the output of the dilated convolution module, f FE Represents the role achieved by the feature enhancement module, O FE represents the output of the dilated convolution module, O FE =f FE (x * ,O DC ) (2) In step 1, the CBAM attention module is added to the last layer of the network to implement the attention mechanism; the CBAM attention mechanism mainly includes two parts, namely channel attention and spatial attention. The attention module uses the two dimensions of channel and space to obtain the weight coefficients of the intermediate feature maps in the AGD defense model using two independent attention mechanisms. Formula (3) represents the processing process of the attention module, where O FE represents the output of the feature enhancement module and is also the input of the attention module. AB Represents the role of the attention module. represents the output of the attention module, i.e., the residual image against the perturbation information, In the design of the loss function of the AGD defense model in step 1, the difference between the output results of the image before and after the disturbance removal in the high-level network of the target model is used as the loss function, as shown in formula (4), where: represents the image after AGD processing to remove the adversarial perturbation, x represents the original image, and F l represents the output result of the l-th layer network in the target model, ||·|| represents the L1 norm, Step 2: Construct an image reconstruction model; design an image reconstruction model based on super-resolution, reconstruct samples processed by the AGD defense model, and reduce the impact of AGD on normal samples; The image reconstruction model in step 2 is composed of three modules, namely a feature extraction module, a multi-scale feature fusion module and a sub-pixel reconstruction module; In step 2, in the feature extraction module, the initial features of the image after AGD processing are extracted as the input of the subsequent module. A 1×1 convolution is used to extract the initial features of the image after AGD defense model processing. The process is expressed by formula (5): in, represents the input low-resolution image, i.e., the image obtained after being processed by the AGD defense model, F E Represents a convolution layer pair with a convolution kernel size of 1×1 The feature extraction process, E Represents the feature map obtained after extracting the initial features; In the multi-scale feature fusion module, the step 2 uses multiple convolution kernels of different sizes to perform convolution operations on the initial features to extract feature information of different dimensions, and uses feature cascade to achieve fusion of multi-scale features to improve the richness of image features acquired by the model. The multi-scale feature fusion module increases the width of the network, can extract feature information of different dimensions, and make full use of this feature information. The process can be expressed by formula (6): O M =F M (O E ), (6) Among them, O E represents the initial features extracted by the feature extraction module, F M represents the role of the multi-scale feature fusion module, O M Represents the new feature information obtained after multi-scale feature fusion. In the multi-scale feature fusion module, there are three convolution kernels of different scales, and the sizes of the convolution kernels are 3×3, 5×5 and 7×7 respectively. Convolution kernels of different sizes are used to obtain feature information of different sizes of input images to improve the richness of features. At the same time, the residual fusion method is used to avoid the loss of feature information in network transmission, ensuring that the feature information extracted by each layer of the network can be fully utilized. In addition, after cascading and fusing multiple features extracted by convolution kernels of different scales, a convolution of size 1×1 is used for dimensionality reduction. The sub-pixel reconstruction module of step 2 performs residual fusion on the high-level features obtained by the multi-scale feature fusion module and the low-level features in the low-resolution image before sub-pixel reconstruction. After that, the sub-pixel reconstruction module performs sub-pixel convolution operation on the fused features to reconstruct the high-resolution image. The module can be expressed by formula (7), where x′ represents the image reconstructed by the image reconstruction model, x′=F S (O E ,O M ), (7) In the training process of the image reconstruction model, the pixel-level mean square error (MSE) is used as the loss function. The samples processed by the AGD defense model are input into the image reconstruction model. The output reconstructed image needs to be as similar as possible to the original image. Therefore, the training goal of the image reconstruction model is to minimize the difference between the reconstructed sample and the original image. The loss function can be expressed by formula (8): in, Represents a parameter set, represents the output of the image reconstruction model, ||·|| represents the L2 norm, and during the training process, the stochastic gradient descent method is used to optimize the loss function; Step 3: Construct the defense framework SR-AGD; Combining the characteristics of the AGD defense model and the image reconstruction model, construct the defense framework SR-AGD to achieve the defense effect against adversarial attacks.

2. The adversarial sample defense method based on image preprocessing according to claim 1, characterized in that: The step 3 directly inputs the adversarial sample into the SR-AGD defense framework. The AGD defense model is first used to remove the adversarial perturbation in the adversarial sample to obtain the processed sample, which is then sent to the image reconstruction model for processing. During the reconstruction process of the image reconstruction model, the residual adversarial perturbation is further removed, ultimately achieving a defense effect against adversarial attacks.

Images