Image reconstruction defense method based on global and local feature collaborative extraction

Abstract

The invention belongs to the field of artificial intelligence and computer vision, and relates to an image reconstruction defense method based on global and local feature collaborative extraction, which comprises the following steps: acquiring an attacked image, and inputting the attacked image into a trained image reconstruction defense model to obtain a reconstructed image; the training process of the image reconstruction defense model comprises the following steps: acquiring an attacked image and inputting the attacked image into a global network module to obtain global features and drop channel convolution features; cutting the attacked image to obtain a plurality of local images; inputting all the local images into a progressive local denoising module to obtain local denoising features of the attacked image; inputting the global feature, the reduced channel convolution feature and the local denoising feature into a reconstruction network module to obtain a reconstructed image; calculating a loss function value according to the reconstructed image and the local denoising features to update parameters of the model until a trained reconstructed defense model is obtained; according to the method, the feature contributing to identity information greatly is enhanced through the channel reduction convolution module, and the denoising effect is improved through the progressive local denoising network.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and computer vision, and relates to an image reconstruction defense method based on the collaborative extraction of global and local features. Background Technology

[0002] With the continuous development of deep learning technology, deep learning-based facial recognition technology has been widely used in various fields such as security and social networking. However, it is undeniable that these models are extremely sensitive to adversarial attacks. Attackers only need to add subtle and imperceptible perturbations to the image to significantly weaken the model's recognition performance, or even cause the system to fail directly in critical scenarios, thus posing a serious threat to the security and reliability of deep learning-based facial recognition systems. Therefore, designing efficient and robust adversarial defense methods is of great significance for ensuring information security and enhancing the application value of facial recognition technology.

[0003] To address this problem, various adversarial defense methods have been proposed, among which input reconstruction is considered an important way to enhance model robustness. These methods weaken potential adversarial perturbations by preprocessing adversarial examples, providing cleaner inputs for face recognition models. However, input reconstruction often involves high-dimensional and complex image data processing. Traditional algorithms often remain at the pixel level, reconstructing images only by learning shallow differences between clean and adversarial samples. Such simple reconstruction may fail to properly eliminate perturbations when facing attacks of greater strength or with significantly different attack patterns, resulting in reconstructed images still retaining some noise and causing the recognition model to still err. In fact, the core challenge of input reconstruction lies not only in removing perturbations but also in effectively preserving the identity information of the original image during the reconstruction process, enabling the recognition model to perform classification based on identity features. In recent years, with the rapid development of deep learning, generative models and deep neural networks, with their multi-layered structures, have demonstrated powerful capabilities in abstract feature extraction and nonlinear modeling, especially excelling in image reconstruction and feature preservation tasks.

[0004] In summary, existing deep neural network-based reconstruction adversarial defense methods destroy identity information while removing perturbations, and the reconstructed image still retains some noise, causing the recognition model to still err. Summary of the Invention

[0005] To address the aforementioned problems in the prior art, this invention employs an image reconstruction defense method based on the collaborative extraction of global and local features, comprising: acquiring the attacked image, inputting the attacked image into a trained image reconstruction defense model, and obtaining a reconstructed image; the image reconstruction defense model includes: a global network module, a progressive local denoising module, and a reconstruction network module; the training process of the image reconstruction defense model includes:

[0006] S1. Obtain the attacked image dataset, input the attacked images in the attacked image dataset into the global network module, and obtain the global features and multiple descending channel convolutional features of the attacked images.

[0007] S2. Crop the attacked image to obtain multiple local region images; input all local region images into the progressive local denoising module to obtain the local denoising features of the attacked image.

[0008] S3. Input the global features, reduced-channel convolutional features, and local denoising features into the reconstruction network module to obtain the reconstructed image;

[0009] S4. Calculate the loss function value based on the reconstructed image and local denoising features, update the parameters of the image reconstruction defense model based on the loss function value, and obtain the trained image reconstruction defense model when the loss function value is minimized.

[0010] Beneficial effects:

[0011] 1. This invention utilizes a global network to extract global features and a progressive local denoising network to extract local details, obtaining local denoising features. The image is then reconstructed based on the extracted local denoising features and global features, thereby preserving more identity information while removing perturbations, thus improving reconstruction defense performance. 2. In the global network, this invention achieves adaptive channel selection through a channel reduction convolution module. During backpropagation, the channel reduction convolution module updates model parameters based on identity loss, enabling the output of the channel reduction convolution module to prioritize and enhance features from the input features that contribute more to identity information, while ignoring features related to adversarial noise as much as possible, thus better serving the final image reconstruction. 1. Maintaining identity enhances reconstruction defense performance; 2. The progressive local denoising network of this invention gradually filters out noise through three stages of progressive processing, and fuses features from different stages through skip connections during reconstruction to recover the denoised local image, thereby improving the denoising effect and enhancing the model's ability to reconstruct clean images, thus improving the model's defense performance; 3. The progressive local denoising network of this invention introduces feature purity evaluation and information retention gating mechanisms, and introduces local noise loss to adaptively adjust information flow, ensure the retention of identity-related features, and learn clean sample representations in local denoising features to further extract clean features, thereby enhancing the model's ability to reconstruct clean images and improving the model's defense performance. Attached Figure Description

[0012] Figure 1 A flowchart of an image reconstruction defense method based on the collaborative extraction of global and local features provided in this embodiment of the invention;

[0013] Figure 2This is a schematic diagram of the network structure of the global network module provided in an embodiment of the present invention;

[0014] Figure 3 A schematic diagram of the network structure of the progressive local denoising module provided in an embodiment of the present invention;

[0015] Figure 4 This is a schematic diagram of the structure of the reconstruction network module provided in an embodiment of the present invention;

[0016] Figure 5 This is a schematic diagram of the discriminator provided in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, this invention employs an image reconstruction defense method based on the collaborative extraction of global and local features. Its features include: acquiring the attacked image, inputting the attacked image into a trained image reconstruction defense model to obtain a reconstructed image; the image reconstruction defense model includes: a global network module, a progressive local denoising module, and a reconstruction network module; the training process of the image reconstruction defense model includes:

[0019] S1. Obtain the attacked image dataset, input the attacked images in the attacked image dataset into the global network module, and obtain the global features and multiple descending channel convolutional features of the attacked images.

[0020] The global network module consists of multiple feature extraction layers and multiple channel-reducing convolutional modules. The feature extraction layers and channel-reducing convolutional modules have the same number of layers. The feature extraction layers include convolutional modules and residual modules (ResBlock). The convolutional modules include: convolutional layers, BatchNorm layers (batch normalization layers), and LeakyReLU activation function. The channel-reducing convolutional modules include: CRC modules and ResBlock. Among them, ResBlock is a residual module, and CRC is Channel-Reduction Convolution, i.e., channel-reducing convolution.

[0021] like Figure 2 As shown, the global network module processes the attacked image in the following ways:

[0022] S11. Input the image to be attacked (128×128×3) into the convolutional module (Conv0+BatchNorm+LRelu) of the first feature extraction layer. Input the output of the convolutional module of the first feature extraction layer into the residual module (ResBlock0) and the first channel-reduction convolutional module (Channel-Reduction Conv0 +ResBlock) of the first feature extraction layer respectively. The first channel-reduction convolutional module outputs the first channel-reduction convolutional feature.

[0023] S12. Input the output of the residual module (ResBlock k-1) of the previous feature extraction layer into the convolution module (Conv i+BatchNorm+LRelu) of the current feature extraction layer. Input the output of the convolution module of the current feature extraction layer into the residual module (ResBlock k) and the current channel-reduction convolution module (Channel-Reduction Convk + ResBlock) of the current feature extraction layer respectively. The current channel-reduction convolution module outputs the current channel-reduction convolution feature. Where k is the layer index.

[0024] After each convolution operation, this invention does not directly use the feature map for skip connections like the traditional U-Net. Instead, it introduces an operation combining CRC and ResBlock. This design can combine identity loss to adaptively filter out the identity-rich descending channel convolution features, which are then used for subsequent feature connections.

[0025] For after the first Features obtained after convolution of the feature extraction layer The de-channel convolution module processes it, including: features Perform a decrease-channel convolution operation (CRC): That is, without changing the spatial dimensions (H×W), only the number of channels C is changed; where, Features obtained after de-channel convolution Features The number of channels, This represents the channel reduction ratio. The number of channels after dimensionality reduction of the convolutional module; the features obtained after channel reduction convolution. Input the residual module to obtain the reduced-channel convolutional features. Decreasing channel convolution features It will be used for skip connections in the subsequent image reconstruction stage.

[0026] In one embodiment, for features Performing a channel-reducing convolution operation includes: setting There are 10 convolutional kernels, each of which performs a weighted summation of the multiple channels of the input features and outputs a single channel.

[0027] S13. Repeat step S12 until the output of the last feature extraction layer ResBlock and the output of the last de-channel convolution module are obtained.

[0028] S14. Input the output of the last feature extraction layer ResBlock into the fully connected layer (FC) to obtain the global feature (ID Feature).

[0029] In this embodiment, the fully connected layer extracts a 256-dimensional identity feature vector, i.e., global features, as a compact representation of the entire identity information.

[0030] S2. Crop the attacked image to obtain multiple local region images; input all local region images into the progressive local denoising module to obtain the local denoising features of the attacked image.

[0031] The cropping of the attacked image includes: using a face detector to detect the attacked image and obtain the coordinates of multiple key points; cropping multiple local region images centered on each key point coordinate; wherein the key point coordinates include the coordinates of the eyes, nose and mouth, so the number of local region images is 4.

[0032] In this embodiment, the face detector uses the YuNet face detector, and the cropping size of each region is set as follows: the height and width of the eye images are both 40 pixels, the nose region is 32×40 pixels (height×width), and the mouth region is 32×48 pixels.

[0033] like Figure 3 As shown, the progressive local denoising module includes multiple progressive local denoising networks. Each progressive local denoising network corresponds one-to-one with the type of local region image. The progressive local denoising network structures for the left eye and the right eye are the same.

[0034] The progressive local denoising module processes local image regions by inputting the local image into the corresponding progressive local denoising network to obtain the corresponding local denoising features. The progressive local denoising network gradually filters out noise through three stages of progressive processing (shallow feature extraction and slight compression, spatial downsampling and moderate compression, and channel compression and deep feature extraction). During decoding, it fuses features from different stages through skip connections to reconstruct the denoised local image. Simultaneously, the network introduces a feature purity evaluation and information retention gating mechanism (i.e., feature weight map M) to adaptively adjust the information flow and ensure the preservation of identity-related features.

[0035] The progressive local denoising network includes a multi-stage feature extraction network, a feature quality control module, and a local image reconstruction network. The progressive local denoising module processes local region images by inputting each local region image into its corresponding progressive local denoising network to obtain corresponding local denoising features. The progressive local denoising network processes local region images by inputting the local region image into the multi-stage feature extraction network, inputting the output of the multi-stage feature extraction network into the feature quality control module to obtain a feature weight map and a purity score, and inputting the output of the multi-stage feature extraction network and the feature weight map into the local image reconstruction network to obtain local denoising features.

[0036] The multi-stage feature extraction network includes: a first-stage network (Stage 1), a second-stage network (Stage 2), and a third-stage network (Stage 3); the multi-stage feature extraction network processes the corresponding local region image in the following ways:

[0037] The local region image is input into the first-stage network for shallow feature extraction to obtain features. ;

[0038] Specifically, the local image (H×W×3) is processed by two 3×3 convolutions (Conv(3×3)+BatchNorm+ReLU) and expanded to 64 channels to obtain the features. (Feature2, H×W×64). This stage keeps the feature map spatial size unchanged and captures all information at full resolution, including real details and adversarial noise. It acts as an "information buffer" to prevent the loss of details in subsequent processing.

[0039] Features The input is processed by the second-stage network for spatial compression to obtain features. ;

[0040] Specifically, features First, a 3×3 convolution with a stride of 2 is applied to halve the feature size. Then, two 3×3 convolutions with 32 channels each are applied, followed by a 1×1 convolution with 64 channels to obtain the feature size. (Feature2, H / 2×W / 2×64). This stage discards redundant spatial information (pixel-level, fragmented, high-frequency details) through spatial compression, but retains the most critical spatial structural framework (object outlines, relative positions, and intermediate semantic information). Noise is usually high-frequency and localized, and is the first to be blurred and discarded in spatial compression. This stage filters out fragmented and unstructured noise components.

[0041] Features The input is processed by the third-stage network for channel compression and deep feature extraction to obtain features. ;

[0042] Specifically, features First, a 1×1 convolution with 16 channels (Conv(1×1)+BatchNorm+ReLU) is applied for channel compression. Then, a 3×3 convolution with a stride of 2 is applied for downsampling to obtain features (H / 4×W / 4×16). Next, 3×3 and 5×5 convolutions (Conv(5×5)+BatchNorm+ReLU) are applied to increase the receptive field. Finally, a 1×1 convolution with 64 channels is applied to restore the features. (Feature3, H / 4×W / 4×64), i.e., deep feature extraction;

[0043] This stage constructs an information bottleneck through channel compression (64 channels → 16 channels). The network is forced to use only a very small number of feature channels to represent the most essential information of the local area. This is a mode similar to information distillation, where identity-irrelevant, random noise features are discarded because they cannot pass through the narrow bottleneck, while highly abstract, identity-related semantic features (such as "eye shape" and "mouth curvature") are preserved and enhanced.

[0044] The feature quality control module processes the output of the multi-stage feature extraction network, including:

[0045] Features Global pooling is performed to obtain global statistical information; a 1×1 convolution (Conv(1×1)+ReLU) is then performed on the statistical information, followed by another 1×1 convolution (Conv(1×1)); and the features after this second convolution are then activated by a Sigmoid function to output a purity evaluation score. (Purity Score);

[0046] Features 1×1 convolution and sigmoid activation are performed to generate features. The same size feature weight map M (Gated Weight). This process is a gating mechanism. The gating mechanism dynamically generates a weight map based on the content of the feature itself. The weight is close to 1 in the region with high feature confidence, and close to 0 in the region with possible residual noise. This achieves adaptive denoising and avoids the loss of feature details or residual noise caused by global uniform processing.

[0047] The local image restoration network processes the output and feature weight map of the multi-stage feature extraction network, including:

[0048] Features Multiply it with its feature weight map M, upsample the resulting feature (H / 4×W / 4×64), and then combine the upsampled feature with the feature weight map M. The concatenated features (H / 2×W / 2×128) are then concatenated using two 3×3 convolutions to obtain the local denoising features. (H / 2×W / 2×64); Characteristics of Stage Two It provides intermediate semantic information to help reconstruct local shape and structure, thus preserving the denoising effect brought by multi-layer compression while avoiding feature loss caused by compression.

[0049] Features Upsampling is performed to combine the upsampled features with the original features. The features are concatenated along the channel dimension, and the concatenated features (H×W×128) are then subjected to two 3×3 convolutions to obtain the local denoising features. (H×W×64); Characteristics of Stage One This ensures that low-level details such as textures and edges can be restored during reconstruction.

[0050] Local denoising features Perform a 3×3 convolution with 3 channels (Conv(3×3) + Tanh activation function) to obtain local denoising features. (H×W×3).

[0051] S3. Input the global features, reduced-channel convolutional features, and local denoising features into the reconstruction network module to obtain the reconstructed image;

[0052] like Figure 4 As shown, the reconstruction network module includes fully connected layers and multiple deconvolutional layers; the reconstruction network module processes global features, reduced-channel convolutional features, and local denoising features, including:

[0053] S31. Concatenate the global feature (ID Feature) with Gaussian noise, and input the concatenated feature into a fully connected layer (FC).

[0054] In this embodiment, the 256-dimensional identity feature vector (i.e., global features) from the global network is concatenated with 100-dimensional Gaussian noise, and the dimensions are adjusted through a fully connected layer to reshape it into a feature tensor of (B, 64, 8, 8).

[0055] S32. Convert the output of the fully connected layer into a de-channel convolution feature of the same size as its output. The fusion is performed, and the fused result is input into the first deconvolutional layer (Deconv0).

[0056] The output of the fully connected layer is fused with the decreased-channel convolutional features of the same size, which is to say, the layers are spliced ​​together in the channel dimension.

[0057] S33. Convert the output of the first deconvolutional layer into a de-channel convolution feature of the same size as its output. The fusion is performed, and the fused result is input into the second deconvolution layer (Deconv1).

[0058] S34. Take the output of the previous deconvolutional layer (Deconv k-1) and its decreasing-channel convolutional feature of the same size. Perform fusion and input the fused result into the current deconvolution layer (Deconv k).

[0059] S35. Repeat step S34 until the output of the penultimate deconvolution layer (Deconv K-2) is obtained; where K is the number of deconvolution layers;

[0060] S36. Take the output of the penultimate deconvolutional layer (Deconv K-2) and its decreasing-channel convolutional feature of the same size. The image is then fused with local denoising features and the fused result is input into the last deconvolution layer (Deconv K-1) to obtain the reconstructed image (128×128×3).

[0061] The process of fusing the output of the penultimate deconvolution layer with its decreasing-channel convolutional feature and local denoising features of the same size includes: constructing a blank feature of the same size as the output of the penultimate deconvolution layer; fusing the local denoising features (features of the left and right eyes, nose, and mouth) output of the last deconvolution layer in the local network module to the corresponding positions of the blank feature to obtain the first fused feature; fusing the local features (features of the left and right eyes, nose, and mouth) output of the penultimate deconvolution layer in the local network module to the corresponding positions of the blank feature to obtain the second fused feature; and concatenating the output of the penultimate deconvolution layer, the decreasing-channel convolutional feature, the first fused feature, and the second fused feature along the channel dimension.

[0062] S4. Calculate the loss function value based on the reconstructed image, local denoising features, and purity score. Update the parameters of the image reconstruction defense model based on the loss function value. When the loss function value is minimized, the trained image reconstruction defense model is obtained.

[0063] After obtaining the reconstructed image, multi-scale pixel loss, identity loss, and discriminator loss are introduced to jointly optimize the model.

[0064] For multi-scale pixel loss First, the original 128×128 clean image is downsampled to 64×64 and 32×32 resolutions respectively; simultaneously, reconstructed images at the corresponding scales are obtained from the generation process of the reconstruction network module. Then, the pixel-level L1 loss between the clean and reconstructed images is calculated at these three scales, with the specific calculation formula as follows:

[0065]

[0066] in, Indicates the number of scales, corresponding to the scale. , and Represented at different scales The reconstructed image and the clean image below, A coefficient used to adjust the magnitude of the loss.

[0067] Loss of identity This is achieved by introducing a face recognition model to calculate the cosine distance between the reconstructed image and the clean image, making them more consistent in the identity representation space. The specific calculation formula is as follows:

[0068]

[0069] in, The Res-Net50 face recognition model was introduced. To reconstruct the image, For a clean image, A coefficient used to adjust the magnitude of the loss.

[0070] This invention introduces a method for calculating local noise loss based on supervisory signals from clean samples. Local noise loss Including noise loss in various local image regions Noise loss in each local region image Including feature loss and score loss .

[0071] Specifically, calculating local noise loss This includes performing the same process on the clean image as on its corresponding attacked image to obtain local denoising features. Purity assessment score (That is, cropping a clean image to obtain a local area image of the clean image) ; to extract a local area from a clean image The images are input into the progressive local denoising module to obtain images of each local region of the clean image. The local denoising features output by the corresponding local image recovery network The feature loss is calculated based on the local denoising features of the attacked image and its clean image. (i.e., L1 loss) This loss aims to guide the network to extract features so that its output approximates the representation of a clean image in the feature space, enhancing the model's ability to reconstruct clean images and thus adaptively learning robust local features. The score loss is calculated based on the purity score of the clean image. (i.e., cross-entropy loss) For a clean image score, we want it to be close to 1, i.e., "clean local features," so that the local progressive denoising network can learn the representation of clean samples, thus becoming more effective in adversarial local image denoising. The specific calculation is as follows:

[0072]

[0073]

[0074]

[0075] in, C, H, and W represent the number of channels, height, and width of the local region image, respectively. For the local denoising features of the i-th local region of the attacked image at pixel... The value, The local denoising features of the i-th local region of the clean image at the pixel level The value, A coefficient used to adjust the magnitude of the loss.

[0076] To further improve image quality, this invention introduces a discriminator that acts as an adversarial force against the reconstruction network module, and optimizes the model through adversarial loss. Specifically, for the reconstruction network, the adversarial loss is calculated as follows:

[0077]

[0078] in, To reconstruct the image, the overall loss of the image reconstruction defense model is... .

[0079] In one embodiment, the image reconstruction defense model and the discriminator are trained alternately, specifically including:

[0080] Step 1: Obtain the reconstructed image and loss L output by the image reconstruction defense model; input the reconstructed image of the attacked image and its clean image into the discriminator to obtain the discrimination result. , (score), calculate the adversarial loss based on the discrimination result. ;

[0081] Combating losses The specific calculations are as follows:

[0082]

[0083] in, For a clean image, For discriminator, To reconstruct the distribution of the image, The distribution of a clean image.

[0084] Step 2: Freeze the discriminator based on the loss. Train the image reconstruction defense model, freeze the image reconstruction defense model, and apply the adversarial loss. Training the discriminator;

[0085] Step 3: Repeat the above steps until the joint loss of the image reconstruction defense model and the discriminator is minimized or the maximum number of iterations is reached. Then, complete the model training and obtain the trained image reconstruction defense model and discriminator.

[0086] like Figure 5 As shown, the discriminator includes multiple convolutional layers and fully connected layers.

[0087] In one embodiment, the training set is obtained through preprocessing based on the publicly available CASIA-WebFace dataset. The preprocessing includes: first, center-cropping the input image to extract a central region of 180×180 pixels, followed by scaling it to a resolution of 128×128 pixels. Next, the pixel values ​​are converted to tensor format and normalized to the range [0,1]. Finally, the image is standardized using the mean (0.5, 0.5, 0.5) and standard deviation (0.5, 0.5, 0.5) so that the pixel values ​​of each channel are ultimately distributed within the range [-1,1]. The test set is constructed based on the publicly available LFW dataset, from which 6,000 pairs of images are selected for testing, including 3,000 pairs of positive samples from the same identity and 3,000 pairs of negative samples from different identities. All images are uniformly center-cropped to 128×128 pixels as global input.

[0088] This embodiment uses the aforementioned training set to train the model and saves the optimal model weights. The training parameters are set as follows: 56,000 training iterations, batch size of 24, and Adam optimizer. , The learning rate is 0.0001.

[0089] This embodiment uses the aforementioned test set to test the best-trained model. To accurately evaluate the model's performance, four attacks are selected for testing: Fast Gradient Sign Method (FGSM), Projected Gradient Descent (PGD), Scale-Invariant and Nesterov-accelerated I-FGSM (SI-NI-FGSM), and Variance-based Momentum Iterative Method (VMI). The FGSM intensity is set to [value missing]. and PGD ​​intensity is The strengths of SI-NI-FGSM and VMI are both .

[0090] The test results for each attack are divided into positive sample pairs, negative sample pairs, and the average of the two. Specifically, the attack misleads the model: causing positive samples (the same face) to be judged as "not the same person," while causing negative samples (different faces) to be judged as "the same person." The defense model successfully restores the correct judgment by reconstructing the attacked image, that is, positive sample pairs are reclassified as "the same person," and negative sample pairs are reclassified as "not the same person."

[0091] To verify the effectiveness and advancement of this invention in dealing with adversarial attacks, the model of this invention was compared with Defense-GAN (Defense Generative Adversarial Network), PixelDefend (Pixel Defense Network), and ComDefned (Compressed Defense Network) under the same experimental configuration. The comparison results are shown in Table 1.

[0092] surface Comparative experimental results

[0093]

[0094] As shown in Table 1, this invention significantly outperforms other algorithms on negative sample pairs, and also outperforms them on positive sample pairs under certain attack conditions. On average, the strength is... Under the FGSM attack, it improved performance by 3.7 percentage points compared to other optimal models. Under the PGD attack, it improved performance by 3.5 percentage points compared to other optimal models. Under the SI-NI-FGSM and VMI attacks, it significantly improved performance by 16.4 and 22.8 percentage points compared to other optimal models, respectively.

[0095] To better evaluate the effectiveness of the local denoising loss in the model, ablation experiments were conducted on the test set for the improved module. The experimental results are shown in Table 2.

[0096] surface Ablation test results

[0097]

[0098] As shown in Table 2, after introducing local noise loss, the model learns a feature representation that is closer to that of clean samples by constraining four local features. Compared with direct feature extraction from local features, this module improves the detection results for both positive and negative samples, proving its effectiveness.

[0099] To investigate different channel reduction ratios in dechannelized convolutional CRC within a global module Regarding the impact on the results, this invention addresses the following: The test was conducted, and the experimental results are shown in Table 3.

[0100] Table 3 Experimental results at different channel reduction ratios

[0101]

[0102] As shown in Table 3, when The model performs best when the time is right. The model's performance on positive samples is lower than... There was only a slight improvement on negative samples, and the overall result was lower than expected. And when At the same time, reducing the number of channels too much may lead to the exclusion of facial identity features, resulting in a significant decrease in effectiveness.

[0103] In summary, this invention proposes an image reconstruction defense method based on the collaborative extraction of global and local features. By introducing a CRC module and local noise loss, the former adaptively selects features containing identity information for skip connections to supplement information by combining identity loss and pixel loss, while the latter learns clean sample representations from local denoising features to further extract clean features, thereby enhancing the model's ability to reconstruct clean images and improving its defense performance. Finally, related experiments verify the effectiveness of the proposed method from multiple dimensions. Therefore, the proposed method can provide important support for dealing with adversarial attacks on faces, improve the security of face recognition by input reconstruction methods, and reduce information security losses caused by adversarial attacks.

[0104] The above-described embodiments further illustrate the purpose, technical solution, and advantages of the present invention. It should be understood that the above-described embodiments are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made to the present invention within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An image reconstruction defense method based on global and local feature collaborative extraction, characterized in that, include: The attacked image is obtained, and then inputted into a trained image reconstruction defense model to obtain a reconstructed image. The image reconstruction defense model includes: a global network module, a progressive local denoising module, and a reconstruction network module; the training process of the image reconstruction defense model includes: S1. Obtain the attacked image dataset, input the attacked images in the attacked image dataset into the global network module, and obtain the global features and multiple descending channel convolutional features of the attacked images. S2. Crop the attacked image to obtain multiple local region images; input all local region images into the progressive local denoising module to obtain the local denoising features of the attacked image. S3. Input the global features, reduced-channel convolutional features, and local denoising features into the reconstruction network module to obtain the reconstructed image; S4. Calculate the loss function value based on the reconstructed image and local denoising features, update the parameters of the image reconstruction defense model based on the loss function value, and obtain the trained image reconstruction defense model when the loss function value is minimized.

2. The image reconstruction defense method based on global and local feature collaborative extraction according to claim 1, characterized in that, The global network module includes multiple feature extraction layers and multiple channel-reducing convolutional modules. The feature extraction layers include convolutional modules and residual modules. The global network module processes the attacked image in the following ways: S11. Input the image to be attacked into the convolution module of the first feature extraction layer, and input the output of the convolution module of the first feature extraction layer into the residual module and the first decrementing channel convolution module of the first feature extraction layer respectively; the first decrementing channel convolution module outputs the first decrementing channel convolution feature; S12. Input the output of the residual module of the previous feature extraction layer into the convolution module of the current feature extraction layer, and input the output of the convolution module of the current feature extraction layer into the residual module and the current decreasing channel convolution module of the current feature extraction layer respectively; the current decreasing channel convolution module outputs the current decreasing channel convolution feature. S13. Repeat step S12 until the output of the residual module of the last feature extraction layer and the output of the last channel-decreasing convolution module are obtained. S14. Input the output of the residual module of the last feature extraction layer into the fully connected layer to obtain global features.

3. The image reconstruction defense method based on global and local feature collaborative extraction according to claim 2, characterized in that, The decreasing channel convolution module includes: decreasing channel convolution and residual modules; the k-th layer decreasing channel convolution module processes its input features including: input features are extracted down-channel convolutions are performed: wherein, is the feature after reducing the channel of the convolution, is the output of the convolution module of the kth feature extraction layer, is the feature is the number of channels of the feature is the channel reduction ratio; features after down-channel convolution input residual module to obtain down-channel convolution features .

4. The image reconstruction defense method based on global and local feature collaborative extraction according to claim 1, characterized in that, The reconstruction network module includes fully connected layers and multiple deconvolutional layers; the reconstruction network module processes global features, de-channel convolutional features, and local denoising features, including: S31. Concatenate the global features with Gaussian noise, and input the concatenated features into the fully connected layer. S32. The output of the fully connected layer is fused with the decreased channel convolutional features of the same size, and the fused result is input into the first deconvolutional layer. S33. The output of the first deconvolutional layer is fused with the descending channel convolutional feature of the same size, and the fused result is input into the second deconvolutional layer. S34. Fuse the output of the previous deconvolution layer with the decreasing channel convolution feature of the same size, and input the fused result into the current deconvolution layer. S35. Repeat step S34 until the output of the penultimate deconvolution layer is obtained; S36. The output of the penultimate deconvolution layer is fused with the decreasing channel convolution feature of the same size and the local denoising feature. The fused result is then input into the last deconvolution layer to obtain the reconstructed image.

5. The image reconstruction defense method based on the collaborative extraction of global and local features according to claim 1, characterized in that, The progressive local denoising module includes multiple progressive local denoising networks, each corresponding to a specific type of local region image. Each progressive local denoising network comprises a multi-stage feature extraction network, a feature quality control module, and a local image restoration network. The progressive local denoising module processes local region images by inputting each type of local region image into its corresponding progressive local denoising network to obtain corresponding local denoising features. The progressive local denoising network further processes local region images by inputting the local region image into the multi-stage feature extraction network, inputting the output of the multi-stage feature extraction network into the feature quality control module to obtain a feature weight map and a purity score, and inputting the output of the multi-stage feature extraction network and the feature weight map into the local image restoration network to obtain local denoising features.

6. The image reconstruction defense method based on the collaborative extraction of global and local features according to claim 5, characterized in that, The multi-stage feature extraction network includes: a first-stage network, a second-stage network, and a third-stage network; the multi-stage feature extraction network processes the corresponding local image in the following ways: The local image is input into the first-stage network for shallow feature extraction to obtain features. ; Features The input is processed by the second-stage network for spatial compression to obtain features. ; Features The input is processed by the third-stage network for channel compression and deep feature extraction to obtain features. .

7. The image reconstruction defense method based on the collaborative extraction of global and local features according to claim 6, characterized in that, The local image restoration network processes the output and feature weight map of the multi-stage feature extraction network, including: Features Multiply it with its feature weight map, upsample the resulting features, and then combine the upsampled features with the feature weight map. The features are concatenated along the channel dimension, and then convolved to obtain local denoising features. ; Local denoising features Perform upsampling, and then combine the upsampled features with the existing features. The features are concatenated along the channel dimension, and then convolved to obtain local denoising features. ; Local denoising features Perform convolution to obtain local denoising features. .

8. The image reconstruction defense method based on the collaborative extraction of global and local features according to claim 7, characterized in that, The loss function includes the local noise loss between the attacked image and its corresponding clean image; the local noise loss includes the noise loss of each local region image. Noise loss in each local region image Including feature loss and score loss ; Calculate feature loss and score loss This includes: cropping a clean image to obtain a local region image of the clean image; inputting the local region image of the clean image into a progressive local denoising module to obtain the local denoising features and purity evaluation score of the clean image; and calculating the feature loss based on the local denoising features of the attacked image and its clean image. Calculate the fractional loss based on the purity score of the clean image. .

9. The image reconstruction defense method based on the collaborative extraction of global and local features according to claim 1, characterized in that, The loss function also includes multi-scale pixel loss and identity loss between the reconstructed image of the attacked image and its corresponding clean image; identity loss ;in, For facial recognition models, A reconstructed image of the attacked image. The clean image corresponding to the attacked image. A coefficient used to adjust the magnitude of the loss.

10. The image reconstruction defense method based on the collaborative extraction of global and local features according to claim 1, characterized in that, The image reconstruction defense model also includes a discriminator, and the loss function includes adversarial loss; the discriminator processes the reconstructed image of the attacked image to obtain the discrimination result of the reconstructed image of the attacked image; The adversarial loss is calculated based on the discrimination results of the reconstructed image of the attacked image.

Images