An image adversarial sample recovery method and system based on multi-representation fusion

Abstract

The application provides an image adversarial sample recovery method and system based on multi-representation fusion, and the image adversarial sample recovery method comprises the following steps: inputting an adversarial sample into a pixel domain residual backbone network to perform pixel domain preliminary purification to obtain a preliminary recovery image; performing sliding window blocking on the preliminary recovery image, performing singular value decomposition on each local block, performing soft thresholding processing on singular values through a learnable threshold, and then reconstructing a structure refinement image through overlapping fusion; performing two-dimensional discrete Fourier transform on the preliminary recovery image, extracting a logarithmic amplitude spectrum and averaging the same into a single-channel feature, splicing the preliminary recovery image in a channel dimension after splicing, and then inputting the same into a lightweight convolution network to generate a frequency domain guide image; and performing adaptive weighted fusion on the preliminary recovery image, the structure refinement image and the frequency domain guide image. The adversarial sample recovery method improves the recovery effect of the adversarial sample and the robustness of a classification model.

Description

Technical Field

[0001] This invention relates to the field of adversarial sample recovery, specifically to an image adversarial sample recovery method and system based on multi-representation fusion. Background Technology

[0002] Deep neural networks have been widely used in image classification tasks, but their vulnerability to adversarial examples seriously threatens the security and reliability of the model. Adversarial examples refer to the addition of tiny perturbations that are difficult for the human eye to detect to the original image, causing the model to make incorrect predictions. In order to improve the robustness of the model in adversarial environments, researchers have proposed a variety of defense strategies. Among them, adversarial example recovery methods have become an important technical path to improve the robustness of the model because they do not require modification of the target classification model and can be deployed independently as an input-level preprocessing module. At present, adversarial example recovery methods are mainly developed in the following three directions: (1) Pixel-domain-based adversarial example recovery methods. These methods directly process the pixel values ​​of the image and weaken the adversarial perturbation by smoothing, compression or denoising. Common practices include: Gaussian smoothing (taking the average value of the surrounding pixels for each pixel in the image to blur the image), median filtering (replacing the current pixel with the median of the neighboring pixels), and JPEG compression (destroying the perturbation structure by reducing the image quality). In addition, some people use deep neural networks for denoising - first let the neural network learn a large number of correspondences between "noisy images → clean images", and then use the trained network to directly denoise adversarial examples. (2) Adversarial example recovery method based on frequency domain. Studies have found that adversarial perturbations are often concentrated in high-frequency components. Therefore, this type of method first transforms the image from "pixel space" to "frequency space" (through Fourier transform or discrete cosine transform), then suppresses those abnormal high-frequency components in the frequency space, and finally transforms back to pixel space to achieve the purpose of weakening the perturbation effect. (3) Adversarial example recovery method based on structural domain. This type of method uses the "structural rules" of natural images themselves for recovery. There is often a strong correlation between adjacent pixels in natural images, and the image matrix has "low-rank" characteristics (that is, the whole image can be approximated with less information). The addition of adversarial perturbations will destroy this low-rank structure. Therefore, the main structural components of the image (corresponding to larger singular values) can be retained by singular value decomposition, and those secondary components that may contain perturbations (corresponding to smaller singular values) can be suppressed to achieve recovery. Another approach to structural domains is to use generative models (such as generative adversarial networks). These methods first have the generative model learn the data distribution of a large number of clean images, and then "project" adversarial examples back into this clean distribution to obtain the reconstructed image.

[0003] Existing methods generally suffer from the following problems: processing adversarial perturbations from only a single representation domain makes it difficult to comprehensively characterize the multi-representational properties of perturbations. Pixel-domain methods only focus on changes in pixel values, failing to utilize the high-frequency clustering features of perturbations in the frequency domain, and also struggling to leverage the low-rank priors of natural images in the structural domain; frequency-domain methods only focus on anomalies in frequency components, failing to perceive the local perturbation distribution in the pixel domain; structural-domain methods rely solely on overall structural priors, having limited ability to repair fine-grained pixel-level textures, and also struggling to identify perturbation features in the frequency domain. However, adversarial perturbations exhibit anomalous features simultaneously in the pixel, frequency, and structural domains—numerical fluctuations in the pixel domain, high-frequency energy clustering in the frequency domain, and low-rank disruption in the structural domain. Single-representation-domain methods can only capture one aspect of the perturbation, leading to a significant decrease in classification accuracy of the restored image when facing complex attacks (such as PGD, CW, and AutoAttack).

[0004] Structural domain methods generally rely on global assumptions, making it difficult to effectively suppress spatially non-uniform local perturbations. Most recovery methods incorporating structural priors employ global low-rank or global sparse constraints, applying a uniform structural prior to the entire image. However, adversarial perturbations are often spatially non-uniformly distributed: they typically concentrate in local regions relevant to classification decisions (such as object contours and densely textured areas), while background areas are less affected. Global low-rank constraints cannot differentiate this characteristic, leading to a dilemma: if the global constraint is too strong, clean regions are over-smoothed, impairing semantic information; if the global constraint is insufficient, anomalous components in densely perturbated regions cannot be effectively suppressed. This deficiency is particularly pronounced in attack scenarios with strong local perturbations, limiting its application in security-critical fields such as autonomous driving, intelligent security, and identity authentication.

[0005] In view of this, the present invention is hereby proposed. Summary of the Invention

[0006] In view of this, the purpose of this invention is to provide an image adversarial sample recovery method and system based on multi-representation fusion. This recovery method simultaneously performs collaborative modeling of adversarial perturbations from three complementary perspectives: the pixel domain, the structural domain, and the frequency domain. It then integrates the output information from these three branches through a fusion mechanism to generate the final recovered image. The recovered image obtained using this method exhibits good performance and can be widely applied in fields with high security requirements.

[0007] Specifically, the present invention is achieved through the following technical solutions:

[0008] This invention provides an image adversarial sample recovery method based on multi-representation fusion, comprising the following steps:

[0009] The adversarial sample is input into the pixel-domain residual backbone network to perform preliminary pixel-domain cleanup and obtain a preliminary restored image.

[0010] The initially restored image is divided into sliding window blocks, and singular value decomposition is performed on each local block. The singular values ​​are then soft-thresholded using a learnable threshold, and finally reconstructed into a structurally refined image through overlapping fusion.

[0011] The preliminary restored image is subjected to a two-dimensional discrete Fourier transform, the logarithmic amplitude spectrum is extracted and averaged into a single-channel feature, and then concatenated with the preliminary restored image in the channel dimension and fed into a lightweight convolutional network to generate a frequency domain guiding image.

[0012] The preliminary restored image, the structurally refined image, and the frequency domain guided image are adaptively weighted and fused.

[0013] Preferably, as a further feasible option, the method for preliminary pixel domain purification is as follows:

[0014] Let the input adversarial sample be... The network learns a residual mapping. Output disturbance components Then, the perturbation is subtracted from the input to obtain a preliminary recovered image. :

[0015]

[0016] in, .

[0017] This invention belongs to the field of deep learning security technology, specifically involving an adversarial sample recovery method. This method has excellent robustness and can be widely applied to the following scenarios with high requirements for model security, reliability and availability: (1) Autonomous driving field. Autonomous driving systems rely heavily on deep learning models for environmental perception and decision-making, including key tasks such as traffic sign recognition, pedestrian detection, and lane line recognition. Studies have shown that by adding small perturbations to road signs, road surfaces or vehicles, attackers can cause autonomous driving systems to make serious misjudgments (such as recognizing a stop sign as a speed limit sign), directly threatening personal safety. This invention can be used as a front-end defense module for vehicle perception systems to perform real-time recovery processing on images collected by cameras, effectively suppressing the impact of adversarial perturbations on driving decisions and improving the safety redundancy of autonomous driving systems. (2) Intelligent security and identity authentication. Face recognition systems have been widely used in sensitive scenarios such as access control, financial payment, and public safety. Adversarial attacks can bypass face recognition verification by wearing special glasses or superimposing perturbations invisible to the human eye on images, causing security risks. This invention can be deployed in the image input stage of an identity authentication system to perform adversarial sample detection and recovery on the collected face images, ensuring that the image data sent to the recognition model is real and reliable, and improving the anti-attack capability of the identity authentication system. (3) Medical image-assisted diagnosis. Deep learning has shown great potential in medical image analysis and is widely used in lesion detection and classification of X-ray, CT and other images. However, medical diagnosis requires extremely high accuracy. Adversarial perturbations may cause the model to misjudge benign lesions as malignant or miss key lesions, resulting in serious medical accidents. This invention can be embedded in the preprocessing process of a medical image analysis system to perform adversarial perturbation purification on the input images, ensuring the reliability of the diagnostic results and providing reliable intelligent assistance for clinical decision-making. (4) Industrial quality inspection. Industrial vision systems are widely used in product defect detection, assembly quality monitoring and other scenarios. Adversarial perturbations may cause the quality inspection system to misjudge, affecting production efficiency and quality control. This invention can be used as a front-end defense component of an industrial vision system to improve the stability and accuracy of the intelligent manufacturing system under abnormal input conditions.

[0018] Preferably, as a further feasible option, the sliding window segmentation method includes: for the initially restored image Where C is the number of channels, H is the height, and W is the width, a sliding window is used to divide it into multiple sections of size [missing information]. The local blocks are converted into column vectors through an expansion operation.

[0019] Preferably, as a further feasible option, the singular value decomposition method includes:

[0020] For each local block Let B be the number of blocks. Perform singular value decomposition:

[0021] ;

[0022] in and It is an orthogonal matrix. For a diagonal matrix, its diagonal elements It is a singular value.

[0023] Preferably, as a further feasible option, a learnable threshold parameter is introduced. Perform the soft thresholding process on the outliers:

[0024]

[0025] Less than the threshold The singular values ​​are set to zero, while singular values ​​greater than the threshold are retained, thereby suppressing perturbation components in local blocks.

[0026] Preferably, as a further feasible option, the method for generating the frequency domain guided image includes:

[0027] For the preliminary restored image Perform a two-dimensional discrete Fourier transform on each channel to obtain a complex spectrum. Then, a logarithmic transform is performed to compress the dynamic range and enhance the sensitivity to mid-to-high frequency components:

[0028]

[0029] The global frequency domain feature map of a single channel is obtained by averaging the logarithmic amplitude spectra of all channels. ;

[0030] Will As an additional channel, with the initially recovered image Concatenate along the channel dimension to form a four-channel feature tensor. ;

[0031] The four-channel feature tensor is input into a lightweight convolutional network. The network consists of two convolutional layers and one ReLU activation layer, and then outputs a frequency domain guided image. : .

[0032] Preferably, as a further feasible option, the adaptive weighted fusion method includes:

[0033] The fusion process can be represented by the following formula: ;

[0034] in: The final output is the restored image; The initial restored image output by the pixel-domain residual backbone network; The structural refinement image output by the local structure modeling module; The frequency domain guidance image output by the frequency domain sensing guidance module; and For weighting;

[0035] Design a small convolutional network Its input is the preliminary restored image. Output two weight scalars and : ;

[0036] in For the network The learnable parameters are jointly optimized with the parameters of the backbone network, structural branches, and frequency domain branches during end-to-end training.

[0037] The present invention also provides a recovery system for the aforementioned adversarial example recovery method, comprising:

[0038] Pixel domain residual module: used to input adversarial examples into the pixel domain residual backbone network to perform preliminary pixel domain cleanup and obtain a preliminary restored image;

[0039] Local structure modeling module: used to divide the preliminary restored image into sliding window blocks, perform singular value decomposition on each local block, perform soft thresholding of singular values ​​through learnable thresholds, and then reconstruct a structure-refined image through overlapping fusion;

[0040] Frequency domain awareness guidance module: used to perform two-dimensional discrete Fourier transform on the preliminary restored image, extract the logarithmic amplitude spectrum and average it into single-channel features, and then concatenate it with the preliminary restored image in the channel dimension before feeding it into a lightweight convolutional network to generate a frequency domain guidance image;

[0041] Learnable fusion module: used to adaptively weight and fuse the preliminary restored image, the structurally refined image, and the frequency domain guided image.

[0042] The present invention provides a computer-readable storage medium having a computer program stored thereon, the program performing the steps of the above-described adversarial sample recovery method.

[0043] Compared with the prior art, the solution of the present invention has the following technical effects:

[0044] I. Multi-representation collaborative modeling significantly improves adversarial example recovery performance.

[0045] This invention models adversarial perturbations from three complementary perspectives: the pixel domain, the structural domain, and the frequency domain, overcoming the incompleteness of information from a single representation domain. On the ImageNet dataset, the method of this invention outperforms existing representative methods (DISCO, IRAD) in post-recovery classification accuracy under all six attacks (FGSM, BIM, PGD, CW, DeepFool, and AutoAttack). Taking the strong ensemble attack AutoAttack as an example, this invention achieves a classification accuracy of 75.1%, which is 3.2 percentage points higher than the best baseline method; on the CIFAR-10 dataset, this invention achieves 91.1% accuracy under AutoAttack, which is 1.4 percentage points higher than the best baseline method.

[0046] II. Local structural modeling effectively suppresses local disturbances caused by spatial non-uniformity.

[0047] To address the difficulty of handling local non-uniform perturbations by the traditional global low-rank assumption, this invention employs a sliding window block singular value thresholding strategy. Ablation experiments show that on the CIFAR-10 dataset, local structure modeling, compared to global low-rank modeling, improves the classification accuracy after recovery from 78.0% to 89.1% under DeepFool attacks, an improvement of 11.1 percentage points; on the ImageNet dataset, it improves from 57.2% to 71.6% under FGSM attacks, an improvement of 14.4 percentage points. This demonstrates that local structure modeling can effectively suppress anomalous components in densely perturbated regions without compromising clean regions.

[0048] III. A learnable fusion mechanism to achieve adaptive information integration

[0049] This invention introduces a dynamic weight prediction mechanism based on a small convolutional network, enabling the fused weights to adaptively adjust according to the perturbation characteristics of the input image. Comparative experiments between the full model and ablation variants validate the effectiveness of this mechanism: on the CIFAR-10 dataset, the full model achieves 91.1% performance with AutoAttack, while this drops to 80.6% when only the pixel-domain backbone network is retained (i.e., removing the structural and frequency domain branches); on the ImageNet dataset, the full model achieves 75.1% performance with AutoAttack, while this drops to 66.7% when only the pixel-domain backbone network is retained. Removing any branch leads to performance degradation, demonstrating the complementarity of multi-representation fusion and the necessity of a learnable fusion mechanism.

[0050] IV. Excellent cross-attack generalization ability

[0051] The method of this invention, after being trained on a single attack type, can maintain a high recovery accuracy for unseen attacks. For example, on the CIFAR-10 dataset, a model trained only on the FGSM attack still achieves a classification accuracy of over 85% after recovery for unseen attacks such as BIM, PGD, and AutoAttack. This indicates that the present invention learns the common features of adversarial perturbations, rather than overfitting to specific attack patterns, and possesses the generalization ability required for practical deployment. Attached Figure Description

[0052] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0053] Figure 1 A flowchart of the anti-recovery method provided in an embodiment of the present invention;

[0054] Figure 2 This is a schematic diagram of the pixel domain residual backbone network structure according to an embodiment of the present invention;

[0055] Figure 3 This is a flowchart of the local structure modeling module processing in an embodiment of the present invention;

[0056] Figure 4 This is a flowchart of the frequency domain sensing guidance module processing according to an embodiment of the present invention;

[0057] Figure 5 This is a flowchart illustrating a computer device provided in an embodiment of the present invention. Detailed Implementation

[0058] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure; rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0059] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of this disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms. Unless the context clearly indicates otherwise, it should be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0060] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information, depending on the context. For example, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0061] This invention provides an image adversarial sample recovery method based on multi-representation fusion, comprising the following steps:

[0062] The adversarial sample is input into the pixel-domain residual backbone network to perform preliminary pixel-domain cleanup and obtain a preliminary restored image.

[0063] The initially restored image is divided into sliding window blocks, and singular value decomposition is performed on each local block. The singular values ​​are then soft-thresholded using a learnable threshold, and finally reconstructed into a structurally refined image through overlapping fusion.

[0064] The preliminary restored image is subjected to a two-dimensional discrete Fourier transform, the logarithmic amplitude spectrum is extracted and averaged into a single-channel feature, and then concatenated with the preliminary restored image in the channel dimension and fed into a lightweight convolutional network to generate a frequency domain guiding image.

[0065] The preliminary restored image, the structurally refined image, and the frequency domain guided image are adaptively weighted and fused.

[0066] Specifically, the overall process is as follows: Figure 1 As shown. The method includes the following steps:

[0067] Step 1: Preliminary cleaning in the pixel domain. The adversarial examples are input into the pixel domain residual backbone network. This network uses a residual learning strategy to predict the perturbation components and subtract them from the original adversarial examples to obtain a preliminary restored image.

[0068] Step 2: Local modeling of the structural domain. The initially restored image is divided into sliding window blocks. Singular value decomposition (SVD) is performed on each local block. Singular values ​​are soft-thresholded using a learnable threshold. Finally, the image is reconstructed into a structurally refined image through overlapping and fusion.

[0069] Step 3: Frequency Domain Awareness Guidance. Perform a two-dimensional discrete Fourier transform (DFT) on the preliminary restored image, extract the logarithmic amplitude spectrum and average it into single-channel features. Concatenate this with the preliminary restored image along the channel dimension and then feed it into a lightweight convolutional network to generate a frequency domain guidance image.

[0070] Step 4: Learnable Fusion Output. Through a learnable fusion mechanism, the preliminary restored image, the structurally refined image, and the frequency-domain guided image are adaptively weighted and fused to generate the final restored image.

[0071] The following is a detailed description of each module with reference to the accompanying drawings:

[0072] As attached Figure 2 As shown, the pixel-domain residual backbone network employs a residual learning structure and introduces a channel attention mechanism (Squeeze-and-Excitation, SE module). Let the input adversarial example be... The network learns a residual mapping. Output disturbance components Then, the perturbation is subtracted from the input to obtain a preliminary recovered image. :

[0073]

[0074] in, .

[0075] Residual learning allows the network to focus its optimization on perturbation components rather than the complete image, reducing the learning difficulty. The SE channel attention module generates channel weights by performing global average pooling, two fully connected layers, and sigmoid activation on the features of each channel, recalibrating the feature maps and enabling the network to adaptively focus on channels and regions with strong perturbations. The output of this module... It will serve as the common input for subsequent structural domain and frequency domain modules.

[0076] To address the problem that traditional global low-rank assumptions struggle to handle spatially non-uniform local perturbations, this invention designs a local structure modeling module based on sliding window singular value thresholding (SVT), the processing flow of which is shown in the appendix. Figure 3 As shown.

[0077] (1) Image segmentation. For the initial image reconstruction (C is the number of channels, H is the height, and W is the width), a sliding window is used to divide it into multiple sections of size [missing information]. The blocks are local units. Each block is converted into a column vector through an unfolding operation. The sliding step size controls the degree of overlap between blocks, usually set to a specific value. To ensure overlapping coverage.

[0078] (2) Singular value decomposition. For each local block (B is the number of blocks), perform singular value decomposition:

[0079]

[0080] in and It is an orthogonal matrix. For a diagonal matrix, its diagonal elements These are singular values. Larger singular values ​​correspond to the main structural components of a local block, while smaller singular values ​​are usually associated with perturbations or noise.

[0081] (3) Learnable soft thresholding. A learnable threshold parameter is introduced. (This parameter is automatically optimized during training using gradient descent), and outliers are handled with a soft threshold:

[0082]

[0083] This operation will be less than the threshold. The singular values ​​are set to zero, while singular values ​​greater than the threshold are retained, thereby suppressing perturbation components in local blocks.

[0084] (4) Local block reconstruction. Using the thresholded singular value matrix. Reconstruct each local block:

[0085]

[0086] (5) Overlapping and fusion. All reconstructed local blocks are fused into a complete structural refinement image by weighted averaging of overlapping regions according to the original sliding window positions. The overlapping and blending strategy ensures a smooth transition at block boundaries, while simultaneously constraining each pixel by multiple local blocks, thus achieving implicit global regularization.

[0087] Counter-perturbations typically manifest as anomalous accumulations of high-frequency energy in the frequency domain. This invention designs a frequency-domain sensing guidance module that assists in recovery by explicitly extracting frequency-domain features; its processing flow is shown in the appendix. Figure 4 As shown.

[0088] (1) Two-dimensional discrete Fourier transform. For the preliminary image reconstruction Perform a two-dimensional discrete Fourier transform on each channel to obtain a complex spectrum. .

[0089] (2) Logarithmic amplitude spectrum extraction. Calculate the amplitude spectrum. Then, a logarithmic transform is performed to compress the dynamic range and enhance the sensitivity to mid-to-high frequency components:

[0090]

[0091] (3) Channel averaging. The logarithmic amplitude spectrum of all channels is averaged to obtain the global frequency domain feature map of a single channel. .

[0092] (4) Feature splicing. (The following text appears to be incomplete and requires further context: "to splice features.") As an additional channel, with the initial restored image Concatenate along the channel dimension to form a four-channel feature tensor. .

[0093] (5) Lightweight convolution processing. The concatenated feature tensors are input into a lightweight convolutional network. The network consists of two convolutional layers and one ReLU activation layer, and outputs a frequency domain guided image. :

[0094]

[0095] This module does not directly perform hard truncation in the frequency domain. Instead, it learns through the network how to use frequency domain information to guide the repair in the spatial domain, thus avoiding the information loss caused by simple high-frequency suppression.

[0096] To achieve adaptive coordination of information from the pixel domain, structural domain, and frequency domain, this invention designs a learnable fusion mechanism. The fusion process is described by the following formula:

[0097]

[0098] in: The final output is the restored image; The initial restored image output by the pixel-domain residual backbone network; The structural refinement image output by the local structure modeling module; The frequency domain guidance image output by the frequency domain sensing guidance module; and The fusion weights are used to balance the contributions of the structural domain and frequency domain branches.

[0099] Unlike the simple fusion method that uses globally fixed scalar parameters, the present invention... and Instead of being a preset constant or a globally trainable scalar, it is dynamically predicted and generated by a lightweight convolutional network based on the input image. Specifically, a small convolutional network is designed. Its input is the preliminary restored image. Output two weight scalars and :

[0100]

[0101] in For the network The learnable parameters are jointly optimized with the parameters of the backbone network, structural branches, and frequency domain branches during end-to-end training.

[0102] This small convolutional network The structure can be designed as follows (but is not limited to): First, a global average pooling layer is used to... The data is compressed into a feature vector, then passed through two fully connected layers and an activation function, and finally mapped to a sigmoid function. The interval is obtained and This design allows the fusion weights to be dynamically adjusted based on the content of the input image and the perturbation pattern: when the adversarial perturbation mainly manifests as structural damage, the network can output a larger weight. The value enhances the contribution of the refined signal to the structure; when the perturbation has significant frequency domain characteristics, the network can output a larger value. The value enhances the contribution of the frequency domain guiding signal; if a certain branch has a limited contribution to the recovery of the current input, the corresponding weight can automatically approach zero, thereby effectively suppressing the interference of invalid information.

[0103] This dynamic prediction mechanism enables the fusion process to be adaptive and sample-specific. Different input images can obtain different fusion weight configurations. Compared with fixed weights or global scalar parameter schemes, it can more flexibly cope with diverse attack types and image content, thereby improving the overall robustness and generalization ability of the recovery model.

[0104] The residual network plus SE channel attention module used in this invention can also employ other attention mechanisms, such as CBAM (which includes both channel and spatial attention) or Transformer self-attention module; the backbone network structure can be replaced with a lighter network (such as MobileNet style) or a deeper network (such as ResNet-101) to adapt to different computing resources or accuracy requirements.

[0105] The sliding window block division + SVD + learnable soft thresholding used in this invention can be achieved using other low-rank approximation methods, such as RPCA (Robust Principal Component Analysis) or matrix factorization; the shape of the sliding window is not limited to a square, and can be a rectangular window or an adaptive window based on image content.

[0106] In the frequency domain sensing guidance module of the present invention, the two-dimensional discrete Fourier transform (DFT) used to extract the logarithmic amplitude spectrum can be replaced by discrete cosine transform (DCT) or wavelet transform.

[0107] Alternatives to pixel-domain cleanup → parallel refinement in structural and frequency domains → fusion include: cascading multiple refinements in structural and frequency domains (iteratively applying SVT and DFT modules) or fusing the outputs of the structural and frequency domain branches with intermediate features of the backbone network (earlier fusion).

[0108] In addition to providing an adversarial example recovery method, this invention also provides an adversarial example recovery system, specifically comprising:

[0109] Pixel domain residual module: used to input adversarial examples into the pixel domain residual backbone network to perform preliminary pixel domain cleanup and obtain a preliminary restored image;

[0110] Local structure modeling module: used to divide the preliminary restored image into sliding window blocks, perform singular value decomposition on each local block, perform soft thresholding of singular values ​​through learnable thresholds, and then reconstruct a structure-refined image through overlapping fusion;

[0111] Frequency domain awareness guidance module: used to perform two-dimensional discrete Fourier transform on the preliminary restored image, extract the logarithmic amplitude spectrum and average it into single-channel features, and then concatenate it with the preliminary restored image in the channel dimension before feeding it into a lightweight convolutional network to generate a frequency domain guidance image;

[0112] Learnable fusion module: used to adaptively weight and fuse the preliminary restored image, the structurally refined image, and the frequency domain guided image.

[0113] In practice, the above modules can be implemented as independent entities or combined in any way to be implemented as the same or several entities. For the specific implementation of each unit, please refer to the previous method implementation examples, which will not be repeated here.

[0114] In summary, the solution of the present invention has the following technical effects:

[0115] 1. This invention simultaneously performs collaborative modeling of adversarial perturbations from three complementary perspectives: pixel domain, structural domain, and frequency domain. It also integrates the output information of the three branches through a fusion mechanism to generate the final restored image.

[0116] 2. The image is divided into overlapping local blocks. Singular value decomposition (SVD) is performed on each local block. A learnable threshold is used to soft-threshold the singular values ​​to suppress perturbations. Finally, the local blocks are fused into a complete image by weighted averaging of the overlapping regions. This scheme differs from the traditional global low-rank assumption and can effectively suppress spatially non-uniformly distributed local perturbations.

[0117] 3. A small convolutional network was designed to dynamically predict the fusion weights (α and β) of the structural domain thinning signal and the frequency domain guiding signal, taking the initially restored image as input. This allows the fusion weights to be adaptively adjusted according to the perturbation pattern and content of the input image, thereby achieving better adversarial sample restoration results.

[0118] Figure 5 This is a schematic diagram of the structure of a computer device disclosed in this invention. (Reference) Figure 5As shown, the computer device 400 includes at least a memory 402 and a processor 401; the memory 402 is connected to the processor via a communication bus 403 and is used to store computer instructions executable by the processor 401. The processor 401 is used to read computer instructions from the memory 402 to implement the steps of the method described in any of the above embodiments.

[0119] For the above-described apparatus embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The apparatus embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0120] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal disks or removable disks), magneto-optical disks, and CD-ROMs and DVD-ROMs. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0121] Finally, it should be noted that although this specification contains many specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily used to describe the features of specific embodiments of a particular invention. Certain features described in the various embodiments of this specification may also be implemented in combination in a single embodiment. On the other hand, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation of a sub-combination.

[0122] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0123] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0124] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for image adversarial sample recovery based on multi-representation fusion, characterized in that, Includes the following steps: The adversarial sample is input into the pixel-domain residual backbone network to perform preliminary pixel-domain cleanup and obtain a preliminary restored image. The initially restored image is divided into sliding window blocks, and singular value decomposition is performed on each local block. The singular values ​​are then soft-thresholded using a learnable threshold, and finally reconstructed into a structurally refined image through overlapping fusion. The preliminary restored image is subjected to a two-dimensional discrete Fourier transform, the logarithmic amplitude spectrum is extracted and averaged into a single-channel feature, and then concatenated with the preliminary restored image in the channel dimension and fed into a lightweight convolutional network to generate a frequency domain guiding image. The preliminary restored image, the structurally refined image, and the frequency domain guided image are adaptively weighted and fused.

2. The adversarial example recovery method according to claim 1, characterized in that, The initial pixel-domain purification method follows the formula below: Let the input adversarial sample be... The network learns a residual mapping. Output disturbance components Then, the perturbation is subtracted from the input to obtain a preliminary recovered image. : in, .

3. The adversarial example recovery method according to claim 1, characterized in that, The sliding window segmentation method includes: for the preliminary restored image Where C is the number of channels, H is the height, and W is the width, a sliding window is used to divide it into multiple sections of size [missing information]. The local blocks are converted into column vectors through an expansion operation.

4. The adversarial example recovery method according to claim 3, characterized in that, The singular value decomposition method includes: for each local block Let B be the number of blocks. Perform singular value decomposition: ； in and It is an orthogonal matrix. For a diagonal matrix, its diagonal elements It is a singular value.

5. The adversarial sample cleanup method according to claim 4, characterized in that, Introducing a learnable threshold parameter Perform the soft thresholding process on the outliers: Less than the threshold The singular values ​​are set to zero, while singular values ​​greater than the threshold are retained, thereby suppressing perturbation components in local blocks.

6. The adversarial example recovery method according to claim 5, characterized in that, The method for generating the frequency domain guided image includes: For the preliminary restored image Perform a two-dimensional discrete Fourier transform on each channel to obtain a complex spectrum. Then, a logarithmic transform is performed to compress the dynamic range and enhance the sensitivity to mid-to-high frequency components: The global frequency domain feature map of a single channel is obtained by averaging the logarithmic amplitude spectra of all channels. ; Will As an additional channel, with the initially recovered image Concatenate along the channel dimension to form a four-channel feature tensor. ; The four-channel feature tensor is input into a lightweight convolutional network. The network consists of two convolutional layers and one ReLU activation layer, and then outputs a frequency domain guided image. : .

7. The adversarial example recovery method according to claim 6, characterized in that, The adaptive weighted fusion method includes: The fusion process can be represented by the following formula: ; in: The final output is the restored image; The initial restored image output by the pixel-domain residual backbone network; The structural refinement image output by the local structure modeling module; The frequency domain guidance image output by the frequency domain sensing guidance module; and For weighting; Design a small convolutional network Its input is the preliminary restored image. Output two weight scalars and : ; in For the network The learnable parameters are jointly optimized with the parameters of the backbone network, structural branches, and frequency domain branches during end-to-end training.

8. The recovery system of the adversarial example recovery method according to any one of claims 1-7, characterized in that, include: Pixel domain residual module: used to input adversarial examples into the pixel domain residual backbone network to perform preliminary pixel domain cleanup and obtain a preliminary restored image; Local structure modeling module: used to divide the preliminary restored image into sliding window blocks, perform singular value decomposition on each local block, perform soft thresholding of singular values ​​through learnable thresholds, and then reconstruct a structure-refined image through overlapping fusion; Frequency domain awareness guidance module: used to perform two-dimensional discrete Fourier transform on the preliminary restored image, extract the logarithmic amplitude spectrum and average it into single-channel features, and then concatenate it with the preliminary restored image in the channel dimension before feeding it into a lightweight convolutional network to generate a frequency domain guidance image; Learnable fusion module: used to adaptively weight and fuse the preliminary restored image, the structurally refined image, and the frequency domain guided image.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed, it implements the steps of the adversarial sample recovery method according to any one of claims 1-7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the adversarial sample recovery method as described in any one of claims 1-7.

Images