Image class adversarial training robustness enhancement method based on multi-scale frequency domain perception, hardware and application

Abstract

The invention relates to an image class adversarial training robustness enhancement method based on multi-scale frequency domain perception, hardware and application, and the method comprises the steps: carrying out the improved multi-scale decomposition of an input feature map, carrying out different disturbance suppression methods for different target neural networks, and outputting a fusion feature map; performing feature recovery and enhancement on the input feature map in a spatial domain to obtain an enhanced feature map, and fusing to obtain a feature map with enhanced robustness; the method implementation module and the electronic equipment are applied to a direct embedded network to perform robustness enhancement on adversarial training of different image classes. According to the method, the anti-disturbance can be selectively weakened in the frequency domain, the anti-disturbance capability of the model is remarkably improved, key information can be recovered while frequency domain suppression is carried out, and the model can keep a relatively high natural sample recognition rate in a non-attack scene; dynamic balance is achieved between robustness and accuracy, and it is ensured that the method is stable in performance under various complex attack scenes; the method can be directly embedded, and has good expansibility and application prospects.

Description

Technical Field

[0001] This invention relates to the technical field of image or video recognition or understanding, and in particular to a method, hardware, and application for enhancing the robustness of image adversarial training based on multi-scale frequency domain perception. Background Technology

[0002] With the significant achievements of deep neural networks (DNNs) in safety-critical scenarios such as medical diagnosis and autonomous driving, their application value is becoming increasingly apparent. However, with the widespread deployment of this technology in safety-sensitive tasks, people are gradually recognizing its potential vulnerabilities. These vulnerabilities not only threaten the stable operation of models in real-world environments but also bring widespread attention to the reliability of DNNs when facing complex environments and potential risks.

[0003] One typical security issue is adversarial attacks. Because deep learning models often directly face open inputs from the outside during the inference phase, attackers can superimpose tiny and imperceptible perturbations onto normal samples, generating adversarial examples. While these perturbations appear almost identical to the original input to the human eye, they can significantly alter the model's judgment results. Analysis shows that models subjected to adversarial attacks typically maintain high accuracy on clean samples, but produce incorrect predictions or even output the attacker's expected classification results when inputting carefully crafted adversarial examples. This covert and efficient attack method poses a serious threat to the reliable application of DNNs. Therefore, in safety-critical fields such as medical diagnosis and autonomous driving, effectively improving the adversarial robustness of models has become a crucial issue requiring urgent attention.

[0004] To enhance the adversarial robustness of models, existing research has proposed various frequency-domain-based defense methods, such as frequency preprocessing (FP) at the input end, spectral regularization (SR) during training, and robustness enhancement methods based on feature fusion. FP typically weakens high-frequency perturbations by applying frequency-domain filtering, compression, or reconstruction to the input image. SR introduces spectral constraints or regularization terms during training to guide the model to focus more on low-frequency semantic features, thereby stabilizing its discrimination boundary. Feature fusion methods combine features at different scales and directions within the frequency domain, integrating the global semantic information contained in low-frequency components with the local detail features carried by high-frequency components to suppress sensitive areas of feature perturbation. While these methods can improve the robustness of models under adversarial attacks to some extent, they generally suffer from insufficient adaptability, reliance on fixed parameters, or lack of co-optimization with deep model features. Therefore, when facing complex adversarial attacks, the defense effect is still prone to decline, and the ability to recognize natural samples in non-attack environments is often insufficient, making it difficult for the model to achieve an effective balance between robustness and accuracy. Summary of the Invention

[0005] This invention addresses the problems existing in the prior art and provides a method, hardware, and application for enhancing the robustness of image adversarial training based on multi-scale frequency domain perception, which maintains natural discriminative information while suppressing perturbations.

[0006] The technical concept of this invention is to introduce discrete wavelet transform into the frequency domain modeling branch to achieve multi-scale decomposition, and selectively suppress frequency domain perturbations by combining channel weighting and gating mechanisms. At the same time, dynamic spectrum regulation strategies are designed for different network structures to improve the model's adaptability to complex perturbations. Subsequently, in the natural feature enhancement branch, discriminative features that may be lost during frequency domain suppression are recovered through multi-directional structural information pooling and convolution fusion. Finally, the outputs of the two branches are adaptively weighted through a fusion regulation mechanism. The method does not rely on additional prior models and can be directly embedded into existing adversarial training frameworks. It can effectively improve the model's defense capabilities under various attack scenarios while maintaining high recognition accuracy for natural samples.

[0007] The technical solution adopted in this invention is an image adversarial training robustness enhancement method based on multi-scale frequency domain perception. This method performs improved multi-scale decomposition on the input feature map, applies different perturbation suppression methods to different target neural networks, and outputs a fused feature map F. fused ;

[0008] The input feature map is subjected to feature recovery and enhancement in the spatial domain to obtain the enhanced feature map F. enh ;

[0009] Fusion F fused and F enh This yields a feature map with enhanced robustness.

[0010] Preferably, the improved multi-scale decomposition includes the following steps:

[0011] S1.1 Obtain the 3D feature map F of the previous layer of the target neural network;

[0012] S1.2 Applying a discrete wavelet transform to F, a low-frequency subband F is obtained by frequency domain decomposition. ll and three orthogonal high-frequency subbands {F lh , F hl , F hh}, constituting a multi-scale frequency domain subband set ;

[0013] S1.3 Calculate the score of the high-frequency channel response. .

[0014] Preferably, the scoring of the high-frequency channel response satisfy,

[0015]

[0016] in, ,j is the index of the C channels of the feature map, Conv 3×3 (.) represents a 3×3 convolution, and AvgPool(.) represents average pooling.

[0017] Preferably, if the target neural network is a CNN, then based on The relationship between the hyperparameter τ and the high-frequency channel features is used to weight and preserve or suppress them.

[0018]

[0019] in, The suppression constant, such as 10 -4 To ensure its differentiability, τ is the mean score of the high-frequency channel response;

[0020] The processed frequency domain feature set The input to the inverse wavelet transform module reconstructs a scale-consistent fused feature map F. fused .

[0021] Preferably, if the target neural network is ViT, then the following steps are performed:

[0022] S2.1 Based on Each subband is divided into several non-overlapping image blocks p according to the patch size. i Through mapping function Align the frequency domain sub-band features after discrete wavelet transform with the corresponding spatial patch, where (u, v) is the frequency index;

[0023] S2.2 Calculate p independently for each image patch i The local frequency response is obtained by performing a two-dimensional Fourier transform. , This represents the spectrum of the i-th patch;

[0024] S2.3 Calculation Spectral energy

[0025]

[0026] Where, p 2 Here, C is the normalization factor, C is the number of channels, and c is the index of the channel.

[0027] S2.4 Normalize each p i Construct a probability distribution based on the corresponding spectral energy. Calculate the normalized Shannon entropy.

[0028] Where N is the number of patches, For probability distribution, k is the index of N patches. The suppression constant, such as 10 -5 Ensure that log() is valid;

[0029] S2.5 utilizes the mean of the spectral energy of the entire feature map in combination. Standard deviation and normalized Shannon entropy To set the suppression threshold ,

[0030]

[0031] Where λ1 and λ2 are adjustable hyperparameters, λ1,λ2∈(0,1);

[0032] S2.6 performs weighted retention or suppression on the features of each patch.

[0033]

[0034] S2.7 The processed frequency domain feature set The input to the inverse wavelet transform module reconstructs a scale-consistent fused feature map F. fused .

[0035] Preferably, feature recovery and enhancement of the input feature map in the spatial domain includes the following steps:

[0036] S3.1 performs structure extraction, convolutional dimensionality reduction, and sigmoid activation on the input feature map in different spatial directions to generate a spatially aware mask. ;

[0037] S3.2 with Set as frequency domain path mask M frep Based on spatially aware mask M fusion Calculate the fusion response value S(u,v) for each channel c.

[0038]

[0039] in, For the c-channel feature map;

[0040] S3.3 Construct a gating graph,

[0041]

[0042] Where G∈[0,1] H×WG(u,v) is the trust weight of spatial location (u,v) to natural features, and σ(·) is the Sigmoid mapping function; here, natural features refer to the discriminative information inherent in the image that is crucial for correct classification.

[0043] S3.4 extends the spatially aware mask M through a broadcast operation. fusion Frequency domain path mask M frep Expanded to features Figure 1 The dimension of consistency, to obtain ;

[0044] S3.5 Constructing an Enhanced Mask ,

[0045]

[0046] Where c is the channel index;

[0047] S3.6 Constructing Enhanced Features , Obtain the enhanced feature map F enh .

[0048] Preferably, the fused feature map , where λ enh To fuse the weight hyperparameters, λ enh ∈(0,1).

[0049] A module employing the aforementioned image class adversarial training robustness enhancement method based on multi-scale frequency domain perception includes:

[0050] The acquisition unit is used to acquire the input feature map;

[0051] The preprocessing unit is used to improve the multi-scale decomposition of the input feature map;

[0052] The first processing unit is used to output a fused feature map F after applying different perturbation suppression methods to different target neural networks. fused ;

[0053] The second processing unit performs feature recovery and enhancement on the input feature map in the spatial domain to obtain the enhanced feature map F. enh ;

[0054] Fusion output unit, used for fusing F fused and F enh After obtaining the robust enhanced feature map, output it.

[0055] An electronic device, comprising:

[0056] One or more processors;

[0057] Memory, used to store one or more programs;

[0058] When the one or more programs are executed by the one or more processors, the one or more processors implement the image class adversarial training robustness enhancement method based on multi-scale frequency domain awareness.

[0059] An application of the image class adversarial training robustness enhancement method based on multi-scale frequency domain perception is provided, which is applied to directly embedded networks to enhance the robustness of adversarial training for different image classes.

[0060] This invention relates to a robustness enhancement method, hardware, and application for image adversarial training based on multi-scale frequency domain perception. It involves improving the multi-scale decomposition of the input feature map, applying different perturbation suppression methods to different target neural networks, and then outputting a fused feature map F. fused The input feature map is subjected to feature recovery and enhancement in the spatial domain to obtain the enhanced feature map F. enh , fusion F fused and F enh The method obtains robust feature maps; it implements modules and electronic devices, which are then applied to directly embedded networks to enhance the robustness of adversarial training for different image classes.

[0061] The beneficial effects of this invention are as follows:

[0062] (1) By introducing a frequency domain suppression mechanism based on channel importance threshold, it is possible to selectively weaken the anti-disturbance in the frequency domain, effectively avoiding the loss of discriminative information caused by the "one-size-fits-all" low-pass strategy of existing methods, thereby significantly improving the model's anti-disturbance capability;

[0063] (2) The natural feature enhancement mechanism constructed by multi-directional structural information pooling and convolution fusion can restore key information such as edges, textures and semantic regions while suppressing frequency domain, so that the model can maintain a high natural sample recognition rate in non-attack scenarios.

[0064] (3) The fusion control mechanism can achieve adaptive weighted integration between frequency domain robust features and natural enhancement features, and achieve a dynamic balance between robustness and accuracy, ensuring that the method performs stably under various complex attack scenarios;

[0065] (4) It does not rely on additional prior models, has low computational overhead, strong adaptability, can be directly embedded into mainstream convolutional networks or Transformer structures, and has good scalability and application prospects. Attached Figure Description

[0066] Figure 1 This is a flowchart of the method of the present invention;

[0067] Figure 2 This is a flowchart illustrating a specific implementation of the present invention;

[0068] Figure 3 This is a structural block diagram of the module of the present invention;

[0069] Figure 4 This is a comparison chart showing the defensive effects of the present invention against typical adversarial attack scenarios. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] This invention relates to a robustness enhancement method for image adversarial training based on multi-scale frequency domain perception. The method performs improved multi-scale decomposition on the input feature map, applies different perturbation suppression methods to different target neural networks, and outputs a fused feature map F. fused ;

[0072] The input feature map is subjected to feature recovery and enhancement in the spatial domain to obtain the enhanced feature map F. enh ;

[0073] Fusion F fused and F enh This yields a feature map with enhanced robustness.

[0074] This invention addresses the phenomenon of overlapping discriminative image details in concentrated areas of the frequency domain with the perturbation. It designs a frequency domain suppression mechanism based on channel importance thresholds and constructs a natural feature enhancement mechanism to address the potential loss of discriminative information during the frequency domain suppression process. This enables the effective resistance to various types of adversarial attacks without relying on additional prior models, and significantly improves the overall robustness and stability of the model while ensuring high-precision recognition of clean samples.

[0075] Specifically, this invention first uses Discrete Wavelet Transform (DWT) to decompose the input features in the multi-scale frequency domain, and then combines channel-level importance weights and gating mechanisms to screen out high-frequency perturbation regions. By suppressing the model's activation response to adversarial components, the perturbation is slowed down in the model.

[0076] To compensate for the lack of sensitivity of ViT to local high-frequency perturbations in feature modeling, a dynamic control strategy based on multi-scale spectral energy distribution is designed. This strategy adaptively adjusts the expression intensity of frequency domain components to achieve fine control and adaptation to different perturbation structures.

[0077] By extracting complementary structural information across directions through multi-directional global information pooling and convolution fusion, and combining the Sigmoid activation function to highlight regions that significantly contribute to the classification of clean samples, the model's ability to discriminate natural semantic features is enhanced.

[0078] Finally, learnable parameters are introduced into the module, and a dynamic balance between robustness and discriminative features is achieved by adaptively fusing frequency domain features (enhancing robustness) and natural features (improving accuracy).

[0079] The following section will elaborate on the process of the method.

[0080] (1) Improved multi-scale decomposition of the input feature map;

[0081] The improved multi-scale decomposition includes the following steps:

[0082] S1.1 Obtain the 3D feature map F(H*W*C) of the previous layer of the target neural network;

[0083] S1.2 Applying Discrete Wavelet Transform (DWT) to F, a low-frequency subband F is obtained by frequency domain decomposition. ll and three orthogonal high-frequency subbands {F lh , F hl , F hh}, constituting a multi-scale frequency domain subband set Low-frequency subband F ll The aggregated graph contains overall structural information, while orthogonal high-frequency subbands represent edge and texture details in different directions. Explicitly characterize frequency components;

[0084] S1.3 Calculate the score of the high-frequency channel response. .

[0085] The perturbation sensitivity of channel d (one of the C channels) in the high-frequency subband is described by a score of the high-frequency channel response. High-frequency channel response score satisfy,

[0086]

[0087] in, ,j is the index of the C channels of the feature map, Conv 3×3 (.) represents a 3×3 convolution, and AvgPool(.) represents average pooling.

[0088] (2) After applying different perturbation suppression methods to different target networks, output the fused feature map F. fused ;

[0089] (2-1) If the target neural network is a CNN (Convolutional Neural Network), then based on The relationship between the hyperparameter τ and the high-frequency channel features is used to weight and preserve or suppress them.

[0090]

[0091] in, τ is the suppression constant, and τ is the mean score of the high-frequency channel response. Here, the hyperparameter τ is used to distinguish the frequency domain contribution of different channels, and the suppression constant ε is used to maintain the complete differentiability of the computation graph while suppressing the frequency domain. When α(d) < τ, the corresponding high-frequency subband feature is multiplied by ε to strongly suppress it and ensure the differentiability of the computation graph. When α(d) ≥ τ, it is scaled according to α(d) to retain the discriminative high-frequency components.

[0092] The processed frequency domain feature set The input to the inverse wavelet transform module reconstructs a scale-consistent fused feature map F. fused .

[0093] (2-2) If the target neural network is ViT (self-attention visual model or patch visual model), construct a dynamic gating generation strategy based on spectral statistical characteristics and execute the following steps:

[0094] S2.1 Based on Each subband is divided into several non-overlapping image blocks p according to the patch size. i Specifically, for p non-overlapping image blocks of size p×p i , ; through mapping functions Align the frequency domain sub-band features after discrete wavelet transform with the corresponding spatial patch, where (u, v) is the frequency index;

[0095] S2.2 Calculate p independently for each image patch i The local frequency response is obtained by performing a two-dimensional Fourier transform (2D-FFT). , This represents the spectrum of the i-th patch;

[0096] S2.3 Calculation The spectral energy is used to quantify the intensity and concentration of potential disturbances within the region.

[0097]

[0098] Where, p 2 Here, C is the normalization factor, C is the number of channels, and c is the index of the channel.

[0099] S2.4 Normalize each pi Construct a probability distribution based on the corresponding spectral energy. The normalized Shannon entropy is calculated to measure the spatial uncertainty of the perturbation distribution, serving as a supplementary indicator for modeling the spatial dimension of spectral energy.

[0100]

[0101] Where N is the number of patches, For probability distribution, k is the index of N patches. It is the suppression constant;

[0102] S2.5 utilizes the mean of the spectral energy of the entire feature map in combination. Standard deviation and normalized Shannon entropy To set the suppression threshold ,

[0103]

[0104] Wherein, λ1 and λ2 are adjustable hyperparameters, λ1,λ2∈(0,1), which control the weights of the perturbation energy term and the spatial diffusion term in the construction of the dynamic threshold, respectively, and are used to balance the contribution of energy intensity and frequency domain distribution mode to the suppression strategy.

[0105] S2.6 performs weighted retention or suppression on the features of each patch.

[0106]

[0107] S2.7 will process the frequency domain feature set. The input to the inverse wavelet transform (IDWT) module reconstructs a scale-consistent fused feature map F. fused .

[0108] (3) Perform feature recovery and enhancement on the input feature map in the spatial domain to obtain the enhanced feature map F. enh ;

[0109] In this invention, multi-directional structural information pooling is used in conjunction with a convolutional fusion mechanism to recover and enhance the discriminative information lost due to false positives in the frequency domain path, thereby maintaining classification performance.

[0110] Feature recovery and enhancement of the input feature map in the spatial domain includes the following steps:

[0111] S3.1 performs structure extraction, convolutional dimensionality reduction, and sigmoid activation on the input feature map in different spatial directions to generate a spatially aware mask. The structure extraction in different spatial directions here includes horizontal pooling (H-GAP), vertical pooling (V-GAP), main diagonal pooling (D-GAP), and secondary diagonal pooling (A-GAP).

[0112] S3.2 with Set as frequency domain path mask M frep Based on spatially aware mask M fusion Calculate the fusion response value S(u,v) for each channel c, representing the discriminative difference between the two paths in the channel dimension.

[0113]

[0114] in, For the c-channel feature map; This represents the activation intensity of channel c at position (u,v). Combining these two values ​​can estimate the degree of confidence in the structural enhancement path at this spatial location.

[0115] S3.3 Construct a gating graph,

[0116]

[0117] Where G∈[0,1] H×W G(u,v) is the trust weight of spatial location (u,v) to natural features, and σ(·) is the Sigmoid mapping function;

[0118] S3.4 extends the spatially aware mask M through a broadcast operation. fusion Frequency domain path mask M frep Expanded to features Figure 1 The dimension of consistency, to obtain This enables cross-dimensional, position-by-position fusion.

[0119] S3.5 Constructing an Enhanced Mask ,

[0120]

[0121] Where c is the channel index; the first term represents the weighted activation under the natural enhancement path, and the second term represents the reserved activation under the frequency domain suppression path. This enhancement mask integrates spatial location confidence and channel path confidence, and can adaptively determine where to favor natural enhancement and where to maintain frequency domain suppression, thereby improving the accuracy of fusion judgment and region discrimination capability;

[0122] S3.6 Constructing Enhanced Features , Obtain the enhanced feature map F enh .

[0123] Without introducing redundant features, scaling up the original features helps to highlight structural edges and improve the discriminative power of the features, while avoiding the problem of false suppression caused by frequency domain suppression.

[0124] (4) Fusion F fused and F enh To obtain a robustly enhanced feature map;

[0125] Feature map after fusion , where λ enh To fuse the weight hyperparameters, λ enh ∈(0,1).

[0126] By adjusting λ enh Depending on the task requirements, a balance can be struck between preserving natural semantic expression and suppressing the influence of perturbations. The fused discriminative features are then input into the next layer.

[0127] This invention also relates to a module employing the aforementioned image class adversarial training robustness enhancement method based on multi-scale frequency domain perception, comprising:

[0128] The acquisition unit is used to acquire the input feature map;

[0129] The preprocessing unit is used to improve the multi-scale decomposition of the input feature map;

[0130] The first processing unit is used to output a fused feature map F after applying different perturbation suppression methods to different target neural networks. fused ;

[0131] The second processing unit performs feature recovery and enhancement on the input feature map in the spatial domain to obtain the enhanced feature map F. enh ;

[0132] Fusion output unit, used for fusing F fused and F enh After obtaining the robust enhanced feature map, output it.

[0133] The present invention also relates to an electronic device, comprising:

[0134] One or more processors;

[0135] Memory, used to store one or more programs;

[0136] When the one or more programs are executed by the one or more processors, the one or more processors implement the image class adversarial training robustness enhancement method based on multi-scale frequency domain awareness.

[0137] In the application, the module loaded with the image adversarial training robustness enhancement program based on multi-scale frequency domain perception is embedded in the intermediate layer of the neural network. The robust enhanced feature map is obtained through the image adversarial training robustness enhancement method based on multi-scale frequency domain perception, which facilitates the improvement of image adversarial training robustness.

[0138] In practical applications, it can also be implemented using computer-readable storage media and / or computer devices.

[0139] This invention also relates to an application of the aforementioned image class adversarial training robustness enhancement method based on multi-scale frequency domain perception, which is applied to directly embedded networks to enhance the robustness of adversarial training for different image classes.

[0140] The following is a specific embodiment of the present invention, such as... Figure 4 The experimental results of this invention are shown on typical datasets and under various attack scenarios. The classification accuracy of WRN-34-10 on the CIFAR-10 dataset under different methods for PGD-10, PGD-20, PGD-50, C&W, and AutoAttack attacks are presented. On the CIFAR-10 dataset, the proposed method (Ours-TR) achieves a classification accuracy of 60.9% under the PGD-10 attack, significantly improving upon PGD-AT (55.2%), TRADES (56.8%), and FAT (50.3%). Under the PGD-20 attack, Ours-TR achieves an accuracy of 60.2%, outperforming both TRADES (56.1%) and PGD-AT (54.3%). Under the C&W attack, Ours-TR maintains an accuracy of 57.4%, significantly better than FAT's 48.6%. Under AutoAttack, Ours-TR reached 55.2%, a significant improvement over both TRADES (52.3%) and FAT (47.5%).

[0141] Experiments have shown that the method of this invention can not only maintain high robustness under various complex attacks, but also achieve acceptable recognition accuracy on clean samples. Its advantages lie in its ability to be directly embedded into existing mainstream neural network models, with low computational overhead and good adaptability. This helps to achieve reliable deployment of deep learning models in security-critical scenarios and also helps researchers quickly locate and improve the vulnerable links of the model in practical applications.

[0142] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0144] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0145] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0146] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the invention.

[0147] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for enhancing the robustness of image-based adversarial training based on multi-scale frequency domain sensing, characterized in that: The method performs an improved multi-scale decomposition on the input feature map, applies different perturbation suppression methods for different target neural networks, and outputs a fused feature map F. fused ; The input feature map is subjected to feature recovery and enhancement in the spatial domain to obtain the enhanced feature map F. enh ; Fusion F fused and F enh This yields a feature map with enhanced robustness.

2. The image class adversarial training robustness enhancement method based on multi-scale frequency domain perception according to claim 1, characterized in that: The improved multi-scale decomposition includes the following steps: S1.1 Obtain the 3D feature map F of the previous layer of the target neural network; S1.2 applies a discrete wavelet transform to F, and the frequency domain decomposition yields a low-frequency subband F. ll and three orthogonal high-frequency subbands {F lh ,F hl , F hh }, constituting a multi-scale frequency domain subband set ; S1.3 Calculate the score of the high-frequency channel response. .

3. The image class adversarial training robustness enhancement method based on multi-scale frequency domain perception according to claim 2, characterized in that: Scoring of high-frequency channel response satisfy, ， in, , j is the index of the C channels of the feature map, Conv 3×3 (.) represents a 3×3 convolution, and AvgPool(.) represents average pooling.

4. The image class adversarial training robustness enhancement method based on multi-scale frequency domain perception according to claim 2, characterized in that: If the target neural network is a CNN, then based on The relationship between the hyperparameter τ and the high-frequency channel features is used to weight and preserve or suppress them. ， in, Let τ be the suppression constant, and τ be the mean score of the high-frequency channel response. The processed frequency domain feature set The input to the inverse wavelet transform module reconstructs a scale-consistent fused feature map F. fused .

5. The image class adversarial training robustness enhancement method based on multi-scale frequency domain perception according to claim 2, characterized in that: If the target neural network is ViT, then perform the following steps: S2.1 Based on Each subband is divided into several non-overlapping image blocks p according to the patch size. i The frequency domain subband features after discrete wavelet transform are aligned with the corresponding spatial patch by a mapping function, where (u, v) is the frequency index. S2.2 Calculate p independently for each image patch i The local frequency response is obtained by performing a two-dimensional Fourier transform. , This represents the spectrum of the i-th patch; S2.3 Calculation Spectral energy ， Where, p 2 Here, C is the normalization factor, C is the number of channels, and c is the index of the channel. S2.4 Normalize each p i Construct a probability distribution based on the corresponding spectral energy. Calculate the normalized Shannon entropy. ， Where N is the number of patches, For probability distribution, k is the index of N patches. It is the suppression constant; S2.5 utilizes the mean of the spectral energy of the entire feature map in combination. Standard deviation and normalized Shannon entropy To set the suppression threshold , ， Where λ1 and λ2 are adjustable hyperparameters, λ1,λ2∈(0,1); S2.6 performs weighted retention or suppression on the features of each patch. ， S2.7 The processed frequency domain feature set The input to the inverse wavelet transform module reconstructs a scale-consistent fused feature map F. fused .

6. The image class adversarial training robustness enhancement method based on multi-scale frequency domain perception according to claim 2, characterized in that: Feature recovery and enhancement of the input feature map in the spatial domain includes the following steps: S3.1 performs structure extraction, convolutional dimensionality reduction, and sigmoid activation on the input feature map in different spatial directions to generate a spatially aware mask. ; S3.2 with Set as frequency domain path mask M frep Based on spatially aware mask M fusion Calculate the fusion response value S(u,v) for each channel c. ， in, For the c-channel feature map; S3.3 Construct a gating graph, ， Where G∈[0,1] H×W G(u,v) is the trust weight of spatial location (u,v) to natural features, and σ(·) is the Sigmoid mapping function; S3.4 extends the spatially aware mask M through an expansion operation. fusion Frequency domain path mask M frep Expanding to the same dimensions as the feature map, we get ; S3.5 Constructing an Enhanced Mask , ， Where c is the channel index; S3.6 Constructing Enhanced Features , Obtain the enhanced feature map F enh .

7. The image class adversarial training robustness enhancement method based on multi-scale frequency domain perception according to claim 1, characterized in that: Feature map after fusion , where λ enh To fuse the weight hyperparameters, λ enh ∈(0,1).

8. A module employing the image class adversarial training robustness enhancement method based on multi-scale frequency domain perception as described in any one of claims 1 to 7, characterized in that: include: The acquisition unit is used to acquire the input feature map; The preprocessing unit is used to improve the multi-scale decomposition of the input feature map; The first processing unit is used to output a fused feature map F after applying different perturbation suppression methods to different target neural networks. fused ; The second processing unit performs feature recovery and enhancement on the input feature map in the spatial domain to obtain the enhanced feature map F. enh ; Fusion output unit, used for fusing F fused and F enh After obtaining the robust enhanced feature map, output it.

9. An electronic device, characterized in that: include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the image class adversarial training robustness enhancement method based on multi-scale frequency domain awareness as described in any one of claims 1 to 7.

10. An application of the image class adversarial training robustness enhancement method based on multi-scale frequency domain perception as described in any one of claims 1 to 7, characterized in that: It is applied to directly embedded networks to enhance robustness for adversarial training on different image classes.

Images