Self-adaptive confrontation image defense method based on intensity perception and related equipment

Abstract

The embodiment of the invention provides a self-adaptive confrontation image defense method based on intensity perception and related equipment, and belongs to the technical field of artificial intelligence security. The method comprises the following steps: firstly, constructing a multi-level confrontation intensity evaluation index, and comprehensively quantifying the disturbance intensity of a confrontation sample from three dimensions of a pixel level, a feature level and a prediction level; an intensity perception adversarial purification network is constructed, and the network is composed of an adversarial intensity predictor and an intensity-guided adversarial purifier in a cooperative mode. The predictor perceives the confrontation intensity of the input image through a sparse self-attention mechanism and generates a dynamic weight; and according to the sensed intensity information and the dynamic weight, the purifier adaptively executes purification operation through an intensity-guided multi-scale feature fusion and feature modulation module, and a clean image is reconstructed. According to the method, the defects of the existing static purification method are overcome, and the purification strategy can be dynamically adjusted according to the attack threat degree, so that the better balance between the anti-robustness and the model standard precision is realized under the wide anti-attack condition.

Description

An intensity-aware adaptive adversarial image defense method and related equipment Technical Field

[0001] This application relates to the field of artificial intelligence security technology, and in particular to an adaptive adversarial image defense method and related equipment based on intensity perception. Background Technology

[0002] Deep neural networks (DNNs) have made groundbreaking progress in computer vision tasks. However, DNNs have proven to be extremely vulnerable to adversarial attacks. Attackers can cause models to make incorrect predictions by adding tiny, carefully designed perturbations to the input image that are imperceptible to human vision; such tampered samples are called adversarial examples. Adversarial examples pose a serious threat to the security of applications in safety-critical fields such as autonomous driving, facial recognition, and medical image analysis.

[0003] Existing adversarial defense methods can be mainly divided into three categories: adversarial robust defense, adversarial detection defense, and adversarial sanitization defense. Adversarial robust defense (such as adversarial training) aims to improve the model's resistance to perturbations, but it often leads to a decrease in the model's standard accuracy on clean samples and is computationally expensive. Adversarial detection defense attempts to identify and reject adversarial examples, but in scenarios such as autonomous driving where decision-making is crucial, simply rejecting input can have serious consequences. Adversarial sanitization defense is a model-agnostic preprocessing technique designed to restore adversarial examples to a clean, original data distribution, offering better scalability and application potential.

[0004] However, most existing adversarial descaling methods employ a static, fixed descaling strategy, applying the same descaling intensity and processing method to attacks of varying strengths and types. This "one-size-fits-all" approach has inherent flaws: for weak attacks, excessive descaling may damage the semantic information and details of the image, leading to decreased cleanliness accuracy; for strong attacks, insufficient descaling cannot effectively eliminate perturbations, resulting in insufficient robustness. Therefore, achieving the optimal balance between adversarial robustness and standard accuracy is the core challenge currently facing adversarial descaling defenses. Summary of the Invention

[0005] The main objective of this application is to propose an intensity-aware adaptive adversarial image defense method and related equipment, which aims to overcome the static and non-adaptive defects of existing adversarial purification defense methods and solve the problem that they are difficult to balance robustness and standard accuracy when facing adversarial attacks of different intensities.

[0006] To achieve the above objectives, one aspect of this application proposes an intensity-aware adaptive adversarial image defense method. The method includes: constructing a multi-level adversarial strength evaluation index (MAIM) to quantify the adversarial strength of adversarial samples from three dimensions: pixel-level, feature-level, and prediction-level; constructing an intensity-aware adversarial sanitization network (IAPN), which includes an adversarial strength predictor and an intensity-guided adversarial sanitizer, used to perceive the adversarial strength of the input image and perform adaptive sanitization; calculating the adversarial strength value of each adversarial sample in the training dataset based on the constructed multi-level adversarial strength evaluation index, and using this value as its strength label; training the adversarial sanitization network using an end-to-end joint optimization strategy, with consistency loss, strength loss, and adversarial loss as supervision, to obtain a trained reconstructed sanitization model; and using the trained reconstructed sanitization model to perform intensity-aware adaptive sanitization on the input adversarial image, restoring the adversarial image to a clean data distribution.

[0007] In some embodiments, the multi-level adversarial strength evaluation index includes: pixel-level perturbation strength ( ), calculated as the mean squared error between the adversarial image and the corresponding clean image; feature-level semantic distortion ( ), calculated as the mean squared error between the class activation features of the adversarial image and the clean image on the pre-trained classifier; prediction-level probability divergence ( ), calculated as the KL divergence between the adversarial image and the clean image in the classifier output probability distribution.

[0008] In some embodiments, the adversarial strength predictor includes: a sparse self-attention Transformer encoder for extracting multi-scale features from an input image; a learnable feature fusion prediction head for predicting pixel-level, feature-level, and prediction-level adversarial strength components from the multi-scale features and obtaining the overall adversarial strength through weighted fusion; and a dynamic weight mapper for generating a dynamic weight vector based on the overall adversarial strength and the multi-scale features to guide dynamic convolution operations in the purifier.

[0009] In some embodiments, the learnable feature fusion prediction head includes three parallel branches, corresponding to pixel-level, feature-level, and prediction-level intensity predictions, respectively. Each branch performs weighted fusion of multi-scale features through a learnable scale factor, and then obtains the corresponding intensity component through global average pooling and a multilayer perceptron.

[0010] In some embodiments, the strength-guided adversarial cleanup includes: a strength-guided multi-scale feature fusion module (IMFF) for adaptively fusing multi-scale input features according to the adversarial strength; a strength-guided feature modulation module (IFM) for modulating the features in terms of channel and spatial dimensions according to the adversarial strength to suppress error amplification; a Swin Transformer encoder for extracting adversarial robust features; and a dynamic convolutional decoder for reconstructing a clean image based on a dynamic weight vector.

[0011] In some embodiments, the intensity-guided multi-scale feature fusion module achieves fusion through the following steps: performing global average pooling on the multi-scale features and concatenating them; inputting the concatenated features and adversarial strength into a multilayer perceptron and a linear layer respectively to obtain scale attention weights and intensity modulation factors; weighting and normalizing the scale attention weights and intensity modulation factors to obtain fusion weights for each scale; and weighting and summing the multi-scale features according to the fusion weights to obtain the fused features.

[0012] In some embodiments, the intensity-guided feature modulation module includes a channel attention branch and a spatial attention branch: the channel attention branch generates channel weights based on the feature global pooling result and the adversarial strength embedding; the spatial attention branch generates spatial weights based on the feature spatial pooling result and the adversarial strength map; the channel weights and spatial weights are combined into a modulation matrix to modulate the input features element by element.

[0013] In some embodiments, the consistency loss includes: pixel consistency loss, calculated as the L1 distance between the cleaned image and the clean image; feature consistency loss, calculated as the L2 distance between the class activation features of the image output by the cleaned network and the corresponding original clean image on the pre-trained classifier; and label consistency loss, calculated as the cross-entropy loss between the predicted label and the true label of the cleaned image on the classifier.

[0014] In some embodiments, the strength loss is the sum of the mean square errors of the predicted adversarial strength and the actual adversarial strength at each level.

[0015] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0016] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0017] To achieve the above objectives, another aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0018] Compared with the prior art, this application has the following significant advantages: 1) It realizes dynamic adaptive defense: By introducing an adversarial strength perception mechanism, this application can dynamically adjust the purification strategy and model parameters according to the specific threat level of the input image, thus overcoming the limitations of static defense.

[0019] 2) Provides a more comprehensive strength assessment: The proposed multi-level adversarial strength evaluation index not only considers pixel-level perturbations, but also delves deeper into feature semantics and model prediction levels for quantification, making the representation of adversarial strength more comprehensive and accurate.

[0020] 3) Achieved better balance: The adaptive mechanism ensures "light" cleansing for weak attacks to preserve details and "heavy" cleansing for strong attacks to eliminate threats, thus maintaining high adversarial robustness and clean sample classification accuracy in a wide range of attack scenarios.

[0021] 4) Excellent scalability: As a model-agnostic preprocessing defense, this method does not require modification or retraining of downstream classifiers, making it easy to integrate into existing vision systems and flexible in deployment. Attached Figure Description

[0022] Figure 1 is an overview framework diagram of the intensity-aware adaptive adversarial image defense method in the embodiments of this application.

[0023] Figure 2 is a schematic diagram of the structure of the learnable feature fusion prediction head in the embodiments of this application.

[0024] Figure 3 is a schematic diagram of the structure of the dynamic weight mapper in an embodiment of this application.

[0025] Figure 4 is a schematic diagram of the structure of the intensity-guided multi-scale feature fusion module in an embodiment of this application.

[0026] Figure 5 is a schematic diagram of the structure of the intensity-guided feature modulation module in an embodiment of this application.

[0027] Figure 6 is a flowchart of the steps of an intensity-aware adaptive adversarial image defense method in an embodiment of this application.

[0028] Figure 7 is a schematic diagram of the hardware structure of the electronic device in an embodiment of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0031] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0032] 1) An intensity-aware adaptive adversarial image defense method, which is named IAP (Intensity-aware Adversarial Purification) in this application. IAP consists of two parts, including a multi-level adversarial intensity metric MAIM (Multi-level Adversarial Intensity Metric) and an intensity-aware adversarial purification network IAPN (Intensity-aware Adversarial Purification Network). The former establishes a specific representation of the adversarial intensity, while the latter performs intensity-aware adversarial image purification.

[0033] Deep neural networks have made significant progress in the field of computer vision. However, they remain vulnerable to adversarial attacks. In such attacks, attackers can induce incorrect predictions by applying carefully designed and almost imperceptible perturbations to the input image. These perturbation samples, known as adversarial examples, pose a serious threat to the reliable deployment of deep neural networks in safety-critical applications such as autonomous driving, facial recognition, and medical image analysis. Therefore, it is urgent to design effective and efficient adversarial defense methods to enhance the adversarial robustness of deep neural networks.

[0034] Existing adversarial defense methods can be broadly categorized into three types: adversarial robustness defense, adversarial detection defense, and adversarial sanitization defense. Adversarial robustness defense enhances the inherent robustness of deep neural networks through adversarial training or model optimization, but often leads to significant performance degradation and high deployment costs. Adversarial detection defense aims to identify adversarial examples and prevent them from being further processed by the model; however, detection-only defenses have fundamental limitations in safety-critical scenarios such as autonomous driving, as simply rejecting input may trigger unsafe behavior. In contrast, adversarial sanitization defense restores adversarial examples to a clean data distribution in a model-agnostic manner, offering better scalability and demonstrating greater potential in practical applications. However, existing adversarial sanitization methods generally employ a static sanitization paradigm, lacking awareness of attack strength and dynamic adaptability, thus making it difficult to achieve a satisfactory balance between adversarial robustness and standard accuracy.

[0035] Overall, constructing an adversarial descaling method that balances semantic preservation and perturbation removal remains a challenging task. To address this, this invention proposes an intensity-aware adaptive adversarial image defense method. This method first constructs a multi-level adversarial strength evaluation index, then introduces an intensity-aware mechanism into the descaling model, and designs an intensity-guided dynamic descaling module. Through this design, intensity-guided adaptive adversarial image descaling is achieved, achieving a better balance between adversarial robustness and standard accuracy.

[0036] In view of this, embodiments of this application provide an intensity-aware adaptive adversarial image defense method, electronic device, computer-readable storage medium, and computer program product. The scheme includes: (a) constructing a multi-level adversarial strength evaluation index; (b) constructing an intensity-aware adversarial decontamination network; (c) using step (a) to generate adversarial strength labels for the training dataset, providing supervision signals for intensity perception; (d) training the network in step (b) using an end-to-end joint optimization method to obtain a trained reconstruction and decontamination model; and (e) using the trained reconstruction model in step (d) to perform intensity-aware adaptive adversarial image decontamination, restoring the adversarial image to a clean data distribution. The multi-level adversarial strength evaluation index proposed in embodiments of this application not only focuses on pixel-level perturbation amplitude but also captures the intrinsic impact of adversarial examples on deep neural networks, achieving a more comprehensive quantitative representation of adversarial strength. Furthermore, the intensity-aware adversarial decontamination network proposed in embodiments of this application introduces a dynamic decontamination paradigm, capable of sensing the adversarial strength of the input image and dynamically adjusting the decontamination strategy and model parameters, thereby achieving adaptive adversarial decontamination and achieving a better balance between adversarial robustness and standard accuracy.

[0037] This application provides an intensity-aware adaptive adversarial image defense method, relating to the field of adversarial sample defense technology. This intensity-aware adaptive adversarial image defense method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, or in-vehicle terminal, but is not limited to these. The server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network. The software can be an application implementing an intensity-aware adaptive adversarial image defense method, but is not limited to the above forms.

[0038] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0039] As shown in Figure 6, this embodiment provides an intensity-aware adaptive adversarial image defense method, which specifically includes the following steps: S1: Construct a multi-level adversarial intensity evaluation index (MAIM).

[0040] Specifically, a multi-level adversarial strength evaluation index, MAIM, was designed. From three dimensions—pixel-level perturbation strength, feature-level semantic distortion, and prediction-level probability divergence—the comprehensive impact of adversarial perturbations on deep neural networks is systematically quantified, thereby constructing an explicit representation of adversarial strength.

[0041] S2: Construct an intensity-aware adversarial sanitization network (IAPN) that includes an adversarial intensity predictor and an intensity-guided adversarial sanitizer to perceive the adversarial intensity of the input image and perform adaptive sanitization.

[0042] Specifically, the predictor receives shallow features from the purifier as input, performs adversarial strength prediction, and feeds the strength information back to the purifier, thereby achieving strength-guided adversarial image purification.

[0043] S3: Based on the constructed multi-level adversarial strength evaluation index, calculate the adversarial strength value of each adversarial sample in the training dataset, and use this value as its strength label.

[0044] In some embodiments, an adversarial attack method is used to generate adversarial samples of different intensities on the training dataset for the pre-trained model, and the MAIM in step S1 is used to generate adversarial strength labels for the training dataset, providing strength supervision signals for the subsequent training of IAPN.

[0045] S4: An end-to-end joint optimization strategy is adopted, with consistency loss, strength loss and adversarial loss as supervision, to train the adversarial decontamination network and obtain the trained reconstructed decontamination model.

[0046] In some embodiments, an end-to-end joint optimization strategy is adopted, guided by pixel, feature and label consistency loss (generating supervision signals with corresponding clean samples as reference), intensity loss (using the adversarial strength label of step S3 as supervision information) and adversarial loss (introducing a discriminator for adversarial training), to train the IAPN in step S2.

[0047] S5: Utilize the trained reconstruction sanitization model to perform intensity-aware adaptive sanitization on the input adversarial image, restoring the adversarial image to a clean data distribution.

[0048] Specifically, the defense is deployed using the IAPN trained in step S4, and intensity-aware adaptive adversarial image cleansing is performed to restore the adversarial image to a clean data distribution.

[0049] The solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings and specific application examples.

[0050] To facilitate understanding of the technical solution of this embodiment, Figure 1 shows an overview of an intensity-aware adaptive adversarial image defense method. As shown in Figure 1, this embodiment designs an intensity-aware adaptive adversarial image defense method, named IAP (Intensity-aware Adversarial Purification). IAP mainly consists of two components: a multi-level adversarial intensity metric (MAIM) for quantifying adversarial intensity, and an intensity-aware adversarial purification network (IAPN) for performing adaptive adversarial image purification.

[0051] First, MAIM ( Evaluating adversarial examples from multiple perspectives The strength of the resistance, including the perturbation strength ( semantic distortion ) and probability divergence ( ), and generate a multi-level intensity vector: (1) Provide supervision signals for adversarial strength prediction during IAPN training.

[0052] Secondly, IAPN employs a collaborative prediction-cleaning architecture and introduces a multi-scale scheme. Given an input image... (where h and w represent the height and width of the image, respectively), we construct a multi-scale input. ,in Corresponding to the original resolution (i.e., mesoscale). and The multi-scale inputs are obtained through 2x upsampling and downsampling (i.e., large-scale and small-scale), respectively. Subsequently, the multi-scale inputs undergo shallow feature extraction through embedding layers with downsampling factors of 4, 2, and 1 to generate multi-scale embeddings. (Where H, W, and C represent the height, width, and number of channels of the feature map, respectively).

[0053] Intensity predictor First, process the mesoscale embedding. The multi-level intensity vectors are predicted. The vector is then linearly aggregated to obtain the adversarial strength. Based on this, based on and multi-scale embedding Mapping the dynamic weight vector (Where n represents the number of kernels in the dynamic convolution). This process can be represented as: (2) Finally, and Joint guidance for air purifiers Reconstruct a clean image: (3) IAPN uses consistency loss (including pixel, feature and label consistency), strength loss and adversarial loss for end-to-end joint optimization. By deploying the trained IAPN, it can perceive the adversarial strength of the input image in real time and dynamically adjust the sanitization strategy and model parameters, thereby achieving adaptive adversarial sanitization and restoring the adversarial image to the clean data distribution, effectively solving the trade-off between adversarial robustness and standard accuracy in existing adversarial sanitization defenses. The following is a detailed description of the specific implementation method: (1) Multi-level adversarial strength evaluation index In order to more fully evaluate the adversarial strength, we designed a multi-level adversarial strength evaluation index to characterize the adversarial strength from multiple perspectives of pixel level, feature level and prediction level, specifically including: a) Pixel level perturbation strength The adversarial perturbation is most directly reflected in the pixel-by-pixel deviation from the clean image. In this embodiment, the perturbation amplitude is defined as the adversarial image Its corresponding clean image Mean square error between: (4)b) Feature-level semantic distortion.

[0054] This embodiment introduces class activation features to compute semantic distortion. Given a pre-trained deep neural network classifier... Class activation features can be obtained by backprojecting the weights of the corresponding class in its output layer onto the last convolutional feature layer. For the input image Its prediction category is denoted as ,in Indicates category The predicted probability. Let the features of the last convolutional layer be... , and prediction category The corresponding classification weights are ,in If the number of channels is used, then the class activation feature can be calculated as follows: (5) Each element Essentially reflects the first Each channel feature corresponds to the category . contributions.

[0055] This embodiment defines semantic distortion as adversarial image. With clean images Mean squared error between category activation features: (6)c) Predictive probability divergence.

[0056] The impact of adversarial examples on deep neural network classifiers ultimately manifests as significant changes in the output probability distribution. To capture the distribution shift at the prediction level, we use KL divergence as a measure of distribution distance. Formally, let... and Let represent the predicted probability distributions of the adversarial image and its corresponding clean image, respectively. Then, the probability divergence is defined as follows: (7) Based on MAIM, this embodiment calculates and normalizes the multi-level intensity vector in the training dataset. This is to provide reliable supervision for intensity prediction.

[0057] (2) As shown in Figure 1, the intensity-aware adversarial purification network (IAPN) achieves intensity-aware adversarial purification through the synergistic effect of the predictor and the purifier.

[0058] 2.1) Adversarial Strength Predictor: Since adversarial perturbations often manifest as semantically independent patterns, sparse self-attention is more effective than traditional dense global self-attention in characterizing such features. Therefore, to better capture perturbation features, this embodiment uses a Transformer based on sparse self-attention (such as a Sparse Transformer) to construct the encoder in the predictor, and designs a prediction head with learnable feature fusion to achieve three-branch prediction. The adversarial strength of each branch is weighted and fused to form the overall adversarial strength, which is used to guide the dynamic module in the purifier. A dynamic weight mapper is used to convert the adversarial strength into dynamic weights to guide the dynamic convolution in the purifier.

[0059] a) As shown in Figure 1, the Sparse Transformer encoder uses Sparse Transformers with different sparsity rates. The encoder forms a hierarchical structure through downsampling. Each layer contains multiple sparsity rates (labeled above the module), and M Sparse Transformer blocks are set under each sparsity rate.

[0060] Features extracted from different sparse layers are complementary: low-sparse features capture global semantic context to facilitate overall understanding, while high-sparse features focus on non-semantic cues to model local perturbations. This design enables the model to adaptively balance semantic and non-semantic features at different granularities, thereby achieving accurate intensity predictions in subsequent prediction heads. Furthermore, sparse self-attention eliminates a large amount of irrelevant key-value computation, thus achieving lightweight intensity perception.

[0061] b) Learnable Feature Fusion Prediction Head This embodiment introduces a learnable scale factor into the prediction head to achieve adaptive multi-scale feature fusion. As shown in Figure 2, the structure consists of three parallel branches, which are used to model perturbation amplitude, semantic distortion, and probability divergence, respectively. The three are then aggregated to generate the predicted adversarial strength (see Figure 1).

[0062] Given mesoscale embeddings Extracting multi-scale features from different encoder layers The first four features are retained. Spatial resolution and There are 1 channel, and the latter two features have been downsampled to a resolution of 1. The number of channels is To obtain a uniform-dimensional representation, all features are transformed through channel expansion and upsampling operations. . First, it passes through a learnable scaling factor. The three parallel branches are processed to generate an aggregated feature map: (8) Subsequently, each layer is processed using global average pooling and a multilayer perceptron (MLP) to predict the adversarial strength of the corresponding layer: (9) Finally, the overall adversarial strength is obtained by aggregating the strengths of each level through weighted summation: (10) Among them, ,and This represents the weighting coefficient. This multi-level fusion method can comprehensively characterize the adversarial strength, providing support for the subsequent strength-guided purification process.

[0063] c) If the dynamic weight mapper converts scalars... Directly mapped to dynamic weights Its expressive power is very limited, resulting in a nearly uniform distribution. To address this issue, the Dynamic Weight Mapper (DWM) generates dynamic weights by fusing scale attention and adversarial strength guidance, as shown in Figure 3.

[0064] Given multi-scale embedding DWM first performs global average pooling on each scale, and then concatenates the pooled features into a vector. Subsequently, a multilayer perceptron (MLP) is used to... Mapped to scale attention weights, and then passed through a linear layer The mapping is to an intensity-guided modulation factor, as follows: (11) (12) The dynamic weights are finally calculated as follows: (13) This design supports intensity-guided modulation of dynamic convolution weights, providing IAP with diverse and adaptive weight selection.

[0065] 2.2) Intensity-Guided Adversarial Purifier As shown in Figure 1, the proposed purifier employs a hybrid asymmetric encoder-decoder architecture, combining the advantages of Transformer and convolution. This purifier comprises four core components to achieve intensity-aware purification: 1) an intensity-guided multi-scale feature fusion (IMFF) module for adaptive multi-scale feature integration; 2) an intensity-guided feature modulation (IFM) module for suppressing error amplification; 3) a Swin Transformer encoder for feature extraction; and 4) a dynamic convolutional decoder for reconstruction. Furthermore, a randomization operation is introduced after the decoder to further enhance the defense robustness.

[0066] a) Intensity-Guided Multi-Scale Feature Fusion Module: Multi-scale features have complementary advantages: high-resolution features can retain discriminative details, while low-resolution features can provide robustness against strong interference. To fully utilize these characteristics, this invention proposes an IMFF module, which adaptively fuses multi-scale features under the guidance of predicted adversarial strength. Figure 4 shows its overall structure.

[0067] Multi-scale embedding As input, the IMFF module first performs global average pooling on the features at each scale, and then concatenates the pooling results to form a vector. Subsequently, and the intensity of confrontation The signals are projected as scale attention scores and intensity-guided modulation factors, respectively, through a multilayer perceptron (MLP) and a linear layer. (14) (15) The fusion weight is obtained by... and Combining the results, we get: (16) Finally, the multi-scale features are fused by weighted summation: (17)IMFF dynamically adjusts the relative contribution of features at different scales based on the adversarial strength, thereby achieving adaptive purification and improving robustness against attacks of varying strength.

[0068] b) Intensity-Guided Feature Modulation Module: While residual connections provide training stability for deep neural networks, they can reintroduce harmful noise patterns, causing residual adversarial perturbations to propagate to downstream layers. To mitigate this error amplification problem, this invention proposes embedding an IFM module into the encoder and decoder to adaptively modulate features based on the predicted adversarial strength. The structure of IFM is shown in Figure 5.

[0069] Given input features The IFM module first extracts the global representation through average pooling and max pooling operations, obtaining... Meanwhile, the predicted intensity of confrontation. Embedded through a linear layer, resulting in (in (This represents the embedding dimension). These three representations are concatenated and input into a multilayer perceptron (MLP) to generate channel-dimensional attention weights: (18) Although the channel attention branch can efficiently model global patterns, it lacks the spatial sensitivity required to capture local details. To compensate for this deficiency, the IFM module further performs average pooling and max pooling operations along the channel dimension to compute spatial representations, resulting in... Predicted intensity of confrontation Intensity map formed after space broadcast Spatial attention weights are then generated through convolutional layers: (19) Combine the channel attention branch with the spatial attention branch to form a dense modulation matrix, and then use this matrix to process the input features. Modulation: (20) of which This indicates element-wise multiplication (i.e., multiplying element by element).

[0070] The IFM module replaces direct residual connections with fine-grained feature modulation, thereby mitigating the propagation of adversarial disturbances along the jump path and suppressing error amplification.

[0071] c) Recent research on the Swin Transformer encoder has shown that the features learned by the Transformer contain less low-level information, which helps improve robustness to adversarial perturbations (which are typically dominated by high-frequency components). Therefore, this embodiment uses the SwinTransformer as the encoder to capture adversarially robust features to support subsequent reconstruction tasks.

[0072] As shown in Figure 1, the encoder consists of N consecutive blocks, each block containing M consecutive Swing Transformer blocks, an IFM module, and a... The second downsampling layer omits the downsampling operation in the last block. This embodiment replaces the standard feed-forward network (FFN) in the Swin Transformer with a locally-enhanced feed-forward network (LeFF) in the Uformer. LeFF incorporates convolutional operations to better capture local contextual information. Furthermore, this embodiment replaces these convolutional layers with intensity-guided dynamic convolutions, enabling the encoder to adaptively learn shared or unique features.

[0073] Therefore, the fused multi-scale representation Deep features are obtained after encoding. .

[0074] d) Dynamic Convolutional Decoder: To achieve robust cleanup while maintaining reconstruction quality, the decoder employs intensity-guided dynamic convolution.

[0075] As shown in Figure 1, the decoder consists of N consecutive blocks, each block containing a... The system consists of an upsampling layer, M consecutive dynamically generated convolutional blocks, and an IFM module. The last upsampling layer uses the same scaling factor as the mesoscale input embedding (i.e., when the embedding layers corresponding to large, medium, and small scale inputs use downsampling factors of 1, 2, and 4, respectively). This paper adopts the convolutional block design of UniFormer and replaces the standard convolution with dynamic convolution.

[0076] Therefore, coding features Feature maps are generated through stepwise decoding. The image is then processed through a final dynamic convolutional layer to obtain the cleaned-up image. .

[0077] e) Randomization layers: Although randomization defenses have been proven ineffective against adaptive white-box attacks when used alone, they can serve as an effective complement to sanitization defenses.

[0078] This embodiment employs a random scaling and padding enhancement strategy with a probability of 0.8: the image is randomly scaled to 90%-98% of its original size, and then zero-padding is performed at random locations to restore the original dimensions. It should be noted that the randomization operation is only performed during the inference phase and is not used during the training phase.

[0079] 2.3) End-to-End Joint Optimization In this embodiment, the overall training objective of the proposed IAPN is constructed as a weighted combination of three losses: consistency loss, strength loss, and adversarial loss. The complete objective function is defined as follows: (21) Consistency loss aims to ensure that the cleaned image is consistent with the clean image at the pixel, feature, and prediction levels: (22) Among them, , ,and These represent pixel consistency loss, feature consistency loss, and label consistency loss, respectively. This represents the cross-entropy loss.

[0080] To supervise multi-layered adversarial strength learning, the strength loss is defined as follows: (23) Among them, and These represent the predicted adversarial strength values ​​and the actual adversarial strength values ​​at three levels: pixel-level perturbation amplitude (pm), feature-level semantic distortion (sd), and prediction-level probability difference (pd).

[0081] To enhance the realism of the cleaned image, this embodiment introduces a discriminator. The generator is trained to deceive the discriminator, which learns to distinguish between the real image (clean image) and the generated image (cleaned image). Therefore, the generator... The GAN loss is defined as: (24) Discriminator The GAN loss is defined as: (25) (3) Beneficial effects In summary, compared with the prior art, the method of this embodiment has the following advantages and beneficial effects: 1) It proposes a multi-level adversarial strength evaluation index (MAIM), which not only focuses on pixel-level perturbation amplitude, but also captures the intrinsic influence of adversarial samples on deep neural networks (including feature-level semantic distortion and prediction-level probability divergence), thus achieving a more comprehensive quantitative characterization of adversarial strength.

[0082] 2) An Intensity-Aware Adversarial Sanitization Network (IAPN) is proposed. This network introduces a dynamic sanitization paradigm, which can sense the adversarial intensity of the input image and dynamically adjust the sanitization strategy and model parameters to achieve adaptive adversarial sanitization, thus achieving a better balance between adversarial robustness and standard accuracy.

[0083] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0084] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0085] Please refer to Figure 7, which illustrates the hardware structure of an electronic device according to another embodiment. The electronic device includes: a processor 701, which can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, for executing related programs to implement the technical solutions provided in the embodiments of this application; and a memory 702, which can be implemented using a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM), etc. The memory 702 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 702 and is called and executed by the processor 701. The input / output interface 703 is used to implement information input and output. The communication interface 704 is used to realize communication interaction between this device and other devices. Communication can be realized by wired means (such as USB, network cable, etc.) or by wireless means (such as mobile network, WIFI, Bluetooth, etc.). The bus 705 transmits information between various components of the device (such as processor 701, memory 702, input / output interface 703 and communication interface 704). The processor 701, memory 702, input / output interface 703 and communication interface 704 realize communication connection between each other within the device through the bus 705.

[0086] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0087] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0088] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0089] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0090] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented in the embodiments of this program product are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0091] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0092] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0093] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0094] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0095] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0096] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0097] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0098] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0099] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0100] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. An intensity-aware adaptive adversarial image defense method, characterized in that, The method includes the following steps: constructing a multi-level adversarial strength evaluation index; constructing an intensity-aware adversarial decontamination network, which includes an adversarial strength predictor and an intensity-guided adversarial decontamination, used to perceive the adversarial strength of the input image and perform adaptive decontamination; calculating the adversarial strength value of each adversarial sample in the training dataset based on the constructed multi-level adversarial strength evaluation index, and using this value as its strength label; training the adversarial decontamination network using an end-to-end joint optimization strategy, with consistency loss, strength loss, and adversarial loss as supervision, to obtain a trained reconstruction decontamination model; and using the trained reconstruction decontamination model to perform intensity-aware adaptive decontamination on the input adversarial image, restoring the adversarial image to a clean data distribution.

2. The method according to claim 1, characterized in that, The multi-level adversarial strength evaluation metrics include: pixel-level perturbation strength, calculated as the mean square error between the adversarial image and the corresponding clean image; feature-level semantic distortion, calculated as the mean square error between the class activation features of the adversarial image and the clean image on the pre-trained classifier; and prediction-level probability divergence, calculated as the KL divergence between the adversarial image and the clean image in the classifier output probability distribution.

3. The method according to claim 1, characterized in that, The adversarial strength predictor includes: a sparse self-attention Transformer encoder for extracting multi-scale features from the input image; a learnable feature fusion prediction head for predicting pixel-level, feature-level, and prediction-level adversarial strength components from the multi-scale features, and obtaining the overall adversarial strength through weighted fusion; and a dynamic weight mapper for generating a dynamic weight vector based on the overall adversarial strength and multi-scale features to guide the dynamic convolution operation in the purifier.

4. The method according to claim 3, characterized in that, The learnable feature fusion prediction head includes three parallel branches, corresponding to pixel-level, feature-level, and prediction-level intensity prediction, respectively. Each branch performs weighted fusion of multi-scale features through a learnable scale factor, and then obtains the corresponding intensity component through global average pooling and a multilayer perceptron.

5. The method according to claim 1, characterized in that, The intensity-guided adversarial cleanup includes: an intensity-guided multi-scale feature fusion module for adaptively fusing multi-scale input features according to the adversarial strength; an intensity-guided feature modulation module for modulating the channel and spatial dimensions of the features according to the adversarial strength to suppress error amplification; a Swin Transformer encoder for extracting adversarial robust features; and a dynamic convolutional decoder for reconstructing a clean image based on dynamic weight vectors.

6. The method according to claim 5, characterized in that, The intensity-guided multi-scale feature fusion module achieves fusion through the following steps: performing global average pooling on the multi-scale features and concatenating them; inputting the concatenated features and adversarial strength into a multilayer perceptron and a linear layer, respectively, to obtain scale attention weights and intensity modulation factors; The scale attention weight and intensity modulation factor are weighted and normalized to obtain the fusion weight for each scale. The multi-scale features are weighted and summed according to the fusion weights to obtain the fused features.

7. The method according to claim 5, characterized in that, The intensity-guided feature modulation module includes a channel attention branch and a spatial attention branch: the channel attention branch generates channel weights based on the feature global pooling result and the adversarial strength embedding. The spatial attention branch generates spatial weights based on the feature space pooling results and the adversarial strength map; The channel weights and spatial weights are combined into a modulation matrix to modulate the input features element by element.

8. The method according to claim 1, characterized in that, The consistency loss includes: pixel consistency loss, calculated as the L1 distance between the cleaned image and the clean image; feature consistency loss, calculated as the L2 distance between the class activation features of the image output by the cleaned network and the corresponding original clean image on the pre-trained classifier; and label consistency loss, calculated as the cross-entropy loss between the predicted label and the true label of the cleaned image on the classifier.

9. The method according to claim 1, characterized in that, The strength loss is the sum of the mean square errors of the predicted and actual confrontation strengths at each level.

10. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method according to any one of claims 1 to 9.

Images