A semantic-guided adversarial training method and device, and a storage medium

Abstract

This invention discloses a semantically guided inverse adversarial training method, apparatus, and storage medium, belonging to the field of artificial intelligence adversarial defense technology. This invention extracts a semantically key region mask of the target category in the input image, constructs an inverse loss function that correlates the confidence difference between the target category and the interference category; calculates the log-odds gradients of the two categories and weights them to obtain a dual collaborative gradient; using the dual collaborative gradient as the perturbation update direction, iteratively updates the adversarial perturbation under the constraint of the semantic mask, completing the inverse adversarial training of the image classification model. This invention confines the adversarial perturbation to the semantically key region, calibrates the perturbation update direction through dual gradient collaborative calibration, balances the accuracy of the model on clean samples with adversarial robustness, and allows for adaptive parameter adjustment, adapting to various adversarial defense scenarios for image classification tasks.

Description

Technical Field

[0001] This invention relates to the field of adversarial defense technology in artificial intelligence and deep learning, and in particular to a semantically guided reverse adversarial training method, apparatus and storage medium. Background Technology

[0002] Existing deep learning models generally possess inherent vulnerabilities and are highly susceptible to adversarial examples: attackers can easily induce the model to output incorrect predictions by simply superimposing minute pixel perturbations that are imperceptible to the human eye onto the original image. This security vulnerability not only significantly reduces the reliability of the model in practical applications but also causes misjudgments in high-risk scenarios such as autonomous driving decision-making, identity verification, and remote sensing disaster assessment, leading to serious application security and business risks.

[0003] To effectively enhance the defensive capabilities of deep neural networks, existing technologies have developed interpretability mechanisms based on Grad-CAM networks. These mechanisms extract feature contribution regions that significantly influence classification decisions from images, use thresholding to generate 0-1 masks, and combine Hadamard products to confine perturbations to semantically critical regions. Simultaneously, they construct objective functions that include adversarial loss and perturbation L2 norm regularization, and use stochastic gradient descent iterative optimization to generate locally imperceptible adversarial patches.

[0004] However, this scheme is only used to construct adversarial attack samples and only realizes the semantic constraints of the perturbation space. It fails to solve the shortcomings of traditional adversarial training, such as perturbation easily destroying clean features, decision boundaries easily oscillating, and difficulty in balancing clean accuracy and robustness. Summary of the Invention

[0005] The main objective of this invention is to provide a semantically guided reverse adversarial training method, apparatus, and storage medium, which aims to solve the technical problems of existing adversarial training perturbations lacking semantic constraints, easily destroying clean image features, ignoring class competition relationships leading to unstable decision boundaries, and making it difficult to balance the accuracy of clean samples and adversarial robustness of the model.

[0006] To achieve the above objectives, this invention provides a semantically guided reverse adversarial training method to improve the clean sample accuracy and adversarial robustness of image classification models. The method includes the following steps: Step 1: Based on the inference output of the image classification model to be trained on the input image, extract the semantic key region mask corresponding to the target category in the input image; After the input image is processed by the image classification model, it will output relevant information such as the prediction confidence of each category. Based on the output results, this step accurately locates and extracts the semantic key regions in the input image that play a decisive role in the classification of the target category (i.e., the category whose classification accuracy the model needs to optimize). The region is marked by masking (the region can be distinguished later by binarization). The core purpose is to define the effective range for subsequent adversarial perturbation updates, avoid perturbation applied to non-target category semantic regions, ensure that adversarial training focuses on the core features of image classification, and lay the foundation for balancing clean sample accuracy and adversarial robustness.

[0007] Step 2: Construct an inverse loss function that compares the prediction confidence of the target category and the interference category. Use the inverse loss function to constrain and adjust the deep features of the image classification model, thereby increasing the feature discrimination between the target category and the interference category. Interference categories refer to non-target categories that interfere with the model's correct classification of the target category. The inverse loss function constructed in this step is based on the difference in prediction confidence between the target category and the interference category. Unlike conventional loss functions that only focus on classification accuracy, it can quantify the difference in confidence between the two categories, providing a clear basis for subsequent gradient calculation, guiding the direction of gradient fusion, and ensuring that subsequent adversarial training can simultaneously take into account the classification accuracy of the target category and the ability to resist interference categories.

[0008] Step 3: Calculate the log-probability gradients of the target class and the interference class respectively, and perform weighted fusion of the two log-probability gradients to obtain the dual cooperative gradients; The log-odds gradient is used to reflect the sensitivity to changes in class prediction confidence. This step first calculates the log-odds gradients for the target class and the interfering class separately, and then weights and fuses the two gradients using a preset balancing hyperparameter to obtain a dual-cooperative gradient. This gradient fusion logic can integrate the information from the two types of gradients, avoiding training bias caused by a single gradient. This ensures that subsequent perturbation updates can both optimize the classification accuracy of the target class and resist adversarial attacks from the interfering class, achieving collaborative optimization of gradient calculation.

[0009] Step 4: Using the dual cooperative gradient as the update direction of the adversarial perturbation, under the constraint of the semantic key region mask, iteratively update the adversarial perturbation until the preset training conditions are met, and complete the reverse adversarial training of the image classification model. This step is the core execution stage of the reverse adversarial training. The dual collaborative gradient obtained in step 3 is used as the update direction for the adversarial perturbation, ensuring that the perturbation update aligns with the core objectives of "improving classification accuracy and enhancing robustness." Simultaneously, the semantic key region mask extracted in step 1 constrains the range of the perturbation's effect, preventing ineffective perturbations from affecting the model's clean sample classification performance. Through multiple iterations of perturbation updates and by controlling training termination through preset training conditions, the reverse adversarial training of the image classification model is ultimately achieved, realizing the goal of improving clean sample accuracy and adversarial robustness.

[0010] Optionally, the extraction of the semantic key region mask corresponding to the target category specifically includes: calculating the Grad-CAM heatmap corresponding to the target category; binarizing the Grad-CAM heatmap based on a preset threshold to obtain a binary semantic mask; wherein, in the Grad-CAM heatmap, the binary semantic mask has a value of 1 for regions with values ​​greater than or equal to the preset threshold, and a value of 0 for the remaining regions; after each adversarial perturbation update, the mask is multiplied element-wise with the binary semantic mask to limit the effective area of ​​the adversarial perturbation; the preset threshold is adaptively adjusted according to the training scenario of the image classification model and the category complexity of the input image.

[0011] Optionally, the inverse loss function is set with a preset suppression coefficient; the inverse loss function is constructed based on the base loss function, which is selected from the cross-entropy loss function; the suppression coefficient is used to adjust the weight of the difference in prediction confidence between the target category and the interference category.

[0012] Optionally, the suppression coefficient ranges from 0.1 to 10, and the value of the suppression coefficient is adaptively adjusted according to the confidence difference between the target category and the interference category.

[0013] Optionally, obtaining the dual collaborative gradient specifically includes: calculating the log-odds gradients of the target class and the interference class respectively; weighting and fusing the two log-odds gradients using a preset balancing hyperparameter to generate the dual collaborative gradient; the weights of the weighted fusion are adaptively allocated by the balancing hyperparameter.

[0014] Optionally, the value range of the balancing hyperparameter is 0.1 to 5, and the value of the balancing hyperparameter is adaptively allocated according to the weight requirements of the two log-probability gradients.

[0015] Optionally, the iterative update of the adversarial perturbation specifically includes: setting a preset perturbation budget to limit the maximum amplitude of the adversarial perturbation; the perturbation budget is set according to the pixel features of the input original image; updating the adversarial perturbation according to a preset iteration step size, wherein the iteration step size is adaptively adjusted according to the perturbation budget and is positively correlated with the perturbation budget; after each adversarial perturbation update is completed, the perturbation amplitude is cropped so that the cropped perturbation amplitude does not exceed the perturbation budget.

[0016] Optionally, the preset training conditions are: the number of adversarial perturbation update iterations reaches a preset iteration threshold, or the difference in prediction confidence between the target category and the perturbation category reaches a preset difference threshold.

[0017] To achieve the above objectives, the present invention also provides a semantically guided reverse adversarial training device, used to cooperate with an image classification model to complete reverse adversarial training, thereby achieving a synergistic improvement in the model's clean sample accuracy and adversarial robustness. The device includes: The mask extraction module is used to receive the target category prediction result output by the image classification model, calculate the Grad-CAM heatmap corresponding to the target category based on the Grad-CAM interpretability mechanism, perform binarization processing on the Grad-CAM heatmap through a preset threshold to generate a binary semantic mask, output the binary semantic mask to the perturbation update module, and receive the current adversarial perturbation fed back by the perturbation update module after each update of the adversarial perturbation, perform element-wise multiplication operation on the current adversarial perturbation and the binary semantic mask, and send the operation result back to the perturbation update module to realize the semantic region constraint of the perturbation; The loss function construction module is used to construct an inverse loss function based on a preset base loss function, configure a preset range of suppression coefficients, construct an inverse loss function that is associated with the difference in confidence levels between the target class and the interference class, and output the inverse loss function to the gradient fusion module to provide a loss constraint basis for gradient calculation. The gradient fusion module receives the prediction confidence of the target class and the interference class output by the image classification model, combines it with the inverse loss function output by the loss function construction module, calculates the log probability gradient of the target class and the interference class respectively, and uses a preset range of balanced hyperparameters to weight and fuse the two gradients to generate a dual collaborative gradient and output it to the perturbation update module as the update direction against perturbation. The perturbation update module receives the dual collaborative gradients output by the gradient fusion module, the semantically constrained perturbation output by the mask extraction module, and the binary semantic mask. It sets a preset perturbation budget to limit the maximum amplitude of the adversarial perturbation, it iteratively updates the adversarial perturbation according to a preset iteration step size, performs a pruning process on the perturbation amplitude after each update to ensure that the perturbation amplitude does not exceed the perturbation budget, and feeds back the current adversarial perturbation after each update to the mask extraction module for semantic constraint verification until the preset training conditions are met, completes the inverse adversarial training of the image classification model, and outputs the trained image classification model.

[0018] To achieve the above objectives, the present invention also provides a computer storage medium for storing a computer program that implements the semantically guided reverse adversarial training method described above. This computer storage medium can be a conventional storage medium such as a hard disk, USB flash drive, flash memory, or optical disc. The stored computer program can be called and executed by a processor, thereby implementing the semantically guided reverse adversarial training method. This method can be transformed into executable program code, facilitating its promotion, application, and deployment. It enables various devices with processors and storage capabilities (such as servers and terminal devices) to implement this reverse adversarial training, thereby improving the performance of image classification models.

[0019] The beneficial effects that this invention can achieve are: This invention proposes a semantically guided reverse adversarial training method, apparatus, and computer storage medium. Relying on the synergistic coupling of core technical solutions, it effectively overcomes the technical pain points of existing image classification model adversarial training, such as adversarial perturbations easily shifting to irrelevant semantic regions and the single dimension of gradient calculation, resulting in an inability to simultaneously achieve clean sample classification accuracy and model adversarial robustness. The specific beneficial effects are detailed below: This invention extracts a mask of the semantic key regions of the target category, strictly limiting adversarial perturbations to the core semantic regions of classification decisions, and avoiding ineffective perturbations acting on non-critical and irrelevant regions. This ensures that adversarial training always focuses on the core classification features of the image. While strengthening the model's resistance to attacks through perturbations, it preserves the model's classification performance on clean, unperturbed samples, fundamentally balancing accuracy on clean samples with adversarial robustness, and overcoming the classification accuracy degradation caused by traditional disordered perturbations.

[0020] This invention constructs an inverse loss function that correlates the confidence differences between the target class and the interference class, fusing the log-odds gradients of the two classes to form a dual collaborative gradient. This breaks the limitation of traditional single gradients that only focus on single-class optimization, providing a global basis for parameter iterative updates. By relying on dual gradient fusion to calibrate the update direction against adversarial perturbations, it optimizes the classification accuracy of the target class while effectively resisting malicious adversarial attacks from the interference class, significantly improving the overall adversarial robustness of the model and solving the problems of training bias and insufficient anti-attack capability caused by single gradient calculation.

[0021] This invention configures controllable parameters such as adaptively adjustable training threshold, suppression coefficient, balancing hyperparameter, perturbation budget, and iteration step size. These parameters can be dynamically adapted and optimized according to the model training scenario, image category complexity, and category confidence differences, greatly improving the scenario adaptability, versatility, and flexibility of the training scheme. It can adapt to the implementation requirements of various image classification tasks.

[0022] The device and method of this invention have one-to-one correspondence in technical features. Each functional module works in concert through signal triggering and data interaction to fully reproduce all execution logic of the method. This transforms the abstract training algorithm into a physical architecture that can be implemented in both software and hardware, reducing the difficulty of engineering implementation and facilitating the industrialization and application of the solution.

[0023] The program instructions corresponding to the training method solidified in the computer-readable storage medium of this invention can be run on various terminal devices with processing and storage capabilities. There is no need to repeatedly develop training logic, which greatly reduces the technical application threshold, broadens the applicable scenarios, and can be widely adapted to adversarial training scenarios in multiple fields such as computer vision and general image recognition.

[0024] In summary, this invention systematically addresses the inherent defects of existing image classification adversarial training through a three-in-one core architecture of semantic mask region constraint, inverse loss function guidance, and dual collaborative gradient fusion. Ultimately, it achieves multiple technical advantages, including simultaneously improving the accuracy and adversarial robustness of the model on clean samples, enhancing the adjustability and scene adaptability of training parameters, simplifying the technology deployment process, and broadening the application scope. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the algorithm architecture of an embodiment of the present invention; Figure 3 This is a schematic diagram of Grad-CAM semantic mask generation and perturbation constraints in an embodiment of the present invention; Figure 4 This is a schematic diagram comparing the clean accuracy of the embodiments of the present invention with various baseline methods on the CIFAR-10 dataset; Figure 5 This is a schematic diagram comparing the robust accuracy of the embodiments of the present invention with those of various baseline methods on the CIFAR-10 dataset; Figure 6 This is a schematic diagram comparing the clean accuracy of the embodiments of the present invention with those of various baseline methods on the EuroSAT dataset; Figure 7 This is a schematic diagram comparing the robust accuracy of the embodiments of the present invention with those of various baseline methods on the EuroSAT dataset; Figure 8 This is a schematic diagram of the device architecture according to an embodiment of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. These embodiments are only some embodiments of the present invention. Other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are all within the protection scope of the present invention.

[0027] Existing traditional adversarial training methods often use forward loss functions, focusing only on the accuracy of target category prediction. While improving adversarial robustness, this reduces the accuracy of clean sample classification. Furthermore, adversarial perturbations are random and unconstrained, easily disrupting image semantics and weakening the model's generalization ability. Single gradient updates also tend to cause training instability, making it difficult to balance clean sample accuracy and adversarial robustness, and failing to meet the high requirements of security-sensitive scenarios such as face recognition, autonomous driving, and remote sensing target recognition. To address these issues, this invention proposes a semantically guided inverse adversarial training method, device, and storage medium. Through semantic mask constraints, the construction of an inverse loss function, and dual collaborative gradient updates, it simultaneously improves the model's clean sample accuracy and adversarial robustness.

[0028] Reference Figure 1 , Figure 2 As shown, this invention provides a semantically guided adversarial training method to improve the clean sample accuracy and adversarial robustness of an image classification model. The method includes the following steps: Based on the inference output of the image classification model to be trained on the input image, extract the semantic key region mask corresponding to the target category in the input image; construct an inverse loss function that correlates the prediction confidence difference between the target category and the interference category, and use the inverse loss function to constrain and regulate the deep features of the image classification model, increasing the feature discrimination between the target category and the interference category; calculate the log-odds gradients of the target category and the interference category respectively, and perform a weighted fusion of the two log-odds gradients to obtain a dual collaborative gradient; using the dual collaborative gradient as the update direction of the adversarial perturbation, iteratively update the adversarial perturbation under the constraint of the semantic key region mask until the preset training conditions are met, thus completing the adversarial training of the image classification model.

[0029] refer to Figures 3-7 As shown, this embodiment takes the CIFAR-10 image classification task as the core application scenario, and uses two models, ResNet18 and EfficientNet-V2-S, to verify the method. The specific implementation process, parameter settings and operation details of the above method are described in detail.

[0030] Step 1: Based on the inference output of the image classification model to be trained on the input image, extract the semantic key region mask corresponding to the target category in the input image.

[0031] First, the preprocessed training samples (input image x) are input into the target image classification model (ResNet18 is preferred in this embodiment, and EfficientNet-V2-S will be used for verification later) to perform forward propagation, obtaining the model's class prediction output for the input image x, i.e., the logits of each class; based on this prediction output, the target class corresponding to the input image x is determined—here, the target class is uniformly defined as the Top-1 class with the highest predicted probability by the model. (If it is supervised training, the Top-1 class is consistent with the true label of the sample), and at the same time, the most competitive interference class with the target class is identified, that is, the Top-2 class with the second highest predicted probability. .

[0032] Based on target category The semantic key regions that play a decisive role in classifying the target category are extracted from the input image x, and a semantic key region mask M is generated. Here, a semantic key region refers to a region in the input image that contains the core features of the target category. For example, when the input image is a "cat" sample, the semantic key regions include the cat's head, torso, limbs, and other core parts, while background and irrelevant clutter are non-semantic key regions. The semantic key region mask M is a binary matrix with the same size as the input image (224×224 in this embodiment), used to accurately label semantic key regions (mask value of 1) and non-semantic key regions (mask value of 0). All subsequent adversarial perturbations are strictly limited to the region with a mask value of 1 to avoid perturbations disrupting the semantic structure of the image and ensure that the classification accuracy of clean samples is not affected.

[0033] The semantic mask is generated using Grad-CAM technology, which calculates the target category. Corresponding GradCAM heatmap Set a threshold τ, and obtain a binary mask M through binarization. Its mathematical expression is: ; in, Represents a binary semantic mask The Middle line, number The values ​​of the column pixels, Represents GradCAM heatmap The value at the corresponding position, τ is a preset threshold (initially set to 0.5 in this embodiment); and after each anti-perturbation update is completed, the following is executed: δ=δ⊙M semantic constraint operations ( ⊙ (This represents element-wise multiplication), ensuring that the perturbation is always confined to the semantically critical region.

[0034] Step 2: Construct an inverse loss function that compares the prediction confidence of the target category and the interference category. Use the inverse loss function to constrain and adjust the deep features of the image classification model, thereby increasing the feature discrimination between the target category and the interference category.

[0035] The core objective of this step is to construct an inverse loss function, which expands the target category. Interference category The difference in prediction confidence enables inverse margin optimization—even for the target class. Increased logarithmic probability, interference category The log odds are reduced, thereby enhancing the model's ability to identify the target category, while weakening the interference effect of the interfering category, thus improving the model's classification stability and adversarial robustness.

[0036] The inverse loss function is constructed based on the cross-entropy loss function (the basic loss function), and is specifically defined as follows: =λ·L( (x+δ),y 2 )−L( (x+δ),y 1 ) The meanings of each parameter are as follows: (x+δ) Add adversarial perturbations to the model pair δ The input image after x+δ The predicted output (log odds). The top-1 category (target category) with the highest predicted probability. The second most predicted category (interference category) is the Top 2 category. L Let cross-entropy be the loss function. λ The suppression strength coefficient (preset suppression coefficient) is used to adjust the weight of the difference in prediction confidence between the target class and the interference class. In this embodiment... λ The initial value was set to 1.0 (which was verified as the optimal value through ablation experiments).

[0037] The core logic of this inverse loss function is: through λ·L( (x+δ),y 2 ) Category of Suppression Interference The higher the prediction confidence (the higher the loss value, the better the model's prediction confidence). The less accurate the prediction, the better the inhibition effect. L( (x+ δ),y1 ) Improve target category The lower the prediction confidence (the smaller the loss value, the better the model's prediction confidence). The more accurate the prediction, the better. Combining the two achieves the effect of "suppressing interference and strengthening the target", thereby expanding the classification margin and improving the robustness of the model.

[0038] Step 3: Calculate the log-probability gradients of the target class and the interference class respectively, and perform weighted fusion of the two log-probability gradients to obtain the dual cooperative gradient.

[0039] Based on the inverse loss function Top1 constructed above, the target category is calculated respectively. The gradient of the log odds with respect to the input image x and interference categories The gradient of the log odds with respect to the input image x The two gradients are weighted and fused using a pre-defined balancing hyperparameter β to construct a dual cooperative gradient. It is used to guide the update direction of adversarial perturbation, avoid training bias caused by a single gradient, and improve training stability.

[0040] The specific formula for constructing the two cooperative gradients is as follows: in, Reflects the effect of pixel changes in the input image x on the target category The influence of prediction confidence, the direction of gradient direction makes The direction in which confidence increases; This reflects the effect of pixel changes in the input image x on the interference category. The influence of prediction confidence, the direction of gradient direction makes The direction in which confidence increases; β is a balancing hyperparameter used to adaptively allocate the weights of the two gradients. In this embodiment, β is initially set to 0.5 (verified as the optimal value through ablation experiments). Item balance To mitigate the impact of gradient explosion or vanishing, we must avoid excessive bias of the cooperative gradient towards the target class, ensure that the interfering class is effectively suppressed, and prevent gradient explosion or vanishing.

[0041] This step corresponds to the dual gradient collaborative adjustment step. Its core function is to provide a more reasonable direction for adversarial perturbation updates through weighted fusion of dual collaborative gradients, taking into account both the goals of "improving the confidence of the target class" and "suppressing the confidence of the perturbation class", thus solving the training instability problem caused by traditional single gradient updates.

[0042] Step 4: Using the dual collaborative gradient as the update direction of the adversarial perturbation, and under the constraint of the semantic key region mask, iteratively update the adversarial perturbation until the preset training conditions are met, thus completing the reverse adversarial training of the image classification model.

[0043] This step uses a dual cooperative gradient. To update the direction, under the constraint of the semantic key region mask M, the adversarial perturbation δ is iteratively updated through an iterative solver until the preset training conditions are met, thus completing the inverse adversarial training of the image classification model. The specific implementation process is as follows: 1) Initialize adversarial perturbation δ: Initialize δ to a zero matrix with the same size as the input image x (δ=0) to ensure that the input image is not perturbed in the initial state and to avoid the initial perturbation affecting the model's recognition of the target category.

[0044] 2) Iterative update of perturbation: The perturbation is updated using an iterative solver. The specific update formula is as follows: δ t ₊ 1 = (δ t +α·sign(∇ x Lᵢ n ᵥ)) in, δ t ₊ 1 For the first t ₊ 1 Adversarial perturbation after the next iteration update. δ t For the first t The perturbation after the next iteration. Here, α is the clipping function, used to clip the perturbation amplitude to a preset perturbation budget ε (to avoid excessive perturbation that could damage image semantics or cause model training failure). α is the iteration step size. sign(∇ x Lᵢ n ᵥ) Inverse loss function Lᵢ n ᵥ The gradient sign of the input image x (which guides the direction of perturbation updates). ∇ x Lᵢ n ᵥ That is, dual cooperative gradients (Both are essentially the same, representing the gradient of the inverse loss function with respect to the input image).

[0045] In this embodiment, the adversarial training parameters are set as follows (experimentally verified as the optimal configuration): the perturbation norm is the ℓ∞ norm, the perturbation budget ε=4 (for fair comparison, the traditional baseline method uses ε=8; this invention achieves better performance with a smaller budget), the step size α=1, and the number of iterations T=5 (i.e., t iterates from 1 to 5). It should be noted that in the subsequent robustness evaluation (white-box attack testing), to avoid overly stringent evaluation that might obscure the model's true performance, the perturbation budget ε=2, the step size α=1, and the number of attack iterations are uniformly set to 4.

[0046] 3) Semantic constraint application: After each perturbation update, the operation δ=δ⊙M is executed to set the perturbation of non-semantic key regions (mask value is 0) to 0, ensuring that the perturbation only acts on semantic key regions and avoids invalid perturbation from destroying the semantic structure of the image.

[0047] 4) Adversarial sample generation: After the iteration is completed, the final adversarial sample x′=Clip(x+δ,0,1) is generated, where Clip(·,0,1) is used to crop the pixel value of the adversarial sample to the range of 0~1, which conforms to the normal value range of image pixels.

[0048] 5) Model parameter update: Using the generated adversarial sample x′ and its true label y, optimize the model parameters through backpropagation to complete one inverse adversarial training; repeat all the above steps (mask extraction → loss construction → gradient calculation → perturbation update → sample generation → parameter update) to iteratively train all training samples in the CIFAR-10 dataset until the model converges (convergence criterion is that the clean accuracy of the validation set does not improve for 5 consecutive epochs), thus completing the inverse adversarial training of the entire image classification model.

[0049] Other parameters for model training are set as follows: the optimizer uses SGDwithmomentum0.9, the initial learning rate is 0.1, and the learning rate is adjusted using a cosine annealing strategy; the training hardware uses a single NVIDIA GPU to ensure training efficiency; the dataset preprocessing is as follows: the CIFAR-10 image size is adjusted to 224×224, and ImageNet statistics are used for normalization to ensure that the data distribution meets the model training requirements.

[0050] The method of this invention has four major advantages: 1. It balances the accuracy of clean samples with adversarial robustness, solves the pain points of traditional adversarial training, and is effective in multi-model datasets; 2. It improves training stability and generalization ability, avoids training fluctuations, and its necessity is verified by ablation experiments; 3. It has high training efficiency, strong practicality, compatibility with existing frameworks, can be directly integrated, and has controllable training time; 4. It is suitable for security-sensitive scenarios, reduces security risks, and has high practical value.

[0051] refer to Figures 6-7 As shown, further, extracting the semantic key region mask corresponding to the target category specifically includes: calculating the Grad-CAM heatmap corresponding to the target category; binarizing the Grad-CAM heatmap based on a preset threshold to obtain a binary semantic mask; wherein, in the region of the Grad-CAM heatmap with a value greater than or equal to the preset threshold, the binary semantic mask is set to 1, and the remaining regions are set to 0; adversarial perturbation is applied within the semantic key region corresponding to the mask; the preset threshold is adaptively adjusted according to the training scenario of the image classification model and the category complexity of the input image.

[0052] This embodiment uses the basic scenario (CIFAR-10 dataset, ResNet18 model) to further refine the extraction process of semantic key region masks, supplement the specific calculation steps of Grad-CAM heatmap and threshold adaptive adjustment scheme, and verify the adaptability of the scheme by combining it with other scenarios (such as EuroSAT remote sensing image classification).

[0053] Grad-CAM heatmap calculation corresponding to the target category; Grad-CAM (Gradient Weighted Class Activation Mapping) is a technique for visualizing regions of interest in a model. Its core advantage is that it can accurately locate regions in the input image that play a key role in the classification of the target category without modifying the model structure. It is compatible with various convolutional neural networks (such as ResNet, EfficientNet, etc.). The specific calculation process of the Grad-CAM heatmap in this embodiment is as follows, explained in detail with reference to the ResNet18 model: 1) Forward propagation to obtain feature maps: Input the input image x into the ResNet18 model and perform forward propagation until the last convolutional layer of the model (the last convolutional layer of ResNet18 is conv5_x). Obtain the feature map F output by the convolutional layer. The dimension of feature map F is C×H×W (C is the number of feature map channels, and H and W are the height and width of the feature map). 2) Calculate the target category gradient: based on the target category (Top-1 category), calculate the gradient ∇_Flogits of the log-odds ratio of that category with respect to the feature map F of the last convolutional layer. The gradient dimension is consistent with that of the feature map F (C×H×W). This gradient reflects the relationship between each channel and pixel of the feature map and the target category. The degree of influence of prediction confidence; 3) Global average pooling of gradients: For the gradients ∇_Flogits( Global average pooling (GAP) is performed to obtain the weight w_k of each feature map channel. The weight dimension is C×1×1. The calculation method is to globally average the gradient of each channel k to obtain the weight w_k of that channel. 4) Feature Map Weighted Fusion and Activation: The weight w_k of each channel is weighted and summed with the corresponding feature map F_k to obtain a preliminary class activation map. Then, the ReLU activation function is applied (to remove negative weight regions and retain only those relevant to the target class). (Regions with positive contributions) are used to obtain Grad-CAM heatmap M_cam with the same size as the input image (224×224).

[0054] In the Grad-CAM heatmap M_cam, the value of each pixel ranges from [0,1]. The larger the value, the more relevant the location is to the target category. The higher the importance of the classification (i.e., semantically key regions), the smaller the value, indicating that the location has less influence on the target category classification (i.e., non-semantically key regions). For example, when the input image is a "dog" sample, the core regions such as the dog's head and limbs in the Grad-CAM heatmap have higher values ​​(close to 1), while the background region has lower values ​​(close to 0), which can accurately distinguish between semantically key regions and non-key regions.

[0055] Grad-CAM thermal Figure 2 Binarization and Binary Semantic Mask Generation: A preset threshold τ is set, and the Grad-CAM heatmap M_cam is binarized to obtain a binary semantic mask M (consistent with the input image size of 224×224). The value of the pixel in the i-th row and j-th column of the binary semantic mask M is determined by the relationship between the value at the corresponding position in the Grad-CAM heatmap and the preset threshold τ. When the value at the corresponding position in the heatmap is greater than or equal to τ, the value at the corresponding position in the mask is 1; otherwise, it is 0.

[0056] In this embodiment, for the CIFAR-10 dataset (medium category complexity, image size 224×224), τ is initially set to 0.5. This threshold has been verified through multiple experiments to ensure accurate extraction of semantic key regions. It avoids both the omission of semantic key regions due to an excessively high threshold (e.g., retaining only a very small part of the target category, resulting in insufficient perturbation range and ineffective improvement of robustness) and the excessively large semantic key regions due to an excessively low threshold (e.g., containing a large amount of background regions, perturbation destroys image semantics, and reduces clean accuracy). Perturbation range limitation: The adversarial perturbation is strictly limited to the semantic key region corresponding to the binary semantic mask M. The specific implementation method is as follows: After each adversarial perturbation update, an element-wise multiplication operation of δ=δ⊙M is performed (⊙ means element-wise multiplication). The perturbation value of non-semantic key regions (the corresponding position of the mask is 0) is set to 0, and only the perturbation of semantic key regions (the corresponding position of the mask is 1) is retained. This ensures that the perturbation always acts on the region that plays a key role in the classification of the target category, and avoids invalid perturbation from destroying the semantic structure of the image. This is also one of the core technical features of this invention that can balance clean accuracy and adversarial robustness.

[0057] The preset threshold τ adopts an adaptive adjustment mechanism and is not a fixed constant. It needs to be dynamically adapted according to the training scenario of the image classification model and the category complexity of the input image to ensure the accuracy of semantic mask extraction. The threshold adaptation scheme for each scenario is explained in conjunction with the specific dataset below: In the Grad-CAM heatmap, pixel values ​​range from 0 to 1. Higher pixel values ​​indicate a greater contribution to the target category classification, suggesting a semantically critical region. Conversely, lower pixel values ​​indicate a smaller contribution, suggesting a non-semantically critical region. Taking an input sample of category "dog" as an example, the Grad-CAM heatmap shows higher values ​​for the dog's head, limbs, and other core areas, while lower values ​​are found in background-irrelevant areas, clearly distinguishing between semantically critical and non-critical regions.

[0058] hot Figure 2 Binarization and Binary Semantic Mask Generation: A preset threshold τ is set to binarize the Grad-CAM heatmap, generating a binary semantic mask with the same size as the input image. The value of each pixel in the mask is determined by the relationship between the corresponding value in the heatmap and the preset threshold τ: if the value at the corresponding position in the heatmap is not less than the threshold, the corresponding position in the mask is marked as a semantically valid region; otherwise, it is marked as a non-semantically valid region.

[0059] For the CIFAR-10 dataset, which has moderate category complexity and uniform image size, the initial threshold was set to 0.5. Multiple experiments verified that this value can accurately extract semantically critical regions: it avoids the loss of semantically critical regions due to a high threshold, preventing the retention of only small local areas of the target category and insufficient perturbation coverage, thus failing to effectively improve model robustness; conversely, it avoids the excessive expansion of semantically critical regions due to a low threshold, preventing the inclusion of too much redundant background areas, perturbation that disrupts the original semantics of the image, and thus reducing the accuracy of clean sample recognition.

[0060] The perturbation range is limited; adversarial perturbations are only applied within the semantically critical regions defined by the binary semantic mask. After each adversarial perturbation update, the perturbation amplitude in non-semantically critical regions is set to null using an element-wise association constraint, retaining only the effective perturbation components in the semantically critical regions. Throughout the process, it is ensured that the perturbation only acts on the core regions that play a decisive role in the target category classification, preventing invalid perturbations from disrupting the overall semantic structure of the image. This is the core design principle that enables this method to simultaneously achieve clean recognition accuracy and adversarial robustness.

[0061] 1) CIFAR-10 dataset: The class complexity is moderate, the target class features are relatively concentrated, and the threshold adaptation range is 0.4~0.5; 2) Remote sensing scene dataset: The category complexity is high, and the distribution of target category features such as farmland and buildings is relatively scattered. The threshold adaptation range is 0.3~0.5, and the optimal value is 0.4, which can prevent the threshold from being too high and missing semantic key regions, and ensure that the perturbation range completely covers all core features of the target category. 3) FDDB face detection dataset: It has concentrated features for a single face target category, but is easily affected by occlusion and pose changes. The threshold adaptation range is 0.5~0.7, with the optimal value being 0.6. It can accurately focus on the core feature regions of the face, such as facial features, weaken background interference, and improve the stability of model detection.

[0062] The core principle of threshold adjustment is: the higher the category complexity and the more dispersed the target features, the lower the value of τ; the lower the category complexity and the more concentrated the target features, the higher the value of τ, ensuring that the semantic mask can accurately cover the core area of ​​the target category while eliminating interference from non-critical areas.

[0063] This approach leverages Grad-CAM heatmaps to accurately extract key semantic regions from images, overcoming the blindness of traditional masks and effectively improving the effectiveness of perturbations and the robustness and accuracy of the model. The semantic mask employs simple and efficient binarization processing, requiring minimal computation and easily integrating for industrial training. It also allows for flexible threshold adjustment to adapt to different scenarios. The threshold can adaptively adapt to training scenarios and task complexity, exhibiting stronger versatility and generalization ability. Furthermore, perturbations are applied only to key regions without disrupting non-critical semantic structures of the image, significantly enhancing the model's adversarial robustness while effectively preserving the recognition accuracy of clean samples.

[0064] Furthermore, the inverse loss function has a preset suppression coefficient; the inverse loss function is constructed based on the base loss function, which is selected from the cross-entropy loss function; the suppression coefficient is used to adjust the weight of the difference in prediction confidence between the target class and the interference class.

[0065] In this embodiment, the selection of the base loss function and the inverse loss function are based on the cross-entropy loss function. The reasons for choosing the cross-entropy loss function as the base loss function are as follows: The cross-entropy loss function is the most commonly used loss function in image classification tasks. It can effectively measure the difference between the model's predicted output and the true label. It is simple to calculate, has a fast convergence speed, and is compatible with various deep learning models (ResNet, EfficientNet, etc.). At the same time, the cross-entropy loss function is sensitive to changes in the predicted probability of the class and can accurately capture the difference in the prediction confidence of the target class and the interference class. It is suitable for constructing the inverse loss function to achieve the effect of "suppressing interference and strengthening the target".

[0066] It should be noted that the basic loss function of this invention is not limited to the cross-entropy loss function. In other implementation scenarios, loss functions with similar functions to the cross-entropy loss function (such as FocalLoss) can also be selected. However, the cross-entropy loss function has the best computational efficiency and adaptability, and is the preferred solution of this invention.

[0067] The specific construction of the inverse loss function: The inverse loss function is constructed by combining the cross-entropy loss function. The construction logic is as follows: the cross-entropy loss function is combined in reverse. The negative cross-entropy loss term maximizes the prediction probability of the target class, so that the model predicts the target class more accurately; the weighted cross-entropy loss term minimizes the prediction probability of the interference class, so that the model predicts the interference class less accurately. The combination of the two realizes the expansion of the difference in prediction confidence between the target class and the interference class, thus completing the inverse margin optimization.

[0068] The role of the preset suppression coefficient λ: λ is the preset suppression coefficient, and its core function is to adjust the weight of the difference in prediction confidence between the target category and the interference category. Specifically, the value of λ determines the suppression strength of the interference category - the larger the λ is, the stronger the suppression strength of the interference category; the smaller the λ is, the weaker the suppression strength of the interference category. By adjusting the value of λ, the strength of "strengthening the target category" and "suppressing the interference category" can be balanced to avoid the problems of insufficient or excessive suppression.

[0069] In this embodiment, λ is initially set to 1.0 (which has been verified as the optimal value through ablation experiments). Under the CIFAR-10 dataset and ResNet18 model, this value allows the inverse loss function to achieve the best results, effectively suppressing the prediction confidence of the interference class without causing gradient explosion or training instability.

[0070] The inverse loss function constructed in this invention has significant advantages: it is built based on cross-entropy loss, which can widen the confidence gap between the target class and the interference class, optimize the decision boundary, and significantly improve the model's adversarial robustness; it does not require modification of the model structure and can be directly integrated into existing training frameworks, making integration convenient and implementation cost-effective; the suppression strength can be adjusted as needed through the suppression coefficient, adapting to different training scenarios and tasks, and has strong versatility; at the same time, it is simple to calculate and has good convergence, which can both accelerate the training convergence speed and avoid gradient anomalies, ensuring stable and efficient training.

[0071] Furthermore, the suppression coefficient ranges from 0.1 to 10, and the value of the suppression coefficient is adaptively adjusted according to the confidence difference between the target category and the interference category.

[0072] The range of the suppression coefficient λ is limited. In this embodiment, the range of λ is specified as 0.1 to 10. This range has been verified by a large number of experiments and can cover the needs of most image classification scenarios. The specific explanation is as follows: 1) When λ is less than 0.1: the suppression strength is too weak and cannot effectively suppress interference categories. The prediction confidence level is low, which prevents the difference between the confidence levels of the target class and the interference class from being widened. The inverse interval optimization effect is poor, and the improvement in model robustness is not significant. 2) λ value greater than 10: The suppression strength is too strong, which will lead to interference categories. Excessive suppression can lead to gradient explosion, making the training process unstable and even causing the model to fail to converge. 3) λ values ​​between 0.1 and 10: can balance the suppression strength and training stability, and can be adaptively adjusted according to the specific scenario to achieve the best inverse interval optimization effect.

[0073] The adaptive adjustment principle of the suppression coefficient λ: The value of λ is adaptively adjusted according to the confidence difference between the target class and the interference class. The core principle is as follows, explained in conjunction with specific scenarios: 1) When the initial confidence difference between the target class and the interference class is small (e.g., the difference is less than 0.2), the value of λ needs to be increased (e.g., 2.0 to 5.0) to enhance the suppression of the interference class and quickly expand the confidence difference; 2) When the initial confidence levels of the target class and the interference class differ significantly (e.g., the difference is greater than 0.5), the value of λ should be reduced (e.g., 0.5 to 1.0) to avoid over-suppression and ensure training stability. 3) For datasets with high class complexity (such as the EuroSAT remote sensing image dataset), the boundary between the target class and the interference class is blurred, so the value of λ needs to be increased appropriately (such as 1.0 to 2.0) to enhance the suppression effect; for datasets with low class complexity (such as CIFAR-10), a value of λ of about 1.0 can achieve the best effect.

[0074] Comparison of experimental results with different λ values ​​(based on CIFAR-10 and ResNet18 models): This embodiment verifies the impact of different λ values ​​on model performance through ablation experiments. The experimental results are as follows (combined with the inverse loss function): =λ·L( (x+δ),y 2 )−L( (x+δ),y 1 ) , 1) λ=0.5: Insufficient suppression, Top-2 category The competition is strong, the confidence difference between the target class and the interference class is small, and the model's robust accuracy is only 30.21%, lower than the optimal value of 36.07%. 2) λ=1.0: The suppression strength is moderate, the training is stable, the confidence difference between the target class and the interference class reaches the maximum, the clean accuracy of the model is 83.90%, and the robust accuracy of PGD is 36.07%, which is the best result; 3) λ=2.0: Oversuppression leads to gradient explosion, resulting in large fluctuations during training, difficulty in model convergence, and a final clean accuracy of 80.15% and a robust accuracy of 32.58%. 4) λ=5.0: Oversuppression is severe, the model cannot converge, and training fails.

[0075] The above experimental results fully verify the rationality of the range of λ values ​​(0.1 to 10) and the necessity of adaptive adjustment.

[0076] In this embodiment, the suppression coefficient λ is set to 0.1 to 10, which not only provides a standard basis for parameter configuration and reduces the difficulty of implementation, but also allows for adaptive adjustment based on confidence differences and dataset complexity to suit various application scenarios. At the same time, this value range can avoid the defects of insufficient suppression effect when λ is too small, and gradient explosion and training failure when λ is too large, thus ensuring training stability and effectively optimizing the overall performance of the model.

[0077] Furthermore, the dual collaborative gradient is obtained by: calculating the log-odds gradients of the target class and the interference class respectively; weighting and fusing the two log-odds gradients using a preset balancing hyperparameter to generate the dual collaborative gradient; and adaptively allocating the weights of the weighted fusion by the balancing hyperparameter.

[0078] Calculating the log-odds gradient between the target class and the interference classes: First, identify the target class as the Top 1 class. The interference category is Top-2. Based on the inverse loss function Find them separately and The gradient of the log odds with respect to the input image x and The specific calculation process is as follows: 1) Gradient Target Category The gradient of the logarithmic odds with respect to the input image x, i.e. Equals the input image x to the target category The gradient of the logarithmic odds, where, ( ) for model pair The predicted log odds, ∇_x represents the partial derivative with respect to the input image x; The dimensions are the same as the input image x (3×224×224, where 3 is the number of RGB channels), and the gradient direction points towards... The direction of increasing logarithmic probability reflects the effect of changes in the input image pixels on the logarithmic probability. The impact of prediction confidence. 2) Gradient Interference Category The gradient of the logarithmic odds with respect to the input image x, i.e. Equals the input image x to the interference category The gradient of the logarithmic odds has the same dimension as the input image x; the gradient direction points towards... The direction of increasing logarithmic probability reflects the effect of changes in the input image pixels on the logarithmic probability. The impact of prediction confidence.

[0079] In this embodiment, gradient calculation employs automatic differentiation technology (compatible with PyTorch and TensorFlow frameworks), eliminating the need for manual derivation of gradient formulas and enabling fast and accurate gradient acquisition. and This reduces the difficulty of implementation.

[0080] Weighted fusion of dual collaborative gradients: using a pre-set balancing hyperparameter β to balance the gradients and Perform weighted fusion to generate dual collaborative gradients The complete fusion formula is as follows: The dual collaborative gradient equals the target class gradient minus the product of the balancing hyperparameter and the perturbation class gradient, where β is the balancing hyperparameter used for adaptive allocation. and The weights are allocated based on the following logic: the value of β determines... The degree of influence on the cooperative gradient—the larger β is, The greater the weight, the better. The stronger the counteracting effect; the smaller the β, The smaller the weight, the more the collaborative gradient is biased towards By adaptively adjusting β, the strength of "improving the confidence of the target class" and "suppressing the confidence of the interfering class" is balanced.

[0081] In this embodiment, β is initially set to 0.5 (which has been verified as the optimal value through ablation experiments). Under the CIFAR-10 dataset and ResNet18 model, this value allows the cooperative gradient G_dual to achieve the best effect, guiding the perturbation towards improvement. The direction of confidence updates can effectively suppress... The confidence level is high enough to ensure training stability.

[0082] Dual collaborative gradients optimize gradient update directions by fusing and weighting the gradients of the target and interference classes, effectively mitigating training fluctuations, gradient explosion, or vanishing problems, improving training stability, and ensuring model performance. Simultaneously, it addresses the dual needs of strengthening the target and suppressing interference, widening the classification margin, enhancing the model's adversarial robustness, and adaptively allocating gradient weights through hyperparameters to suit diverse application scenarios. Furthermore, this approach relies on automatic differentiation, making it computationally simple, requiring no changes to the model structure, and easily integrated into existing training frameworks, thus enhancing its practicality and applicability.

[0083] Furthermore, the value of the balancing hyperparameter ranges from 0.1 to 5, and the value of the balancing hyperparameter is adaptively allocated according to the weight requirements of the two log-probability gradients.

[0084] The range of values ​​for the balancing hyperparameter β is limited: This embodiment specifies that the range of values ​​for β is 0.1 to 5. This range has been verified by a large number of experiments and can cover the needs of most image classification scenarios. The specific explanation is as follows: 1) β is less than 0.1: The weights are too small, and the collaborative gradient is too small. excessive bias This leads to interference categories The suppression is insufficient, the confidence difference between the target class and the interference class cannot be effectively amplified, and the improvement in model robustness is not significant. 2) β is greater than 5: The weights are too large, and the collaborative gradient is too large. excessive bias This leads to the target category Insufficient reinforcement may even inhibit The confidence level of the model decreases the clean accuracy. 3) β values ​​between 0.1 and 5: can achieve balance and The weights can be adaptively adjusted according to the specific scenario to achieve the best collaborative gradient effect, ensuring stable training and optimal model performance.

[0085] The adaptive adjustment principle of the balancing hyperparameter β: the value of β is determined based on the two log-probability gradients ( and The weight requirements are adaptively allocated, and the core principles are as follows, illustrated with specific scenarios: 1) When The amplitude is too large (e.g.) The L2 norm is greater than To double the value of β, the value of β needs to be increased (e.g., 1.0 to 2.0) to enhance its effectiveness. The weights, offset To avoid excessive influence and gradient explosion; 2) When The amplitude is too large (e.g.) The L2 norm is greater than (twice as), so the value of β needs to be reduced (e.g., 0.2–0.5), to lower Weighting to avoid over-suppressing the target category. ; 3) For datasets with high category complexity (such as EuroSAT). The magnitude of β is usually large, so the value of β needs to be reduced appropriately (e.g., 0.3 to 0.5). For datasets with low class complexity (e.g., CIFAR-10), a value of β of around 0.5 can achieve the best results.

[0086] Comparison of experimental results with different β values ​​(based on CIFAR-10 and ResNet18 models): This embodiment verifies the impact of different β values ​​on model performance through ablation experiments. The experimental results are as follows (combined with the dual collaborative gradient formula): 1) β=0.2: Cooperative gradient ,right Due to insufficient suppression and insufficient margin expansion, the model's robust accuracy was only 33.15%, lower than the optimal value of 36.07%. 2) β=0.5: The best balancing effect, stable training, clean model accuracy of 83.90%, PGD robust accuracy of 36.07%, and AutoAttack robust accuracy of 34.09%; 3) β=0.8: Over-suppression This leads to a distortion of the decision boundary, causing the model's clean accuracy to drop to 82.03% and its robust accuracy to 32.89%. 4) β=5.0: Over-suppression The clean accuracy of the model dropped to 78.56%, and the robust accuracy was only 29.32%.

[0087] The above experimental results fully verify the rationality of the β value range (0.1~5) and the necessity of adaptive adjustment.

[0088] This approach limits the balancing hyperparameter β to 0.1–5, providing a clear basis for parameter setting and reducing implementation difficulty. It also allows for adaptive parameter tuning based on gradient weight requirements and dataset complexity, making it suitable for various scenarios. Furthermore, it reasonably balances the influence of the two sets of gradients, avoiding the problems of insufficient suppression due to a small β and dragging down the target category recognition accuracy due to a large β, thus ensuring stable training and optimizing the overall performance of the model.

[0089] Furthermore, the iterative update of the adversarial perturbation specifically includes: setting a preset perturbation budget to limit the maximum amplitude of the adversarial perturbation; the perturbation budget is set according to the pixel features of the input original image; updating the adversarial perturbation according to a preset iteration step size, the iteration step size being adaptively adjusted with the perturbation budget and positively correlated; after each perturbation update, constraining the perturbation by a semantic key region mask, setting the perturbation amplitude of non-semantic key regions to 0; repeating the above perturbation update and semantic constraint steps until the adversarial perturbation meets the preset convergence condition, thus completing the iterative update of the adversarial perturbation.

[0090] This embodiment, based on the basic scenario (CIFAR-10 dataset, ResNet18 model), further refines the iterative update process of adversarial perturbation, and clarifies the setting principles of perturbation budget, iteration step size, and adaptive adjustment logic.

[0091] Setting and Basis of the Preset Perturbation Budget: The perturbation budget is a core parameter that limits the maximum magnitude of adversarial perturbations. It is used to avoid excessive perturbations that damage the semantic structure of the image (leading to a decrease in accuracy for clean samples), while ensuring that the perturbation has sufficient strength (to effectively improve the model's adversarial robustness). In this invention, the perturbation budget is set based on the pixel features of the input original image. The core basis includes the value range of image pixels, pixel distribution density, and the sensitivity of target category features. Specific implementation details are as follows:

[0092] 1) Definition of perturbation budget: In this embodiment, the perturbation budget is limited by the infinite norm, denoted as ε. That is, the maximum amplitude of the perturbation is constrained by the budget threshold. This norm is used to characterize the maximum absolute value of the perturbation at each pixel position, which can accurately control the maximum perturbation amplitude of a single pixel and avoid excessive local perturbation from damaging the image semantics.

[0093] 2) Impact of pixel features on perturbation budget: The pixel features of the original image directly determine the value of the perturbation budget. The specific principles are as follows: If the pixel value range of the original image is narrow (e.g., grayscale image, pixel value 0~255, and overall brightness is uniform), the pixels are less sensitive to perturbation, and the perturbation budget ε can be appropriately increased (e.g., ε=4~6); If the pixel value range of the original image is wide and the pixel distribution is uneven (e.g., remote sensing image, with areas of large brightness difference), the pixels are more sensitive to perturbation, and the perturbation budget ε needs to be appropriately reduced (e.g., ε=2~4) to avoid perturbation causing semantic distortion of the image; If the target category features are sensitive to pixel changes (e.g., facial features in a face image), the perturbation budget ε needs to be reduced (e.g., ε=2~3) to prevent perturbation from damaging the core features; If the target category features are not sensitive to pixel changes (e.g., remote sensing targets such as buildings and farmland), ε can be appropriately increased (e.g., ε=4~5).

[0094] In this embodiment, the pixel features of the CIFAR-10 dataset (RGB images, pixel values ​​normalized to 0~1, target category features are relatively concentrated and moderately sensitive to perturbation) are combined, and the perturbation budget ε=4 is preset (consistent with Example 1). This value has been experimentally verified to ensure that the perturbation has sufficient adversarial power without destroying the semantic structure of the image, thus achieving a balance between clean accuracy and robustness.

[0095] Setting and adaptive adjustment of the preset iteration step size: The iteration step size α is a key parameter for controlling the update speed of adversarial perturbation. An excessively large step size can easily lead to perturbation exceeding the budget and training oscillations; an excessively small step size will increase the number of iterations and reduce training efficiency. In this invention, the iteration step size is adaptively adjusted and positively correlated with the perturbation budget, i.e., the larger the perturbation budget ε, the larger the iteration step size α; the smaller the perturbation budget ε, the smaller the iteration step size α, ensuring that the perturbation update speed matches the budget, balancing training efficiency and stability.

[0096] 1) Initial setting of iteration step size: Based on the perturbation budget ε, the initial value of the iteration step size α follows the principle of α=ε / T, where T is the preset number of iterations. This ensures that within the preset number of iterations, the perturbation can gradually approach the upper limit of the budget, avoiding an excessively large or small step size. In this embodiment, the perturbation budget is 4, and the preset number of iterations is 5, consistent with Embodiment 1. Thus, the initial iteration step size is calculated to be 1. This value has been experimentally verified to achieve smooth updates of the perturbation and effectively avoid oscillations during the training process.

[0097] 2) Adaptive Adjustment Logic: During the perturbation iteration update process, the iteration step size α is dynamically adjusted based on the difference between the current perturbation amplitude and the budget. The specific adjustment rules are as follows: When the current perturbation amplitude does not exceed 0.5 times the perturbation budget, the perturbation amplitude is too small and far from reaching the budget limit. The step size α is increased by 10%~20%, for example, the step size is adjusted from 1 to 1.1, in order to speed up the perturbation update speed. When the current perturbation amplitude is in the range of 0.5 to 0.8 times the perturbation budget, the perturbation amplitude gradually approaches the budget limit. The step size α is kept unchanged to ensure that the perturbation smoothly approaches the budget limit. When the current perturbation amplitude exceeds 0.8 times the perturbation budget, the perturbation amplitude is close to the budget limit and is very likely to exceed the budget. The step size α is decreased by 10%~20%, for example, the step size is adjusted from 1 to 0.9, to prevent the perturbation amplitude from exceeding the preset budget.

[0098] In this embodiment, the adaptive adjustment of the iteration step size can be achieved through simple condition judgment without complex calculations. It can be quickly embedded into the iteration update process while taking into account both model training efficiency and operational stability.

[0099] Combination of perturbation update and semantic constraints: After each perturbation update, constraints are applied to the perturbation in combination with the extracted semantic key region mask M, so that the perturbation only acts on the semantic key region. Through element-wise multiplication, the perturbation value of non-semantic key region is set to 0, and only the perturbation value of semantic key region is retained.

[0100] The core function of this operation is to prevent invalid perturbations from damaging the semantic structure of non-critical regions of the image, ensuring that the accuracy of clean sample recognition is not affected; at the same time, it allows the perturbations to be concentrated on the core feature regions of the target category, thereby improving the overall effect of adversarial training.

[0101] Determination of Iteration Termination Condition: The above perturbation update and semantic constraint process is executed repeatedly until the adversarial perturbation meets the preset convergence condition, at which point the iterative update can be terminated. Iteration can be stopped if any of the following conditions are met: 1) The number of iterations reaches the preset value: The preset number of iterations in this embodiment is 5, which is consistent with Embodiment 1. After completing 5 iterations, the perturbation update is stopped and the final adversarial perturbation is generated; 2) The perturbation amplitude tends to be stable: If the difference between the perturbation amplitudes of two consecutive iterations is less than the preset threshold, it indicates that the perturbation state has basically stabilized, and continuing the iteration cannot significantly improve the adversarial effect, so the perturbation update is stopped immediately.

[0102] In this embodiment, the first convergence condition (iteration number T=5) is preferred. This method is simple and efficient, ensuring the consistency of the training process while avoiding the decrease in training efficiency caused by overly strict conditions for the stability of the perturbation amplitude. Combining the above steps, the complete iterative update process of the adversarial perturbation is as follows: 1) Initialize the perturbation as a zero matrix, set the perturbation budget ε=4, the initial iteration step size α=1, and the number of iterations T=5; 2) For t from 0 to T-1, perform the following operations: update the perturbation based on the dual collaborative gradient, and constrain the perturbation amplitude to not exceed the preset budget through the pruning function; adaptively adjust the iteration step size α: dynamically adjust the step size value according to the difference between the current perturbation amplitude and the budget threshold; semantic constraints: limit the updated perturbation to the semantic key region and complete the mask constraint processing; 3) After the iteration is completed, generate the final adversarial perturbation, construct adversarial examples based on the final perturbation, and then use them for model parameter updates.

[0103] By pre-setting and adaptively setting the perturbation budget, the perturbation amplitude can be controlled, the semantics of the image can be preserved, and the model can be kept clean with minimal loss of accuracy. The iteration step size is adaptively adjusted with the budget, balancing training efficiency and stability, effectively shortening the training time and reducing training fluctuations. Each perturbation update introduces semantic constraints, focusing on key regions of the image, reducing invalid perturbations, and significantly improving the model's adversarial robustness. At the same time, the parameters can be adaptively configured according to the image pixel features, without the need for manual parameter tuning. It can be adapted to multiple datasets and visual tasks, and is highly versatile and easy to implement.

[0104] Furthermore, the preset training conditions are: the number of adversarial perturbation update iterations reaches a preset iteration threshold, or the difference in prediction confidence between the target class and the perturbation class reaches a preset difference threshold.

[0105] This embodiment, based on the CIFAR-10 dataset and the ResNet18 model, further clarifies the specific meaning of the preset training conditions, the setting principles of the two thresholds, and the judgment logic. Combined with the parameter configuration of the previous embodiment, it ensures that the technical solution is fully disclosed and implementable, while maintaining consistency with the previous content and fully retaining the relevant judgment criteria.

[0106] The preset training conditions are the core criteria for terminating the reverse adversarial training of the image classification model. They are used to balance training effect and training efficiency, avoiding overtraining (leading to overfitting) or undertraining (leading to poor model performance). In this embodiment, the two training conditions are related by "OR", and training will be terminated if either one is satisfied. The details are as follows: Condition 1: The number of iterations to counteract disturbances reaches a preset iteration threshold. This condition is the preferred method for preset training conditions. It is simple, efficient, and easy to control. The core is to ensure that the perturbation can fully approximate the optimal value by limiting the number of iterations of the adversarial perturbation, while avoiding the decrease in training efficiency caused by too many iterations.

[0107] 1) Definition of preset iteration threshold: The preset iteration threshold is the maximum number of iterations to resist perturbation, denoted as T. It is consistent with the number of perturbation iterations in Examples 1 and 7, and is used to limit the number of update rounds of δ to ensure the consistency of the training process.

[0108] 2) Threshold setting principle: The preset iteration threshold T is adaptively set according to the perturbation budget ε and the iteration step size α. The core principle is to "ensure that the perturbation can approach the upper limit of the budget within the iteration threshold, while taking into account the training efficiency". The specific setting method is T=ε / α (rounded to the nearest integer).

[0109] In this embodiment, based on the parameters mentioned above (ε=4, α=1), the preset iteration threshold T=5. That is, after the adversarial perturbation has completed 5 iterations, regardless of whether the perturbation has reached a stable state, the training is terminated, and the final adversarial perturbation δ is generated and used for model parameter updates. This value has been experimentally verified to ensure that the perturbation has sufficient adversarial strength while avoiding the increase in computational overhead caused by too many iterations.

[0110] 3) Judgment logic: After each perturbation iteration update, the current iteration number t is counted. When t≥T, the training termination condition is triggered, the perturbation update and model training are stopped, and the reverse adversarial training process is completed.

[0111] Condition 2: The difference in prediction confidence between the target category and the interference category reaches a preset difference threshold. This condition is a supplementary training condition used to terminate training early when the number of perturbation iterations has not reached the threshold, but the model performance has met the requirements, thereby improving training efficiency. The core is to determine whether the model has achieved inverse interval optimization by monitoring the confidence difference between the target class and the perturbation class.

[0112] 1) Calculation of confidence difference: target category Interference category The difference in prediction confidence is calculated by the difference in prediction probabilities between the two, denoted as Δq. Specifically, Δq equals the target class... Predicted probability minus interference category The predicted probability. Where q_{ } represents the model's target category The predicted probability (obtained from the log odds using the softmax function), q_{ } represents the model's response to interference categories The predicted probability, Δq, ranges from [0,1]. The larger Δq is, the stronger the model's ability to identify the target category and the weaker the interference effect of the interference category.

[0113] 2) Setting the preset difference threshold: The preset difference threshold is denoted as Δq0. It is adaptively set according to the complexity of the dataset categories and the model training objective. The core principle is to "ensure that the model has achieved effective inverse margin optimization and can stably distinguish between the target category and the interference category." Examples of threshold settings for different scenarios are as follows: CIFAR-10 dataset (medium category complexity): Δq0=0.6, that is, when Δq≥0.6, it indicates that the confidence difference between the target category and the interference category is large enough, the model decision boundary is clear, and training can be terminated early; EuroSAT remote sensing image dataset (high category complexity): Δq0=0.5, due to the blurred category boundary, the threshold is appropriately reduced to ensure that the model can effectively distinguish between the target and the interference category; FDDB face detection dataset (focused on target features): Δq0=0.7, increasing the threshold ensures the model's recognition accuracy of face targets and reduces the influence of interference categories. In this embodiment, for the CIFAR-10 dataset, a preset difference threshold Δq0=0.6 is set. Experiments have verified that when Δq reaches this threshold, the clean accuracy and robust accuracy of the model are close to the optimal level, and further training cannot significantly improve performance, thus effectively shortening the training time.

[0114] 3) Decision logic: After each perturbation update and initial optimization of model parameters, calculate Δq = q_{ }-q_{ When Δq≥Δq0, the training termination condition is triggered, stopping the perturbation update and model training in advance without waiting for the number of iterations to reach the preset threshold.

[0115] In this invention, two preset training conditions are determined in parallel, and the execution logic is as follows: During the reverse adversarial training process, after each "perturbation update → semantic constraint → model parameter update" is completed, two conditions are determined simultaneously: if the number of iterations reaches a preset iteration threshold, or the confidence difference reaches a preset difference threshold, the training is terminated immediately; if neither condition is met, the next round of training continues until either condition is met.

[0116] This execution logic balances training stability and efficiency, avoiding both insufficient training due to insufficient iterations and increased computational overhead and overfitting due to overtraining, forming a complete closed loop with the training process described in the previous embodiment.

[0117] This solution uses quantitative thresholds to limit the preset training conditions, providing clear convergence criteria and facilitating implementation. It employs a dual-condition parallel decision based on iteration and confidence difference, ensuring sufficient training while allowing early termination to save training time, with minimal impact on model performance. The dual thresholds can be adaptively adjusted according to dataset complexity and training objectives, making it suitable for various image classification tasks and more versatile. Furthermore, it effectively avoids overfitting or underfitting issues, ensuring the model consistently reaches optimal performance.

[0118] refer to Figure 8 As shown, this embodiment of the invention provides a semantically guided reverse adversarial training device. The device includes: a mask extraction module, used to receive category prediction information output by an image classification model, extract a semantic key region mask corresponding to the target category in the input image, and after completing mask extraction, send a trigger signal to a loss function construction module, and simultaneously send the semantic key region mask to a perturbation update module; a loss function construction module, used to receive the trigger signal sent by the mask extraction module, construct an inverse loss function associated with the difference in prediction confidence between the target category and the perturbation category, and send the inverse loss function to a gradient fusion module; The gradient fusion module is used to calculate the log-odds gradients of the target class and the interference class based on the inverse loss function output by the loss function construction module, and to perform weighted fusion of the two log-odds gradients to obtain a dual collaborative gradient, and send the dual collaborative gradient to the perturbation update module; the perturbation update module is used to receive the semantic key region mask output by the mask extraction module and the dual collaborative gradient output by the gradient fusion module, and to iteratively update the adversarial perturbation under the constraint of the semantic key region mask with the dual collaborative gradient as the update direction, until the preset training conditions are met, thus completing the inverse adversarial training of the image classification model.

[0119] The device in this embodiment is compatible with datasets such as CIFAR-10 and EuroSAT, as well as models such as ResNet18 and EfficientNet-V2-S. It is seamlessly compatible with existing deep learning training frameworks (PyTorch, TensorFlow). Specific implementation details are as follows: Mask Extraction Module: 1) Core Function: Receives category prediction information (including log odds and prediction probabilities for each category) from the image classification model, extracts the semantic key region mask, and enables signal and data transmission between modules. It is the foundational module of the entire device. 2) Input and Output: Inputs are category prediction information from the image classification model and the original input image x; outputs are the semantic key region mask M and a trigger signal sent to the loss function construction module. Mask M is sent to the perturbation update module, and the trigger signal is used to start the loss function construction module, ensuring that each module executes in an orderly manner. 3) Implementation: Grad-CAM technology is used to calculate the target category. y 1 The heatmap is binarized to obtain a binary mask M. The preset threshold τ is adaptively adjusted according to the training scenario. At the same time, the threshold adaptive adjustment logic can be integrated as needed to ensure the accuracy of mask extraction.

[0120] Loss Function Construction Module: 1) Core Function: After receiving the trigger signal from the mask extraction module, it constructs the inverse loss function to optimize the inverse interval between the target category and the interference category, providing a foundation for gradient calculation. It is one of the core computational modules of the device. 2) Input and Output: The input is the trigger signal sent by the mask extraction module; the output is the constructed inverse loss function. The data is sent to the gradient fusion module for gradient calculation. 3) Specific implementation: The loss function is constructed based on the cross-entropy loss function, with a preset suppression coefficient λ (range 0.1~10). λ can be adaptively adjusted according to the confidence difference between the target class and the interference class, while fully retaining the inverse loss function formula. =λ·L( (x+ δ),y 2 )−L( (x+δ),y 1 ) This ensures a complete match with the method implementation.

[0121] Gradient Fusion Module: 1) Core Function: Based on the inverse loss function, calculates the log-odds gradients of the target class and the perturbation class, and obtains a dual collaborative gradient through weighted fusion, providing direction for adversarial perturbation updates and solving the training instability problem caused by traditional single gradients. 2) Input and Output: The input is the inverse loss function output by the loss function construction module. The output is a dual cooperative gradient. The data is sent to the perturbation update module to guide the perturbation update direction. 3) Specific implementation: Gradient calculation adopts automatic differentiation technology, calculating the gradient separately. = and = Then, weighted fusion is performed using the balancing hyperparameter β (ranging from 0.1 to 5), and the fusion formula is as follows: The value of β is adaptively adjusted according to the weight requirements of the two gradients.

[0122] Perturbation Update Module: 1) Core Function: Receives the mask M from the mask extraction module and the dual cooperative gradient from the gradient fusion module. The core execution module of the device completes the iterative update of adversarial perturbations, semantic constraints, and termination determination of model training. 2) Input and output: The input is the semantic key region mask M, the dual collaborative gradient G_dual, and the original input image x; the output is the trained image classification model (after parameter optimization) and the finally generated adversarial example x′. 3) Specific implementation: Set the preset perturbation budget ε (set according to the pixel features of the original image) and the iteration step size α (adaptively adjusted with ε). After each perturbation update, execute the semantic constraint δ=δ⊙M, and iterate until the preset training conditions are met (the number of iterations reaches the threshold or the confidence difference reaches the threshold). At the same time, the adversarial example is generated and the model parameters are updated to ensure that the process and parameters are completely matched with those of the method embodiment.

[0123] Each module works collaboratively in the order of “mask extraction module → loss function construction module → gradient fusion module → perturbation update module” to form a complete reverse adversarial training process: first, the mask extraction module completes mask extraction and triggers subsequent modules, then the loss function construction module and gradient fusion module complete the core calculation, and finally the perturbation update module performs iterative updates and training termination. Data transmission and signal transmission of each module are carried out in an orderly manner to ensure stable operation of the device.

[0124] This device adopts a clear modular division, with each module having a well-defined responsibility and working collaboratively, facilitating development, maintenance, and functional upgrades. The overall modular design offers good compatibility, seamlessly integrating with existing deep learning frameworks and models, enabling rapid application in various scenarios such as facial recognition and autonomous driving, and demonstrating strong portability. It can automatically complete the entire process from mask extraction to training termination, requiring only the setting of initial parameters to run automatically, significantly reducing manual costs and adapting to large-scale industrial applications. At the same time, each module is implemented strictly according to the method implementation examples, ensuring stable and reliable performance, and simultaneously improving the model's clean accuracy and adversarial robustness.

[0125] This invention provides a computer storage medium storing a computer program. When the computer program is executed by a processor, it implements the semantic-guided reverse adversarial training method described in the above embodiments.

[0126] The computer storage medium in this embodiment is used to store the computer program that implements the above-described reverse adversarial training method, providing a carrier for the implementation of the method, adapting to various computer devices, and ensuring that the method can be executed stably and efficiently. Specific details are as follows: Type of storage medium: The computer storage medium of the present invention may be selected from commonly used storage media types in the art, including but not limited to: hard disk (HDD), solid-state hard disk (SSD), USB flash drive, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, etc. All media capable of storing computer programs are within the protection scope of the present invention.

[0127] The computer program contains the following: a computer program stored on a storage medium, which includes all steps of implementing the method described in any one of embodiments 1-8, specifically including: semantic key region mask extraction step, inverse loss function construction step, dual collaborative gradient calculation step, adversarial perturbation iterative update step, and model training step. It also includes parameter settings, formula calculations, logical judgments, and other code for each step, ensuring that the entire process of the above method can be completely reproduced when the program is executed by the processor.

[0128] Execution environment requirements: This computer program can be executed on any computer device with data processing capabilities, including but not limited to: personal computers (PCs), servers, workstations, embedded devices, etc. The execution device must have basic hardware such as processors (such as CPUs, GPUs) and memory, and have the corresponding operating system (such as Windows, Linux, Unix) and deep learning training framework (such as PyTorch, TensorFlow) installed to ensure that the program can be compiled and executed normally.

[0129] Execution process: When the computer program is executed by the processor, the processor will execute the steps of the inverse adversarial training in sequence according to the instructions in the program: First, read the input image and model parameters, and extract the semantic key region mask; then construct the inverse loss function and calculate the dual collaborative gradient; then iteratively update the adversarial perturbation and apply semantic constraints until the preset training conditions are met; finally, complete the model parameter update and output the trained model and adversarial samples.

[0130] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A semantic-guided adversarial training method, characterized in that, To improve the clean sample accuracy and adversarial robustness of image classification models, the following steps are included: Based on the inference output of the image classification model to be trained on the input image, extract the semantic key region mask corresponding to the target category in the input image; An inverse loss function is constructed to measure the difference in prediction confidence between the target category and the interference category. This inverse loss function is then used to constrain and regulate the deep features of the image classification model, thereby increasing the feature discrimination between the target category and the interference category. The log-probability gradients of the target class and the interference class are calculated separately, and the two log-probability gradients are weighted and fused to obtain the dual cooperative gradients. Using the dual collaborative gradient as the update direction of the adversarial perturbation, and under the constraint of the semantic key region mask, the adversarial perturbation is iteratively updated until the preset training conditions are met, thus completing the reverse adversarial training of the image classification model.

2. The method of claim 1, wherein, Extracting the semantic key region mask corresponding to the target category in the input image, specifically including: Calculate the Grad-CAM heatmap corresponding to the target category; The Grad-CAM heatmap is binarized based on a preset threshold to obtain a binary semantic mask; wherein, the binary semantic mask is set to 1 for regions in the Grad-CAM heatmap with values ​​greater than or equal to the preset threshold, and 0 for the remaining regions. Adversarial perturbations are applied only within the semantically critical region corresponding to the mask; The preset threshold is adaptively adjusted based on the training scenario of the image classification model and the category complexity of the input image.

3. The method of claim 1, wherein, The inverse loss function is set with a preset suppression coefficient; The inverse loss function is constructed based on the base loss function, which is selected from the cross-entropy loss function; The suppression coefficient is used to adjust the weight of the difference in prediction confidence between the target class and the interference class.

4. The method of claim 3, wherein, The suppression coefficient ranges from 0.1 to 10, and its value is adaptively adjusted according to the confidence difference between the target category and the interference category.

5. The method according to claim 1, characterized in that, The acquisition of the dual cooperative gradient specifically includes: Calculate the log-probability gradients of the target class and the interference class respectively; The two log-probability gradients are weighted and fused using a preset balancing hyperparameter to generate the dual collaborative gradient. The weights for the weighted fusion are adaptively assigned by the balancing hyperparameter.

6. The method according to claim 5, characterized in that, The value of the balancing hyperparameter ranges from 0.1 to 5, and the value of the balancing hyperparameter is adaptively allocated according to the weight requirements of the two log-probability gradients.

7. The method according to claim 1, characterized in that, The iterative update to counteract disturbances specifically includes: A preset perturbation budget is set to limit the maximum amplitude of the perturbation; the perturbation budget is set based on the pixel features of the input original image; The system updates the anti-disturbance according to a preset iteration step size, and the iteration step size is adaptively adjusted with the disturbance budget and is positively correlated. After each counter-disturbance update, the disturbance amplitude is pruned so that the pruned disturbance amplitude does not exceed the disturbance budget.

8. The method according to claim 1, characterized in that, The preset training conditions are: the number of adversarial perturbation update iterations reaches a preset iteration threshold, or the difference in prediction confidence between the target category and the perturbation category reaches a preset difference threshold.

9. A semantically guided reverse adversarial training device, characterized in that, include: The mask extraction module is used to receive category prediction related information output by the image classification model, extract the semantic key region mask corresponding to the target category in the input image, and send a trigger signal to the loss function construction module after completing the mask extraction, while sending the semantic key region mask to the perturbation update module. The loss function construction module is used to receive the trigger signal sent by the mask extraction module, construct an inverse loss function associated with the difference in prediction confidence between the target category and the interference category, and send the inverse loss function to the gradient fusion module. The gradient fusion module is used to calculate the log-odds gradients of the target class and the interference class based on the inverse loss function output by the loss function construction module, perform weighted fusion of the two log-odds gradients to obtain the dual cooperative gradients, and send the dual cooperative gradients to the perturbation update module. The perturbation update module is used to receive the semantic key region mask output by the mask extraction module and the dual cooperative gradient output by the gradient fusion module. Using the dual cooperative gradient as the update direction of the adversarial perturbation, it iteratively updates the adversarial perturbation under the constraint of the semantic key region mask until the preset training conditions are met, thus completing the reverse adversarial training of the image classification model.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, implements the semantically guided reverse adversarial training method according to any one of claims 1-8.

Images