Robustness enhancement method for traffic sign recognition model based on adversarial attack and sensitive layer visualization

Abstract

The invention discloses a traffic sign recognition model robustness enhancement method based on adversarial attack and sensitive layer visualization. According to the method, a convolutional layer of a model is visualized through a sensitive layer visualization module by using a module class activation mapping (CAM) technology, the most sensitive convolutional layer of the model is found when an adversarial sample is processed, and the layer is determined as a key layer for further optimization. And in the adversarial training module, adversarial training is emphatically performed on the key layer determined by the sensitive layer visualization module, so that the model can better adapt to adversarial disturbance. And finally, in an adversarial purification module, a disturbed adversarial sample is detected by using a loss function and is subjected to adversarial purification operation, so that the interference of the adversarial sample on the model is reduced, and the robustness of the model when the model encounters a novel attack or is not attacked is further improved.

Description

Technical Field

[0001] This invention relates to the field of deep learning and adversarial defense technology, especially the study of combining the interpretability of deep neural networks with adversarial training methods, aiming to improve the robustness and classification performance of traffic sign recognition models. In the academic field, this method can be used to study the behavior and improvement strategies of models under adversarial attacks; in the industrial field, especially in high-risk scenarios such as autonomous driving and intelligent transportation systems, this method can be applied to improve the safety and reliability of models. Background Art

[0002] With the rapid development of deep learning technology, deep neural networks (DNNs) have achieved great success in computer vision, natural language processing, and speech recognition. In particular, in the field of traffic sign recognition, models based on deep learning provide important technical support for autonomous driving technology. Traffic sign recognition models can quickly and accurately identify traffic signs and provide key information for autonomous vehicles. However, with the widespread application of deep neural networks, their adversarial vulnerability problems have gradually been exposed. Studies have shown that deep neural networks are vulnerable to adversarial attacks, that is, by applying small perturbations to the input data, attackers can significantly change the output results of the model, and these perturbations are usually difficult for humans to perceive. This problem is particularly serious in safety-critical scenarios such as autonomous driving, because adversarial attacks may cause vehicles to misrecognize traffic signs, thereby causing safety accidents. Therefore, improving the robustness of traffic sign recognition models has become a crucial research topic. Adversarial attacks are one of the hot research directions in the field of machine learning and deep learning in recent years. For traffic sign recognition models, small adversarial perturbations may cause the system to mistakenly identify speed limit signs as stop signs, or recognize no-entry signs as pass signs. This misrecognition will cause serious safety hazards in autonomous driving. Therefore, it is of great practical significance to study how to defend against adversarial attacks and improve the robustness of traffic sign recognition models. Adversarial training is one of the main methods to improve the robustness of models. In adversarial training, the model is trained not only on clean samples, but also on adversarial samples. By exposing the model to adversarial perturbations during the training process, the model can learn to recognize and resist such attacks. However, adversarial training also faces some challenges, such as a more complex training process, higher computational overhead, and more prone to overfitting of the model. To this end, many researchers are committed to developing more efficient and robust adversarial training methods. In addition to adversarial training, other adversarial defense strategies have also been widely studied. Adversarial Purification is a common defense method that preprocesses the input data during the test phase to remove adversarial perturbations, thereby reducing the impact of adversarial attacks on the model. By combining adversarial training with adversarial purification, the robustness of the model can be further improved. Although adversarial training and defense strategies can improve the robustness of the model, how to understand the behavior of the model when facing adversarial attacks remains an important issue. In practical applications, deep neural networks are often considered black box models, and their internal decision-making process is difficult to explain. This problem is particularly serious in high-risk scenarios. For example, in an autonomous driving system, if the system incorrectly recognizes a traffic sign, the operator or developer needs to be able to quickly find the root cause of the problem in order to take appropriate measures.Explainability techniques aim to reveal the decision-making process of deep neural networks. One common method is Class Activation Mapping (CAM). CAM visualizes the activations of different layers in the model and reveals the areas that the model focuses on when making decisions. In the scenario of adversarial attacks, using CAM can help us discover the sensitive layers of the model, that is, the layers that are most sensitive to adversarial perturbations. By conducting targeted training on these sensitive layers, the robustness of the model can be improved when facing adversarial samples.

[0003] Although significant progress has been made in the field of interpretability and adversarial defense in recent years, there are still many challenges. First, when performing adversarial defense on a model, the model is in a black box state, with poor interpretability and cannot be used for targeted adversarial training; second, when performing adversarial defense, it can only respond to certain attacks and has poor robustness to other types of attacks. Therefore, the present invention proposes a model robustness enhancement method based on adversarial attacks and sensitive layer visualization, which is applied to the classification of deep neural network models for traffic sign recognition. Summary of the invention

[0004] Purpose of the invention: To address the problem that traffic sign recognition models are vulnerable to adversarial attacks and the decision-making process is unexplainable, a method for enhancing the robustness of traffic sign recognition models based on adversarial attacks and visualization of sensitive layers is invented. This method uses the class activation mapping (CAM) technology to discover sensitive layers in convolutional neural networks, identify the areas affected by adversarial perturbations, and perform targeted adversarial training on these areas. By combining adversarial purification technology in the testing phase, the robustness of the model under adversarial samples is further improved, ultimately achieving model security enhancement and explainability verification.

[0005] Technical solution:

[0006] 1. The robustness enhancement method of traffic sign recognition model based on adversarial attack and sensitive layer visualization generally includes the following steps:

[0007] Step 1.1: Construct an adversarial attack module, generate adversarial samples through common adversarial attack methods (such as FGSM, PGD, etc.), and apply small perturbations to simulate attack scenarios. Input these adversarial samples into the traffic sign recognition model to detect the behavior of the model under attack. This module is used to simulate actual adversarial attack scenarios and evaluate the robustness of the model;

[0008] Step 1.2: Construct a sensitive layer visualization module to visualize the convolutional layers of the model based on the class activation mapping technique. By analyzing the activation maps of each layer of the model, the convolutional layers that are most sensitive to adversarial samples are found. This module is used to identify which convolutional layers are most susceptible to adversarial perturbations, thereby determining the key layers that need further optimization;

[0009] Step 1.3: Construct an adversarial training module, perform adversarial training on the model, and conduct targeted adversarial training on the sensitive layers found in step 1.1 to enhance the intensity of sensitive layer training. By optimizing the objective function, the robustness of the model under adversarial perturbations is enhanced, and the layers with the greatest impact of adversarial perturbations are trained in a focused manner, so that the model can make more robust decisions when facing adversarial samples;

[0010] Step 1.4: Construct an adversarial purification module to preprocess the input samples during the test phase and remove possible adversarial perturbations through adversarial purification techniques. According to the defense strategies learned during the training process, dynamically select appropriate purification operations to ensure that the model can maintain high robustness and recognition accuracy under various attack methods during the test phase.

[0011] 2. The method of the adversarial attack module in step 1.1 is as follows:

[0012] Step 2.1: Select the traffic sign image as input and input it into the adversarial attack module. The input can be the original image or the processed adversarial sample for adversarial attack evaluation;

[0013] Step 2.2: Apply an adversarial attack algorithm (such as FGSM or PGD) to calculate adversarial perturbations and generate adversarial samples. First, calculate the loss function of the model, calculate the gradient of the image, and obtain the perturbation direction. Then, according to the set perturbation intensity, adjust the input image to generate adversarial samples, which can be expressed as:

[0014]

[0015] Where x is the original input sample, x′ is the adversarial sample, and ∈ represents the perturbation intensity, which controls the perturbation amplitude of the original image x. The larger the value, the more significant the perturbation, and the adversarial sample may cause the model to misclassify;

[0016] Step 2.3: Input the generated adversarial examples into the traffic sign recognition model, record the output results of the model under the adversarial examples, and evaluate the robustness of the model and the impact of adversarial attacks.

[0017] 3. The method of the sensitive layer visualization module in step 1.2 is as follows:

[0018] Step 3.1: Use the Class Activation Mapping (CAM) technique to calculate the class activation map based on the activation map of a specific convolutional layer and the corresponding weights. First, select the convolutional layer and obtain the activation feature map of the layer:

[0019] A=f(x;θ conv )

[0020] Where f is the output of the convolutional layer, x is the input image, and θ conv are the parameters of the convolutional layer.

[0021] Then, get the fully connected layer weights:

[0022] w k =Weights of class k in the fully connected layer

[0023] Finally, for each class k, according to the activation feature map A and weight w k Compute class activation map CAM k :

[0024]

[0025] Among them, A i is the i-th channel in the feature map, is the i-th weight associated with class k;

[0026] Step 3.2: Compute the sensitivity of the convolutional layer by comparing the activation maps of the original image and the adversarial sample:

[0027]

[0028] in, is the i-th activation feature map of the l-th layer, is the corresponding adversarial sample activation feature map, N is the number of feature maps;

[0029] Step 3.3: Normalize the generated class activation maps to visualize them. Show the activation strength in the form of heat maps to highlight the most sensitive areas in the adversarial examples. Based on the heat map analysis results, identify the convolutional layers that show high sensitivity under adversarial perturbations. These layers will be considered key layers that need further optimization and adversarial training.

[0030] 4. The method of the adversarial training module in step 1.3 is as follows:

[0031] Step 4.1: Select the adversarial samples and original samples generated in step 1.1 and input them into the adversarial training module;

[0032] Step 4.2: Extract the activation feature maps of these convolutional layers based on the sensitive layer information found in step 1.2;

[0033] Step 4.3: Define the optimization objective function for each sensitive layer, which can calculate the performance of the model under adversarial examples based on the cross entropy loss, expressed as:

[0034]

[0035] Among them, L(θ) is the target loss function, N is the number of samples, and y i is the true label, f(x i ′; θ) is the output of the model;

[0036] Step 4.4: During the training process, the convolution kernel of the sensitive layer is dynamically adjusted to adapt to the characteristics of the adversarial sample. The parameters of the sensitive layer are optimized by the gradient descent method to make the model better adapt to the adversarial perturbation. The update rule is:

[0037]

[0038] Among them, η is the learning rate and θ′ is the updated parameter.

[0039] 5. The method for the antagonistic purification module in step 1.4 is as follows:

[0040] Step 5.1: In the testing phase, receive the input sample x, evaluate the characteristics of the input sample based on the defense strategy learned by the model in the training phase, and determine whether it is affected by the adversarial perturbation by calculating the loss function L of the input sample:

[0041]

[0042] Where N is the number of samples, y i is the true label, f(x i ; θ) is the output of the model;

[0043] Step 5.2: Perform feature space projection operation on the input sample that has been subjected to adversarial perturbation to purify the input sample:

[0044] x clean =P(f(x;θ))

[0045] Where P represents the feature space projection operation;

[0046] Step 5.3: Re-input the cleaned sample into the model for testing.

[0047] 6. The overall loss function L in the robustness enhancement method of traffic sign recognition model based on adversarial attack and sensitive layer visualization is:

[0048] L=L pred +λ1L adv +λ2L sensitivity +λ3L regularization

[0049] Among them, the loss function is divided into four items, which are explained as follows:

[0050] (1) Prediction loss L pred :

[0051] This item is used to measure the classification accuracy of the model on the training data, usually expressed as a cross entropy loss function:

[0052]

[0053] Where N is the number of samples, y i is the true label, f(x i ; θ) is the output of the model, θ is the model parameter;

[0054] (2) Adversarial Loss L adv :

[0055] This term is used to evaluate the robustness of the model under adversarial samples and is usually calculated through adversarial training:

[0056]

[0057] Among them, x adv,i is the generated adversarial sample, ∈ is the tolerated error threshold;

[0058] (3) Sensitivity loss L sensitivity :

[0059] This term is used to quantify the sensitivity of the model to a specific convolutional layer under adversarial perturbations, usually by analyzing the activation map:

[0060]

[0061] Among them, A j is the activation map of the jth layer, A j,clean is the activation map of the corresponding unperturbed sample;

[0062] (4) Regularization term L regularization :

[0063] This term is used to prevent the model from overfitting and is usually expressed using L2 regularization:

[0064]

[0065] Among them, W j is the weight matrix of the jth layer;

[0066] (5) Hyperparameters λ1, λ2, λ3:

[0067] These hyperparameters are used to balance the weights between the various losses and adjust the bias of the model during training to achieve optimal performance and robustness.

[0068] Beneficial effects of the present invention:

[0069] 1. In the field of academic research, this method can be used to improve the robustness of traffic sign recognition models, especially in the context of adversarial attacks. Through visualization of sensitive layers and targeted adversarial training, researchers can gain a deep understanding of the model's performance under different input perturbations, thereby promoting the development of interpretability research on deep learning models.

[0070] 2. In the field of engineering applications, this method can be widely used in decision-making and classification tasks in high-risk scenarios such as intelligent transportation systems and autonomous driving. By enhancing the robustness of the model, it ensures that it still maintains high recognition accuracy under adversarial samples, thereby improving the reliability and security of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 Schematic diagram of the method steps;

[0072] Figure 2 Schematic diagram of the traffic sign classification network model; DETAILED DESCRIPTION

[0073] The present invention will be further described below in conjunction with the accompanying drawings.

[0074] The schematic diagram of the steps of the traffic sign recognition model robustness enhancement method based on adversarial attack and sensitive layer visualization is as follows Figure 1 As shown, the overall steps include:

[0075] (1) Construct an adversarial attack module, such as Figure 1 As shown in the adversarial attack module, adversarial samples are first generated through common adversarial attack methods (such as FGSM, PGD, etc.), and small perturbations are applied to simulate attack scenarios. These adversarial samples are input into the traffic sign recognition model to detect the behavior of the model under attack. This module is used to simulate actual adversarial attack scenarios and evaluate the robustness of the model;

[0076] (2) Construct the sensitive layer visualization module, such as Figure 1As shown in the sensitive layer visualization module, the convolutional layers of the model are visualized based on the class activation mapping (CAM) technology. By analyzing the activation maps of each layer of the model, the convolutional layers that are most sensitive to adversarial samples are found. This module is used to identify which convolutional layers are most susceptible to adversarial perturbations, thereby determining the key layers that need further optimization;

[0077] (3) Construct an adversarial training module, such as Figure 1 As shown in the adversarial training module in , adversarial training is performed to conduct targeted adversarial training on sensitive layers. By optimizing the objective function, the robustness of the model under adversarial perturbations is enhanced, and the layers with the greatest impact of adversarial perturbations are trained in a focused manner, so that the model can make more robust decisions when facing adversarial samples;

[0078] (4) Constructing a confrontational purification module: Figure 1 As shown in the adversarial purification module in Figure 2, the input samples are preprocessed during the test phase, and possible adversarial perturbations are removed through adversarial purification techniques. According to the defense strategies learned during the training process, appropriate purification operations are dynamically selected to ensure that the model can maintain high robustness and recognition accuracy under various attack methods during the test phase.

[0079] Our custom traffic sign classification model is as follows Figure 2 As shown, the network consists of four convolutional layers, two pooling layers, two Dropout layers, a flattening layer, a fully connected layer, and an output layer.

[0080] In the adversarial attack module stage, adversarial samples are first generated through common adversarial attack methods (such as FGSM, PGD, etc.):

[0081] x′=x+δ

[0082] Among them, δ is a perturbation generated based on the gradient, satisfying:

[0083]

[0084] where ∈ is the disturbance amplitude, is the prediction loss function, Represents the gradient of the loss function with respect to the input sample x.

[0085] The purpose of this step is to simulate actual adversarial attack scenarios, detect the behavior of the model under attack, and evaluate the robustness of the model.

[0086] Furthermore, each convolutional layer of the model is visualized using the Class Activation Mapping (CAM) technique:

[0087]

[0088] Among them, A k (x) is the activation map of the input sample x at the kth convolution kernel, w k are the class-related weights in the fully connected layer.

[0089] By analyzing the activation maps of each layer of the model, we found that the convolution layer is the most sensitive layer of the model when processing adversarial samples, and identified this layer as the key layer that needs further optimization.

[0090] In the adversarial training module stage, the convolution kernel of the sensitive layer is dynamically adjusted according to the key layer determined in the sensitive layer visualization module. Optimize the parameters of the sensitive layer, η is the learning rate, so that the layer can better adapt to adversarial perturbations.

[0091] Finally, before testing the adversarially trained traffic sign classification model, an adversarial purification operation is performed on the perturbed input sample, using the loss function of the input sample Determine whether the sample is affected by the disturbance, map the sample affected by the disturbance into the feature space, remove the disturbance part, and retain the useful features to achieve the effect of adversarial purification. The formula of adversarial purification can be expressed as:

[0092] x clean =P(f(x;θ))

[0093] Through the above steps, we found the convolutional layer that is most sensitive to adversarial attacks, and improved the model's ability to cope with attacks by focusing on adversarial training of this layer. In the test phase, adversarial purification is performed on disturbed samples to reduce the interference of adversarial samples on the model, further improving the robustness of the model when encountering new or unseen attacks, forming a more comprehensive defense mechanism.

[0094] The series of detailed descriptions listed above are only specific descriptions of feasible implementation methods of the present invention. They are not intended to limit the scope of protection of the present invention. All equivalent implementation methods or changes that do not deviate from the technology of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for enhancing the robustness of traffic sign recognition models based on adversarial attacks and visualization of sensitive layers, characterized in that: The main steps include: Step 1.1: Construct a sensitive layer visualization module, use class activation mapping technology to visualize each convolutional layer of the model, and generate activation maps for each layer of the model. By analyzing the activation maps of each layer of the model, the convolutional layer that is most sensitive to adversarial samples is found. This module is used to identify which convolutional layers are most susceptible to adversarial perturbations, thereby determining the key layers that need further optimization; Step 1.2: Construct an adversarial training module, perform adversarial training on the model, and conduct targeted adversarial training on the sensitive layers found in step 1.1 to enhance the intensity of sensitive layer training. By optimizing the objective function, the robustness of the model under adversarial perturbations is enhanced, and the layers with the greatest impact of adversarial perturbations are trained in a focused manner, so that the model can make more robust decisions when facing adversarial samples; Step 1.3: Construct an adversarial purification module to preprocess the input samples during the test phase and remove possible adversarial perturbations through adversarial purification techniques. According to the defense strategies learned during the training process, select appropriate purification operations to ensure that the model can maintain high robustness and recognition accuracy under various attack methods during the test phase.

2. The method for enhancing the robustness of a traffic sign recognition model based on adversarial attacks and sensitive layer visualization according to claim 1 is characterized in that: The method of the sensitive layer visualization module in step 1.1 is as follows: Step 2.1: Use the class activation mapping technique to calculate the class activation map based on the activation map of a specific convolutional layer and the corresponding weights. First, select the convolutional layer and obtain the activation feature map of the layer: A=f(x;θ conv ) Where f is the output of the convolutional layer, x is the input image, and θ conv are the parameters of the convolutional layer. Then, get the fully connected layer weights: w k =Weights of class k in the fully connected layer Finally, for each class k, according to the activation feature map L and weight w k Compute class activation map CAM k : Among them, A i is the i-th channel in the feature map, is the i-th weight associated with class k; Step 2.2: Compute the sensitivity of the convolutional layer by comparing the activation maps of the original image and the adversarial example: in, is the i-th activation feature map of the l-th layer, is the corresponding adversarial sample activation feature map, N is the number of feature maps; Step 2.3: Normalize the generated class activation maps to visualize them. Show the activation strength in the form of heat maps to highlight the most sensitive areas in the adversarial examples. Based on the heat map analysis results, identify the convolutional layers that show high sensitivity under adversarial perturbations. These layers will be considered key layers that need further optimization and adversarial training.

3. The method for enhancing the robustness of a traffic sign recognition model based on adversarial attacks and sensitive layer visualization according to claim 1, characterized in that: The method of the adversarial training module in step 1.2 is as follows: Step 3.1: Use the method network in reinforcement learning adversarial training to generate an attack method, and use this method to generate adversarial samples. Input the generated adversarial samples and original samples into the model for adversarial training, and update the model parameters by gradient descent. Step 3.2: Extract the activation feature maps of these convolutional layers based on the sensitive layer information found in step 1.1; Step 3.3: Define an optimization objective function for each sensitive layer, which can calculate the performance of the model under adversarial examples based on the cross entropy loss, expressed as: Among them, L(θ) is the target loss function, N is the number of samples, and y i is the true label, f(x i ′; θ) is the output of the model; Step 3.4: During the training process, the convolution kernel of the sensitive layer is dynamically adjusted to adapt to the characteristics of the adversarial sample. The parameters of the sensitive layer are optimized by the gradient descent method to make the model better adapt to the adversarial perturbation. The update rule is: Among them, η is the learning rate and θ′ is the updated parameter.

4. The method for enhancing the robustness of a traffic sign recognition model based on adversarial attacks and sensitive layer visualization according to claim 1, characterized in that: The method of the antagonistic purification module in step 1.3 is as follows: Step 4.1: In the testing phase, receive the input sample x, evaluate the characteristics of the input sample based on the defense strategy learned by the model in the training phase, and determine whether it is affected by the adversarial perturbation by calculating the loss function L of the input sample: Where N is the number of samples, y i is the true label, f(xi;θ) is the output of the model; Step 4.2: Perform feature space projection on the adversarially perturbed input sample to purify the input sample: x clean =P(f(x;θ)) Where P represents the feature space projection operation; Step 4.3: Re-input the cleaned sample into the model for testing.

5. The method for enhancing the robustness of a traffic sign recognition model based on adversarial attacks and sensitive layer visualization according to claim 1, characterized in that: The overall loss function L is: Among them, the loss function is divided into four items. The first item is the prediction loss, N is the number of samples, y i is the true label, f(x i ; θ) is the output of the model, θ is the model parameter. The second term is the adversarial loss, x adv,i is the generated adversarial sample, ∈ is the tolerated error threshold. The third term is the sensitivity loss, A j is the activation map of the jth layer, A j,clean is the activation map of the corresponding unperturbed sample. The fourth term is the regularization term, W j is the weight matrix of the jth layer. λ1, λ2, λ3 are hyperparameters.

Images