Deep learning model hidden attack method and system
Optimizing the color and transparency of RGB wafers through Grad-CAM and evolutionary strategies, solving the problem of insufficient concealment and targeting of adversarial samples, achieving efficient and hidden adversarial samples generation in deep learning models, and is suitable for security-sensitive fields. .
Patent Information
- Application Number
- CN202410769620.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-14
- Publication Date
- 2024-11-22
AI Technical Summary
In the prior art, the poor concealment of the adversarial sample and the lack of targeted regional attack strategies and insufficient adjustment of element transparency have led to poor attack effects, especially in safety-sensitive fields such as medical image analysis and autonomous driving systems, which are difficult to achieve high concealment and Effectively misleading.
Grad-CAM technology combined with evolutionary strategies to accurately locate key areas of deep learning models, and generate highly concealed adversarial samples through the color and transparency optimization of RGB wafers. Grad-CAM generation model focuses on the focus Thermogram, identify key areas for patch deployment, initialize RGB disks and tune their color and transparency through evolutionary strategies to achieve efficient generation of adversarial samples.
It significantly improves the concealment and targeting of the adversarial samples, can be visually almost impossible to detect, and is difficult to be detected by automated defense systems, and improves the success rate and efficiency of attacks, and is especially suitable for safe and sensitive application scenarios.
Smart Images

Figure CN119006949A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of computer vision and machine learning, and in particular to a covert attack method and system for a deep learning model. Background Art
[0002] With the rapid development of deep learning technology, deep neural networks have been widely used in many fields such as image recognition, natural language processing, and autonomous driving. However, these models face security challenges, especially adversarial sample attacks, which are attacks that make models make wrong decisions by making subtle modifications to input data. These attacks have become an important means of testing the robustness of models.
[0003] Fast Gradient Sign Method (FGSM):
[0004] Source: "Goodfellow, Ian J., Jonathon Shlens, and Christian Szegedy. "Explaining and harnessing adversarial examples." arXiv preprint arXiv:1412.6572(2014).
[0005] Main content: Generate adversarial samples by using the gradient information of input data. Specifically, by simple operations on the gradient sign, the original image is updated once to generate adversarial samples. For example, the existing invention patent application document "A method, system, device and storage medium for generating and testing adversarial samples" with publication number CN117332843A, the existing method includes: obtaining initial sample data; calculating the initial sample data according to the RS-FGSM algorithm to obtain adversarial sample data; inputting the adversarial sample data into the neural network model to obtain test results; this method calculates the adversarial sample data obtained by the RS-FGSM algorithm on the initial sample data, and tests the generated adversarial sample data.
[0006] Problems and disadvantages of the aforementioned existing solutions: Although the operation is simple and fast, the adversarial samples generated by FGSM often lack concealment and are easily detected by defense mechanisms.
[0007] Projected Gradient Descent (PGD):
[0008] Source: "Madry, Aleksander, et al." Towards deep learning models resistant to adversarial attacks." arXiv preprint arXiv:1706.06083 (2017). Main content: The optimal adversarial perturbation is found in the constrained region through an iterative optimization process. This method is considered to be an effective means of generating strong adversarial samples. In the prior art, projected gradient descent PGD is used to generate adversarial samples.
[0009] Problems and disadvantages of the aforementioned existing solutions: The PGD process is computationally intensive and also faces the problem of insufficient concealment of generated samples.
[0010] The aforementioned traditional solutions have the following problems and disadvantages:
[0011] Existing technologies mainly focus on improving the success rate of attacks, but lack research on the concealment of adversarial samples. In security-sensitive fields, such as medical image analysis and autonomous driving systems, highly concealed adversarial samples are particularly important because these fields have extremely low tolerance for wrong decisions. Adversarial samples generated by existing methods are often visually easy to identify, which greatly reduces the effectiveness of attacks. In addition, existing technologies rarely consider the precise positioning of key areas and the detailed adjustment of specific area features during the generation of adversarial samples, so as to maintain the concealment of samples while improving the attack effect.
[0012] Although deep learning models have achieved remarkable application results in many fields, they are vulnerable to carefully designed input perturbations, namely adversarial examples. Currently, adversarial example generation methods mainly face the following shortcomings:
[0013] 1. Lack of concealment: Although existing adversarial sample generation methods (such as FGSM and PGD) can effectively mislead deep learning models, the generated adversarial samples are often visually easy to be recognized by human observers or detected by automated defense systems. This is especially important in security-sensitive fields such as autonomous driving and medical image analysis, because the concealment of adversarial samples is directly related to the practicality of the attack.
[0014] 2. Lack of targeted regional attack strategies: Most existing technologies fail to effectively utilize the internal structural information of the model, such as the key areas that the model relies on when making predictions. Failure to accurately locate and target these key areas will reduce the efficiency and success rate of adversarial attacks.
[0015] 3. Insufficient transparency adjustment: When generating adversarial samples, the adjustment of element transparency is often not fully considered. Transparency is a key parameter for controlling the concealment of adversarial samples. By fine-tuning transparency, the concealment of samples can be greatly improved while maintaining the effectiveness of the attack.
[0016] The existing invention patent application document "A method for generating adversarial samples based on low-perceptual perturbations in key areas" with publication number CN116258867A, the existing method includes: obtaining an original image containing an adversarial target; inputting the original image into a preset adversarial sample generation network to obtain an adversarial sample; the adversarial sample generation network includes a perception module and a generator; inputting the original image into the perception module to obtain a perception area mask; inputting the original image into the generator to generate an initial perturbation; filtering the initial perturbation through a smoothing filter module and multiplying it with the perception area mask to obtain a final perturbation; superimposing the final perturbation with the original image to generate an initial adversarial sample; calculating the network loss L between the initial adversarial sample and the original image; iteratively optimizing the adversarial sample generation network according to the network loss L; using the optimized adversarial sample generation network to convert the original image into a final adversarial sample; the generated adversarial sample is closer to the real image. From the specific implementation content of the prior art, it can be seen that in the prior scheme, the original image is extracted to obtain a multi-scale feature map A, the feature map A is input into the RPN network for processing, the candidate region is obtained after non-maximum suppression processing, the gradient weighted class activation mapping is performed on the candidate region to obtain the weight value of the candidate region, the weight values of all candidate regions are processed by the ReLU function and accumulated to obtain the feature m, and then the feature m is normalized.
[0017] 1. Generation of heatmap and perception area mask:
[0018] In the above-mentioned prior art, a multi-scale feature map and an RPN network are used to determine the perception area, and the weights of the candidate areas are calculated by gradient weighted class activation mapping, and the perception area mask is obtained by accumulating the weights after processing with the ReLU function. This method is based on the existing target detection framework, and the focus is on identifying key areas through the network structure itself. This existing solution only provides an in-depth visual explanation of the model decision-making process, and the targeting and concealment of the attack still need to be strengthened.
[0019] 2. Patch generation process:
[0020] In the above-mentioned prior art, the original image is input into the perception module to obtain the perception area mask, and then the original image is input into the generator to generate the initial perturbation, and then the initial perturbation is filtered by the smoothing filter module and multiplied with the perception area mask to obtain the final perturbation, and finally the final perturbation is superimposed with the original image to generate the initial adversarial sample. In the above-mentioned prior art, the simple operation of exchanging individual pixels and then compounding them with the perturbation generated by the perception module lacks specificity.
[0021] 3. Adjustment and optimization strategies for adversarial samples:
[0022] In the above-mentioned prior art, it mainly relies on generating initial disturbances, reducing noise through smoothing filtering, and combining perception area masks to form the final disturbance. Although this method takes into account reducing noise and controlling the disturbance area, it provides limited control over the fine-tuning of the disturbance and transparency adjustment.
[0023] 4. Iterative optimization process:
[0024] In the above-mentioned prior art, the generative network is iteratively optimized based on network losses, including perceptual loss, GAN loss, etc. This optimization method relies on a clearly defined loss function and may require multiple iterations to achieve the desired misleading effect.
[0025] And the existing public document "Study on the Interpretability of Image Classification with Deep Convolutional Neural Networks Based on Gradient Localization", in which an element-by-element class activation mapping method is proposed based on the idea of CAM. This method uses element-by-element gradient information and activation information to obtain a class activation map, and explains the relationship between the class discrimination area and the category output. The paper derives that ECAM is a generalization of the original CAM method, and can be applied to deep learning models based on convolutional networks without being restricted by the network structure in the CAM method. A gradient-weighted class activation mapping (Grad-CAM) method is proposed. This method calculates the mean of the gradient of the class output with respect to each feature map in the last convolutional layer of the network as the weight of the corresponding feature map, and then obtains the class discrimination area through weighted combination.
[0026] 1. Application improvements of Gradient Weighted Class Activation Mapping (Grad-CAM):
[0027] The ECAM used in the above-mentioned prior art generates a class activation map based on element-level gradient and activation information. Although this method has high granularity, it may lead to overly dispersed focus when facing complex images or multi-target scenes, thereby reducing the ability to concentrate on representing key features.
[0028] 2. Innovative application of evolutionary strategies:
[0029] In the aforementioned prior art: no evolutionary algorithm is used to optimize specific attributes (such as color and transparency) in adversarial sample generation.
[0030] 3. Improve the concealment of adversarial samples:
[0031] In the aforementioned prior art, although ECAM improves positioning accuracy, the visual explanation it generates may not be sufficient for generating highly concealed adversarial samples.
[0032] And the existing public document "An Adversarial Sample Generation Method Based on Improved Genetic Algorithm", in which Grad-CAM uses the mean of the channel gradient obtained in back propagation as the channel weight, introduces the Grad-CAM interpretation method into population evolution, groups pixels according to the thermal value of the image, and controls the evolution of the perturbation in the key areas of the image to reduce the perturbations added in the image and improve the concealment of adversarial samples. The Grad-CAM interpretation method is introduced into population initialization, random perturbations are added to the key areas of the image to generate the initial solution, and the perturbation evolution area is limited to the key areas of the image to improve the concealment of the sample and accelerate the convergence of the algorithm.
[0033] 1. Improve the pertinence and accuracy of misleading effects:
[0034] Although the genetic algorithm in the aforementioned existing solution "Adversarial Sample Generation Method Based on Improved Genetic Algorithm" perturbs important areas of the image, the precise control of this perturbation is still not specific enough, especially in scenarios where it is necessary to accurately mislead the deep learning model.
[0035] 2. Optimize the efficiency and iteration process of the algorithm:
[0036] The aforementioned existing solution "Adversarial Sample Generation Method Based on Improved Genetic Algorithm" uses adaptive probability functions and elite retention strategies to improve the efficiency of genetic algorithms, but these improvements may still require multiple model evaluations, which may lead to inefficiency in practical applications.
[0037] As well as the existing public literature "Research on Adversarial Sample Generation and Defense Methods for Image Classification", in which the CNN interpretability method Grad-CAM is introduced, and a two-stage single-pixel adversarial attack is proposed, and experimental verification is carried out on the attack success rate, concealment and transferability; a two-stage one-pixel adversarial attack algorithm based on Grad-CAM (Two-Stage One Pixel Adversarial Attack based on Grad-CAM, TOPAA) is proposed, and a class activation map is introduced.
[0038] 1. More flexible attacks:
[0039] Prior art: Single-pixel adversarial sample generation based on Grad-CAM. The innovation of this method is that only a single pixel is modified. Single-pixel adversarial attack is a local perturbation adversarial attack method under extreme conditions. It implements adversarial attack by only modifying a single pixel of the image, which improves concealment. However, in some complex scenarios, single-pixel modification may not be enough to cause model prediction errors.
[0040] 2. Optimized iteration efficiency:
[0041] Existing technology: Adversarial sample generation based on Gaussian kernel and momentum iterative gradient, generating a key Gaussian circle for each foreground target in the sample, using momentum iterative gradient method to generate adversarial samples, combining Gaussian circle and adversarial sample to generate adversarial patches and replace them on the original image. This method enhances the targeting of the attack through targeted perturbations in specific areas, but may require repeated iterative processes to find effective attack points, which is inefficient.
[0042] It reduces computing resource consumption and model access times, and improves the efficiency of adversarial sample generation.
[0043] In summary, the existing technologies have technical problems such as poor concealment of adversarial samples, lack of targeted regional attack strategies, and insufficient adjustment of element transparency. Summary of the invention
[0044] The technical problem to be solved by the present invention is: how to solve the technical problems in the prior art of poor concealment of adversarial samples, lack of targeted regional attack strategies and insufficient adjustment of element transparency.
[0045] The present invention adopts the following technical solutions to solve the above technical problems: The deep learning model covert attack method includes:
[0046] S1. Analyze the target deep learning model using the Gradient Weighted Class Activation Mapping (Grad-CAM) method to generate a model focus heat map to identify the image region that contributes most to the prediction decision of the target deep learning model.
[0047] S2. Determine the key patch deployment area and patch center based on the model focus heat map, image size and resolution, and the image area with the largest contribution, and perform patch deployment operations accordingly;
[0048] S3, in each patch deployment key area, initialize the RGB disk, where the RGB disk is located at the center of the pixel point in the applicable patch, randomly assign an initial color value to each RGB disk, and set the transparency range;
[0049] S4. By presetting the evolution strategy, the color and transparency of the RGB disc are tuned, the fitness function is defined, and the optimization is performed in combination with the misclassification confidence and the concealment of the adversarial sample; wherein the concealment of the adversarial sample is controlled according to the transparency;
[0050] S5, performing mutation operation, selection operation and recombination operation, and obtaining the optimal RGB wafer attribute combination through iterative optimization;
[0051] S6. Apply the optimal RGB disc attribute combination to the original image to generate adversarial samples and verify the misleading effect of adversarial samples on the target deep learning model and their concealment to human vision.
[0052] The present invention develops a new adversarial sample generation strategy, which can generate adversarial samples that are almost visually imperceptible and difficult to be detected by automated defense systems. The strategy is particularly suitable for security-sensitive application scenarios and improves the concealment of adversarial samples.
[0053] The present invention can accurately locate the focus of the deep learning model, so that the attack can be more concentrated on the key judgment area of the model, improving the pertinence and efficiency of adversarial sample generation. Compared with blindly or comprehensively modifying the image, the present invention can achieve the purpose of attack with less disturbance while maintaining the original appearance of the image. The present invention combines the Gradient Weighted Class Activation Mapping (Grad-CAM) technology with Evolution Strategies (ES) to achieve efficient and imperceptible misleading of the deep learning model.
[0054] In a more specific technical solution, in S1, in the gradient weighted class activation mapping Grad-CAM method, the last convolution layer of the target deep learning model is used to process the target category-related gradient information to obtain the gradient weight, and the gradient weight is used to perform a weighted operation on the feature map output by the convolution layer to generate a model focus heat map.
[0055] The present invention realizes an efficient and concealed adversarial sample generation process through the combination of the above steps. Specifically, the precise regional positioning of Grad-CAM, combined with targeted patch deployment and transparency control of RGB discs, not only improves the generation efficiency and success rate of adversarial samples, but also significantly enhances the concealment of adversarial samples. This method provides an effective tool for evaluating and enhancing the security of deep learning models, especially showing its unique advantages in situations with high requirements for concealment.
[0056] The present invention uses Grad-CAM to not only identify the key decision areas more accurately, but also fine-tune each parameter of the RGB disk, such as color depth and transparency, through evolutionary strategies to ensure better effects of perturbations in these areas. Such targeted operations can greatly improve the success rate of misleading while reducing the impact on the overall image quality.
[0057] In a more specific technical solution, S1 includes:
[0058] S11. Use the Gradient Weighted Class Activation Mapping (Grad-CAM) method to calculate the gradient of the target category score relative to the output of the last convolutional layer of the target deep learning model. Through the global average pooling operation, the gradient weight of each feature map is obtained.
[0059] S12, then use the gradient weight to generate the class activation map Based on the highlighted image, the image area with the greatest contribution to the target deep learning model is predicted for model decision-making;
[0060] S13. Use gradient weights Weighted feature map A k , apply the following ReLU function to generate the model focus heat map L c :
[0061]
[0062] Specifically, Grad-CAM uses gradient information to highlight the areas that have the greatest impact on the prediction results of the deep learning model. By weighting the output of the convolutional layer, a heat map is generated that reflects the contribution of each part to the model's decision.
[0063] The above method can accurately locate the most critical part of the image that affects the model output, making the subsequent perturbation more focused and effective. Compared with traditional full-image or random perturbation methods, Grad-CAM makes perturbations more purposeful and efficient, significantly improving the success rate of adversarial samples and reducing the computing resources required to generate adversarial samples.
[0064] The present invention utilizes the internal information of deep learning models, such as Gradient Weighted Class Activation Mapping (Grad-CAM) technology, to accurately locate the key areas that the model relies on when making predictions, and attacks these areas to improve the efficiency and success rate of the attacks.
[0065] The present invention adopts the Grad-CAM method to directly obtain the focus from the last convolutional layer of the target deep learning model and generate a heat map. This method not only provides an in-depth visual explanation of the model decision-making process, but also allows precise control of the deployment location of adversarial samples, improving the targeting and concealment of attacks.
[0066] In a more specific technical solution, in S11, the following logic is used to process and obtain the gradient weighted class activation map SGrad-CAM:
[0067]
[0068] In the formula, represents the class activation map for category c, A k represents the activation of the kth channel in the feature map, is the importance weight of feature map channel k for category c.
[0069] This technology: Using Grad-CAM, not only does it locate the key areas that affect the decision-making of deep learning models, but it also centrally deploys adversarial operations based on the characteristics of these areas. This approach is more focused and efficient in identifying the key factors of model decision-making, and is particularly suitable for generating highly targeted adversarial samples.
[0070] In a more specific technical solution, in S12, the score of the target category c is used to calculate the feature map A. k The gradient of , performs a global average pooling operation to obtain the gradient weight
[0071]
[0072] In the formula, y c is the model’s prediction score for category c, It is the feature map A k The activation value at position (i, j), Z is the feature map A k The total number of elements.
[0073] In a more specific technical solution, S2 includes:
[0074] S21. Design an area restriction patch, adjust the size and shape of the area restriction patch to cover the main focus points in the model focus heat map, minimize the interference of non-target areas, and obtain an applicable area restriction patch; wherein the length l of the applicable patch is determined using the following logic:
[0075]
[0076] S22. Within the patch deployment critical area, restrict patches according to the patch center deployment area.
[0077] The present invention limits the size and shape of patches during the deployment of regional restriction patches and accurately deploys them in key areas, which not only improves the effectiveness of the attack, but also significantly enhances the concealment of adversarial samples. This method solves the problem of adversarial samples being easily detected in the prior art and is particularly suitable for security-sensitive application scenarios.
[0078] In a more specific technical solution, S3 includes:
[0079] S31, according to the size of the applicable patch and the detail level information of the key area of patch deployment, adjust the size of the RGB disk. In this case, the evolutionary algorithm is used to calculate the attribute vector of the RGB disk according to the following logic: To optimize:
[0080]
[0081] In the formula, r represents the radius, (R, G, B) represents the color, and α represents the transparency;
[0082] S32, define and based on the attribute vector And the fitness function Evaluate the impact of each preset configuration information on the misclassification probability of the target deep learning model, using the following fitness function Perform disturbance evaluation, optimization and selection operations:
[0083]
[0084] In the formula, S(x orig ,x adv ) represents the original image x orig With adversarial sample x adv The similarity measure between them, λ is a trade-off parameter used to balance the importance between misclassification confidence and sample concealment. In this embodiment, the similarity measure uses the structural similarity index SSIM, and the concealment is the similarity of the original image;
[0085] S33. Initialize the radius r, color (R, G, B), and transparency α of the RGB circle.
[0086] The present invention directly deploys patches in key areas obtained by the Grad-CAM method, and through a preset evolutionary strategy, tunes the color and transparency of the RGB disc, defines a fitness function, and generates targeted and concealed adversarial patches.
[0087] The use of evolutionary strategies in the present invention not only improves the efficiency of perturbation search, but also achieves better results in fewer iterations through a carefully designed fitness function. In addition, by precisely controlling the deployment of perturbations, this method can reduce invalid queries to the model, thereby further improving the overall attack efficiency.
[0088] The present invention not only locates the key areas through Grad-CAM, but also optimizes the color and transparency of the RGB discs in these areas using evolutionary strategies. It is more flexible than the single-pixel attack used in traditional technologies, and can adjust the perturbation strategy according to the specific response of the target model, thereby improving the success rate of the attack. It improves the success rate of attacks in complex scenarios and solves the problem that single-pixel attacks may fail to deceive the model due to their small impact.
[0089] Compared with the traditional method based on Gaussian kernel and momentum iteration, the present invention optimizes the perturbation through evolution strategy, and can find more effective perturbation combinations in fewer iterations. This method directly affects the evolution process by evaluating the effects of different mutations, thereby improving the convergence speed and efficiency of the algorithm.
[0090] In a more specific technical solution, S33 includes:
[0091] S331, randomly determine the radius of the disc: r~U(r min ,r max ), where r min and r max Respectively represent the minimum and maximum values of the radius;
[0092] S332, randomly generate RGB color values;
[0093] S333, randomly set the transparency of the disc: α~U(0,1); in this embodiment, 0 represents completely transparent, and 1 represents completely opaque; wherein, the transparency of the disc is randomly set using the following logic:
[0094] α~U(0,1)
[0095]
[0096] In the formula, S(x orig ,x adv ) represents the similarity measure between the original image and the adversarial sample, and λ is a trade-off parameter that balances the importance of both.
[0097] This paper introduces the concept of RGB discs, and dynamically adjusts the color and transparency of these discs through evolutionary strategies, which can not only simulate and adjust the visual effects of adversarial samples more accurately, but also optimize them according to the model response. This method enhances the concealment of adversarial samples and improves the misleading effect.
[0098] The present invention's fine control of the transparency and color of the RGB disc allows the generation of adversarial samples that are almost indistinguishable from the original image in appearance, greatly improving stealth. The fine adjustment mechanism of transparency is one of the key innovations of the present invention, which makes the attack more difficult to detect visually while retaining the ability to mislead the model.
[0099] By adjusting the color and transparency of the RGB disc, the adversarial sample generated by the present invention is almost indistinguishable from the original image visually, greatly improving its concealment.
[0100] In a more specific technical solution, S5 includes:
[0101] S51, perform mutation operation to adjust the properties of the RGB disc to explore new solution space. The mutation operation is performed using the following logic:
[0102]
[0103] S52, performing a selection operation to retain the configuration of the RGB disk that maximizes the fitness function value, wherein the selection operation is performed using the following logic:
[0104]
[0105] S53, when the configuration of the RGB disk is applicable, a reorganization operation is performed, and features corresponding to the configurations of at least two RGB disks are combined to generate a new candidate solution, and an iteration termination condition is set, wherein the following reorganization operation is performed using the following logic:
[0106]
[0107] S54, when the iteration termination condition is met, stop the iteration operation and determine the optimal RGB wafer attribute combination, wherein the iteration termination condition is determined using the following logic:
[0108] Iteration number ≥ N max Or Δf≤ò.
[0110] Specifically, the evolutionary strategy optimizes the properties of the RGB disc by simulating natural selection processes, such as mutation, selection, and inheritance, to maximize the effect of misleading the model. The color and transparency of each generation of RGB discs are selected and mutated based on their contribution to the success rate of model misleading.
[0111] The aforementioned method allows for dynamic adjustment of the perturbation strategy to cope with the model's defense mechanism, improving the concealment of the perturbation and the success rate of the attack.
[0112] Defects resolved:
[0113] Compared with the method of single perturbation or a small amount of parameter adjustment, evolutionary strategies provide a systematic optimization process that can find the optimal or near-optimal perturbation combination in a shorter time, thereby effectively improving the efficiency and success rate of adversarial sample generation.
[0114] The present invention uses evolutionary strategies to finely adjust the color and transparency of the RGB disc on the area-restricted patch, aiming to find the optimal transparency setting, thereby maximizing the concealment of the adversarial sample while ensuring the effectiveness of the attack, and achieving fine control of the transparency of the element.
[0115] The present invention adopts evolutionary strategy for optimization, and performs iterative optimization through mutation, selection and recombination operations. This not only provides a wider range of exploration capabilities, but also can more flexibly adapt to various attack scenarios and characteristics of target models to find the optimal adversarial sample configuration.
[0116] The present invention uses evolutionary strategies to optimize the color and transparency of RGB discs, and continuously improves the concealment and misleading performance of adversarial samples through operations such as mutation, selection, and recombination. This method improves the effectiveness of adversarial samples in practical applications and their ability to be difficult to detect.
[0117] In a more specific technical solution, the deep learning model covert attack system includes:
[0118] The target model analysis module is used to analyze the target deep learning model using the Gradient Weighted Class Activation Mapping (Grad-CAM) method and generate a model focus heat map to identify the image area that contributes most to the prediction decision of the target deep learning model.
[0119] A patch area determination module is used to determine the key patch deployment area and patch center according to the model focus heat map, image size and resolution, and the image area with the largest contribution, and perform patch deployment operations accordingly. The patch area determination module is connected to the target model analysis module;
[0120] An RGB disc initialization module is used to initialize the RGB disc in each patch deployment key area, wherein the RGB disc is located at the center of the pixel point in the applicable patch, and an initial color value is randomly assigned to each RGB disc, and a transparency range is set. The RGB disc initialization module is connected to the patch area determination module;
[0121] The adversarial sample concealment control module is used to adjust the color and transparency of the RGB disc through a preset evolutionary strategy, define a fitness function, and optimize the misclassification confidence and adversarial sample concealment; wherein, the adversarial sample concealment is controlled according to the transparency, and the adversarial sample concealment control module is connected to the RGB disc initialization module;
[0122] Iterative optimization module, used to perform mutation, selection and recombination operations, obtain the optimal RGB wafer attribute combination through iterative optimization, and the iterative optimization module is connected with the adversarial sample concealment control module;
[0123] The adversarial sample generation and verification module is used to apply the optimal RGB disc attribute combination to the original image to generate adversarial samples and verify the misleading effect of the adversarial samples on the target deep learning model and their concealment to human vision. The adversarial sample generation and verification module is connected to the iterative optimization module.
[0124] Compared with the prior art, the present invention has the following advantages:
[0125] The present invention develops a new adversarial sample generation strategy, which can generate adversarial samples that are almost visually imperceptible and difficult to be detected by automated defense systems. The strategy is particularly suitable for security-sensitive application scenarios and improves the concealment of adversarial samples.
[0126] The present invention can accurately locate the focus of the deep learning model, so that the attack can be more concentrated on the key judgment area of the model, improving the pertinence and efficiency of adversarial sample generation. Compared with blindly or comprehensively modifying the image, the present invention can achieve the purpose of attack with less disturbance while maintaining the original appearance of the image. The present invention combines the Gradient Weighted Class Activation Mapping (Grad-CAM) technology with Evolution Strategies (ES) to achieve efficient and imperceptible misleading of the deep learning model.
[0127] The present invention realizes an efficient and concealed adversarial sample generation process through the combination of the above steps. Specifically, the precise regional positioning of Grad-CAM, combined with targeted patch deployment and transparency control of RGB discs, not only improves the generation efficiency and success rate of adversarial samples, but also significantly enhances the concealment of adversarial samples. This method provides an effective tool for evaluating and enhancing the security of deep learning models, especially showing its unique advantages in situations with high requirements for concealment.
[0128] The present invention utilizes the internal information of deep learning models, such as Gradient Weighted Class Activation Mapping (Grad-CAM) technology, to accurately locate the key areas that the model relies on when making predictions, and attacks these areas to improve the efficiency and success rate of the attacks.
[0129] The present invention limits the size and shape of patches during the deployment of regional restriction patches and accurately deploys them in key areas, which not only improves the effectiveness of the attack, but also significantly enhances the concealment of adversarial samples. This method solves the problem of adversarial samples being easily detected in the prior art and is particularly suitable for security-sensitive application scenarios.
[0130] The present invention's fine control of the transparency and color of the RGB disc allows the generation of adversarial samples that are almost indistinguishable from the original image in appearance, greatly improving stealth. The fine adjustment mechanism of transparency is one of the key innovations of the present invention, which makes the attack more difficult to detect visually while retaining the ability to mislead the model.
[0131] The present invention uses evolutionary strategies to finely adjust the color and transparency of the RGB disc on the area-restricted patch, aiming to find the optimal transparency setting, thereby maximizing the concealment of the adversarial sample while ensuring the effectiveness of the attack, and achieving fine control of the transparency of the element.
[0132] The present invention solves the technical problems existing in the prior art, namely, poor concealment of adversarial samples, lack of targeted regional attack strategies, and insufficient adjustment of element transparency. BRIEF DESCRIPTION OF THE DRAWINGS
[0133] Figure 1 This is a schematic diagram of the basic steps of the deep learning model covert attack method of Example 1 of the present invention;
[0134] Figure 2 This is a schematic diagram of the model structure of the deep learning model covert attack method according to Example 1 of the present invention;
[0135] Figure 3 This is a schematic diagram of specific implementation steps of Example 2 of the present invention applied to a medical image analysis application scenario. DETAILED DESCRIPTION
[0136] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0137] Example 1
[0138] like Figure 1 and Figure 2 As shown in the figure, the deep learning model covert attack method includes the following basic steps:
[0139] S1, target model analysis;
[0140] In this embodiment, Gradient Weighted Class Activation Mapping (Grad-CAM) is applied to the target deep learning model to generate a heat map to identify the image regions that contribute most to the model's prediction decision.
[0141] In this embodiment, Grad-CAM uses the output of the last convolutional layer of the target model to calculate the gradient information related to the target category, and uses these gradient weights to weight the feature map output by the convolutional layer to generate a heat map representing the focus of the model.
[0142] In this embodiment, in order to achieve a visual explanation of the decision-making process of the deep learning model, the present invention adopts the Gradient Weighted Class Activation Mapping (Grad-CAM) method. Specifically, the present invention calculates the gradient of the target category score relative to the output of the last convolutional layer of the model, and obtains the weight of each feature map through a global average pooling operation.
[0143] These weights are then used to generate class activation maps The plot provides a visual explanation of the model’s decisions by highlighting the regions of the image that contribute most to the model’s predictions.
[0144] In this embodiment, Grad-CAM is calculated by the following formula:
[0145]
[0146] in, represents the class activation map for category c, A k represents the activation of the kth channel in the feature map, is the importance weight of feature map channel k for category c, which is calculated by the score of target category c for A k The gradient of is globally averaged and pooled to obtain the following formula:
[0147]
[0148] Among them, y c is the model’s prediction score for category c, It is the feature map A k The activation value at position (i, j), Z is the feature map A k The total number of elements.
[0149] S2, patch area determination;
[0150] In this embodiment, according to the results of the heat map, the key areas where patches need to be deployed are determined. The area-restricted patches are designed, and their size and shape are adjusted to cover the main points of interest in the heat map while minimizing interference with non-target areas.
[0151] In this embodiment, within the critical area identified by S1, a cross-shaped patch with restricted size and shape is deployed in the identified critical area.
[0152] In this embodiment, the size, shape and position of the cross patch in the key area are selected based on the dense area of the heat map to determine the center of the patch, and the size and shape of the patch are adjusted to cover the main focus points in the heat map, while taking into account the size and resolution of the image. In this embodiment, the cross patch includes but is not limited to: two elongated line segments, and the two line segments intersect vertically at their midpoints. The line segment length is randomly selected from half the width of the picture to the width of the picture, and the line segment width is randomly selected between 1, 2, 3, and 4 pixels, and the most appropriate position and size are selected through iteration.
[0153] S3, RGB disc initialization;
[0154] In this embodiment, an RGB disc is initialized in each patch area, and each disc is located at the center of a pixel in the patch. An initial color value is randomly assigned to each disc, and a controllable transparency range is set.
[0155] In this embodiment, the initialization attributes of the RGB disc select the color and transparency of the randomly generated RGB disc to ensure the diversity of the initial search space, and the size of the disc is adjusted according to the patch size and the detail level of the target area. To optimize:
[0156]
[0157] Where r represents the radius, (R, G, B) represents the color, and α represents the transparency. In this embodiment, by defining the attribute vector of the disk And the fitness function Evaluate the impact of each configuration on the misclassification probability of the target model. First, initialize:
[0158] Radius r: Randomly determine the radius of the disc r~U(r min ,r max ), where r min and r max Respectively represent the minimum and maximum values of the radius;
[0159] Color (R,G,B): Randomly generate RGB color values. R,G,B~U(0,255) Here it is assumed that the color values are evenly distributed between 0 and 255;
[0160] Transparency α: Randomly set the transparency of the disc, usually between 0 and 1. α~U(0,1); in this embodiment, 0 means completely transparent, and 1 means completely opaque;
[0161] In this embodiment, the evaluation, optimization and selection of disturbances using the fitness function requires comprehensive consideration of multiple factors, such as misclassification confidence and sample concealment. The fitness function is as follows:
[0162]
[0163] Among them, S(x orig ,x adv ) represents the original image x orig With adversarial sample x adv The similarity measure between them is λ, which is a trade-off parameter used to balance the importance between the misclassification confidence and the sample concealment. In this embodiment, the similarity measure uses the structural similarity index SSIM, and the concealment is the similarity of the original image.
[0164] In this embodiment, a mutation operation is used:
[0165]
[0166] Explore new solution spaces by performing the following selection operations:
[0167]
[0168] Keep the optimal configuration. If necessary, perform the following reorganization operations:
[0169]
[0170] Used to generate new candidate solutions. The whole process continues to iterate until the iteration termination condition is met:
[0171] Iteration number ≥ N max or Δf≤ò
[0172] Thus, the optimal wafer configuration is determined.
[0173] S4, transparency and color adjustment;
[0174] In this embodiment, the color and transparency of the disc are optimized by evolutionary strategy. A fitness function is defined, and the optimization is performed by combining the misclassification confidence and the concealment of the adversarial sample; in this embodiment, the concealment of the adversarial sample is controlled by transparency.
[0175] In this embodiment, an RGB disc with an initial random color and precisely controlled transparency is initialized at the center of each pixel of the cross-shaped patch, so that the adversarial sample visually maintains the coherence of the original image while functionally achieving effective misleading of the target model.
[0176] The present invention uses Gradient Weighted Class Activation Mapping (Grad-CAM) and Evolution Strategies (ES). First, Grad-CAM generates a heat map by analyzing the output of the last convolutional layer of the target deep learning model and calculating the gradient information related to the target class. The heat map highlights the key areas that the model relies on when making predictions. Next, using this information, we deploy patches of specific shapes and sizes in these key areas and initialize RGB circles on the patches.
[0177] The color and transparency of the RGB disc are optimized using evolutionary strategies. Evolutionary strategies are a population-based optimization algorithm that iteratively improves solutions by simulating the mutation, recombination, and selection mechanisms in the natural selection process. In the present invention, evolutionary strategies are used to find RGB disc attribute configurations that can maximize the model's misclassification probability while keeping the adversarial sample highly similar to the original image.
[0178] In this embodiment, the evolution strategy defines a fitness function to evaluate the impact of the current wafer configuration on the misclassification probability of the target model; implements a mutation operation to slightly adjust the wafer attributes to explore a new solution space, implements a selection operation to retain those wafer configurations that maximize the fitness function value; if applicable, implements a recombination operation to combine the features of no less than 2 configurations to generate a new candidate solution, and sets an iteration termination condition; such as reaching a predetermined number of iterations or the fitness improvement is no longer significant.
[0179] S5, iterative optimization;
[0180] In this embodiment, the operations of executing the evolution strategy include but are not limited to: selection, mutation and recombination. Iteration continues until the iteration termination condition is met, such as the fitness function value reaches a preset threshold or the optimization process is stable.
[0181] Repeat the above step S4 until the optimal RGB disc attribute combination is found by passing the defined fitness function, which comprehensively considers the misclassification rate and the concealment of the adversarial sample.
[0182] The fitness function specifically considers the improvement in the misclassification rate and the similarity between the adversarial sample and the original sample to ensure that the generated adversarial samples are both effective and imperceptible.
[0183] S6, optimal configuration confirmation;
[0184] In this embodiment, the RGB disk configuration that can maximize the probability of misclassification of the target model is determined, including the final color and transparency settings, to ensure that the adversarial sample under the optimal configuration is highly consistent with the original image visually, thereby achieving a covert attack.
[0185] In this embodiment, based on the basic implementation, the advanced implementation introduces more complex fitness functions and optimization strategies, such as introducing a genetic algorithm to perform global optimization of RGB wafer properties to further improve the concealment and attack efficiency of adversarial samples.
[0186] In this embodiment, the iteration process ends, and the final adversarial sample configuration is confirmed when the value of the fitness function reaches a preset threshold, or the improvement of the fitness function in multiple consecutive iterations is lower than a predetermined minimum improvement value.
[0187] In this embodiment, the iteration termination condition includes but is not limited to: the fitness function value reaches a preset threshold, or the improvement of the fitness function in multiple consecutive iterations is lower than a minimum improvement value.
[0188] S7. Adversarial sample generation.
[0189] In this embodiment, the optimal RGB disc configuration is applied to the original image to generate an adversarial sample. The generated adversarial sample is tested to verify its misleading effect on the target deep learning model and its concealment from human vision. The comprehensive application of the present invention is not limited to general image recognition tasks, and its principles and methods can also be adjusted according to specific application requirements, such as medical image analysis, autonomous driving systems and other fields, showing extremely high adaptability and wide application potential. Compared with the prior art, the present invention greatly improves the concealment of adversarial samples while ensuring the success rate of the attack, providing a new perspective and method for the security analysis and enhancement of deep learning models.
[0190] Example 2
[0191] In this embodiment, the application scenario of medical image analysis is taken as an example for explanation. In the application of medical image analysis, the working principle of the present invention particularly emphasizes the fine adjustment of the transparency and color of the RGB disc on the basis of accurately locating the lesion area in the medical image (i.e., the key prediction area of the model). In this way, even in the medical field where image diagnostic information is critical, visually imperceptible adversarial samples can be generated without significantly changing the diagnostic information of the image. This process not only tests the robustness of the medical diagnosis model, but also ensures that the generation of adversarial samples will not cause direct interference to medical diagnosis.
[0192] The following implementation content demonstrates how the present invention utilizes the internal information of the deep learning model and the global optimization capability of the evolution strategy to generate adversarial samples that are visually imperceptible and can effectively mislead the target model. The working principle of the present invention is not only applicable to general image recognition tasks, but can also be adjusted and optimized according to different application fields, such as medical image analysis, showing a wide range of application potential and flexibility.
[0193] like Figure 3 As shown, in this embodiment, the deep learning model covert attack method also includes the following specific implementation steps:
[0194] S1', gradient weighted class activation mapping to obtain heat map;
[0195] In this embodiment, the convolutional layer feature map (Feature Maps) is obtained: the feature map A outputs y about the target category c c The gradient is:
[0196]
[0197] In the global average pooling (GAP) process of the gradient in this embodiment, the weight is obtained by global average pooling. It shows the contribution of different feature maps to the prediction of category c:
[0198]
[0199] Among them, Z is the feature map A k The total number of units in , i and j represent the row and column indices of the feature map, respectively.
[0200] In the process of generating the weighted feature map in this embodiment, use To weight the feature map A k , and apply the ReLU function to generate the final heat map L c :
[0201]
[0202] In the application process of the ReLU function in this embodiment, the definition of the ReLU function ensures that only positive activation values are retained and negative values are set to 0:
[0203] ReLU(x)=max(0,x) (4)
[0204] In the operation of obtaining the final thermal map in this embodiment, the thermal map L obtained through the previous steps is c It can indicate which regions contribute most to the model's prediction of category c.
[0205] In this embodiment, the heat map is superimposed on the input image. Specifically, the heat map L c Rescaled to match the input image size and overlaid on the original input image, this allows us to visualize the areas the model focuses on when making decisions.
[0206] S2′, cross patch selection;
[0207] In this embodiment, the patch center (x_c, y_c) can be obtained by analyzing the heat map L in formula (4). c to be sure.
[0208] In this embodiment, the heat map L c The highlighted image regions are where the model relies on when making classification decisions. The center of the dense region in the heat map is determined;
[0209] S3', adjust the size length l and width w of the cross-shaped patch to cover the main points of interest in the heat map, taking into account the size and resolution of the image;
[0210] In this example, the length l of the patch is randomly chosen between half the image width and the image width, while the width w is randomly chosen between 1 and 4 pixels:
[0211]
[0212] S4', select the most appropriate position (x_c, y_c) and size (l, w) in an iterative manner, perform mutation operations, selection operations, and recombination operations, and determine the optimal wafer configuration;
[0213] While maximizing the misclassification probability of the target model, the adversarial samples are kept visually invisible.
[0214] In this embodiment, the iterative process includes evaluating the fitness function for each candidate patch position and size. And choose the configuration that maximizes fitness.
[0215] In this embodiment, the attribute vector of the RGB disk is: Determined by initialization randomness, where:
[0216] Radius r: Randomly determine the radius of the disk, satisfying:
[0217] r~U(r min ,r max )(7)
[0218] Color (R,G,B): Randomly generate RGB color values, satisfying:
[0219] R,G,B~U(0,255)(8)
[0220] Transparency α: Randomly set the transparency of the disc to satisfy:
[0221] α~U(0,1)(9)
[0222]
[0223] Among them, S(x orig ,x adv ) represents the similarity measure between the original image and the adversarial sample, and λ is a trade-off parameter that balances the importance of both.
[0224] In this embodiment, the mutation operation: Used to explore new solution space;
[0225] Select an action: Keep the optimal configuration;
[0226] Reorganization operation: Generate new candidate solutions;
[0227] In this embodiment, the iteration process continues until the following iteration termination condition is met:
[0228] Iteration number ≥ N max or Δf≤ò
[0229] Determine the optimal wafer configuration to maximize the model's misclassification probability while maintaining the concealment of adversarial examples.
[0230] In summary: Application of Grad-CAM technology: Grad-CAM technology is used to accurately locate the key image areas that deep learning models rely on when making prediction decisions, which is a prerequisite for the implementation of this invention. Compared with the prior art, this invention does not blindly modify the entire image, but carries out targeted attacks on the "weaknesses" of the model, which improves the efficiency and success rate of adversarial sample generation.
[0231] Design and deployment of area-restricted patches: Based on the heat map results generated by Grad-CAM, cross-shaped patches with restricted size and shape are deployed in key areas. The design of this patch takes into account the concealment and attack effectiveness of adversarial samples, which is an innovation of this invention.
[0232] Transparency control of RGB discs: For RGB discs deployed on patches, the present invention fine-tunes their color and transparency, especially transparency control, through evolutionary strategies to achieve high concealment of adversarial samples. This transparency control mechanism is the key innovation of the present invention, which solves the problem of easy identification of adversarial samples in the prior art. The improved features of the present application are:
[0233] Adversarial sample generation method based on Grad-CAM and evolutionary strategies: This method combines the internal decision-making mechanism of the deep learning model with the optimization capability of evolutionary computing, and proposes a new adversarial sample generation framework.
[0234] Design of region-restricted patches and transparency control mechanism: The patch design not only takes into account the key areas of the model, but also significantly improves the concealment of adversarial samples by fine-tuning the transparency of the RGB disc on the patch. This design and control mechanism are significantly innovative in the field of adversarial sample generation.
[0235] Definition and optimization process of fitness function: By defining a fitness function that covers misclassification confidence and sample concealment, and using evolutionary strategies to optimize the RGB disc properties, the present invention achieves a balance between adversarial sample effectiveness and concealment, which is a completely new attempt in the prior art.
[0236] The core difference between the present invention and the prior art is:
[0237] The core difference of the present invention is the introduction of Grad-CAM-based key area positioning and RGB disc transparency control based on evolutionary strategies to improve the concealment and pertinence of adversarial samples. The existing technology usually fails to effectively utilize the internal information of the model for precise attacks, and also lacks consideration and optimization of the concealment of adversarial samples. The present invention significantly improves the practicality of adversarial samples by finely controlling the accuracy and concealment of attacks, especially in the application value of security-sensitive fields.
[0238] The present invention solves the technical problems existing in the prior art, namely, poor concealment of adversarial samples, lack of targeted regional attack strategies, and insufficient adjustment of element transparency.
[0239] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A covert attack method for deep learning models, characterized in that: The method comprises: S1. Analyze the target deep learning model using the Gradient Weighted Class Activation Mapping (Grad-CAM) method to generate a model focus heat map to identify the image region that contributes most to the prediction decision of the target deep learning model; S2, determining the key patch deployment area and patch center according to the model focus heat map, image size and resolution, and the image area with the largest contribution, and performing patch deployment operations accordingly; S3, in each of the patch deployment key areas, initialize an RGB disk, wherein the RGB disk is located at the center of the pixel point in the applicable patch, randomly assign an initial color value to each of the RGB disks, and set a transparency range; S4. By presetting the evolution strategy, the color and transparency of the RGB disc are tuned, a fitness function is defined, and the optimization is performed in combination with the misclassification confidence and the concealment of the adversarial sample; wherein the concealment of the adversarial sample is controlled according to the transparency; S5, performing mutation operation, selection operation and recombination operation, and obtaining the optimal RGB wafer attribute combination through iterative optimization; S6. Apply the optimal RGB disc attribute combination to the original image to generate an adversarial sample, and verify the misleading effect of the adversarial sample on the target deep learning model and its concealment to human vision.
2. The deep learning model covert attack method according to claim 1, characterized in that: In S1, in the gradient weighted class activation mapping Grad-CAM method, the last convolution layer of the target deep learning model is used to process the target category related gradient information to obtain the gradient weight, and the gradient weight is used to perform a weighted operation on the feature map output by the convolution layer to generate the model focus heat map.
3. The deep learning model covert attack method according to claim 1, characterized in that: The S1 includes: S11, using the Gradient Weighted Class Activation Mapping (Grad-CAM) method, calculate the gradient of the target category score relative to the last convolutional layer output of the target deep learning model, and obtain the gradient weight α of each feature map through a global average pooling operation. k c ; S12, then use the gradient weight to generate a class activation map L c Grad-CAM , thereby predicting the image area with the largest contribution of the target deep learning model based on the highlighted image for model decision-making; S13, using the gradient weight α k c Weight the feature map A k , apply the following ReLU function to generate the model focus heat map L c :
4. The deep learning model covert attack method according to claim 3 is characterized in that: In S11, the following logic is used to obtain the gradient weighted class activation map SGrad-CAM: In the formula, represents the class activation map for category c, A k represents the activation of the kth channel in the feature map, is the importance weight of feature map channel k for category c.
5. The deep learning model covert attack method according to claim 3, characterized in that: In S12, the score of the target category c is used to calculate the feature map A. k The gradient of , performs the global average pooling operation to obtain the gradient weight In the formula, y c is the model’s prediction score for category c, It is the feature map A k The activation value at position (i, j), Z is the feature map A k The total number of elements.
6. The deep learning model covert attack method according to claim 1, characterized in that: The S2 includes: S21, designing an area restriction patch, adjusting the size and shape of the area restriction patch to cover the main focus points in the model focus heat map, minimizing the interference of non-target areas, and obtaining an applicable area restriction patch; wherein the length l of the applicable patch is determined using the following logic: S22: deploying the area-restricted patch in the patch deployment key area according to the patch center.
7. The deep learning model covert attack method according to claim 1, characterized in that: The S3 includes: S31, adjusting the size of the RGB disk according to the size of the applicable patch and the detail level information of the patch deployment key area. The evolutionary algorithm is used to calculate the attribute vector of the RGB disk according to the following logic: To optimize: Where r represents the radius, (R, G, B) represents the color, and α represents the transparency. S32, define and according to the attribute vector And the fitness function Evaluate the impact of each preset configuration information on the misclassification probability of the target deep learning model, wherein the fitness function described below is used Perform disturbance evaluation, optimization and selection operations: In the formula, S(x orig ,x adv ) represents the original image x orig With adversarial sample x adv The similarity measure between them, λ is a trade-off parameter used to balance the importance between misclassification confidence and sample concealment. In this embodiment, the similarity measure uses the structural similarity index SSIM, and the concealment is the similarity of the original image; S33, initializing the radius r, color (R, G, B), and transparency α of the RGB disk.
8. The deep learning model covert attack method according to claim 7, characterized in that: The S33 includes: S331, randomly determine the radius of the disc: r~U(r min ,r max ), where r min and r max Respectively represent the minimum and maximum values of the radius; S332, randomly generate RGB color values; S333, randomly set the transparency of the disc: α~U(0,1); in this embodiment, 0 represents completely transparent, and 1 represents completely opaque; wherein, the transparency of the disc is randomly set using the following logic: α~U(0,1) In the formula, S(x orig ,x adv ) represents the similarity measure between the original image and the adversarial sample, and λ is a trade-off parameter that balances the importance of both.
9. The deep learning model covert attack method according to claim 1, characterized in that: The S5 includes: S51, performing the mutation operation to adjust the properties of the RGB disk to explore a new solution space. The mutation operation is performed using the following logic: S52: Perform the selection operation to retain the configuration of the RGB disk that maximizes the fitness function value. The selection operation is performed using the following logic: S53, when the configuration of the RGB disk is applicable, perform a recombination operation, combine the features corresponding to the configurations of at least two RGB disks to generate a new candidate solution, and set an iteration termination condition, wherein the following recombination operation is performed using the following logic: S54. When the iteration termination condition is met, stop the iteration operation and determine the optimal RGB wafer attribute combination, wherein the iteration termination condition is determined using the following logic: Iteration number ≥ N max Or Δf≤ò.
10. Deep learning model covert attack system, characterized by: The system comprises: A target model analysis module is used to analyze the target deep learning model using a Gradient Weighted Class Activation Mapping (Grad-CAM) method to generate a model focus heat map to identify the image region that contributes most to the prediction decision of the target deep learning model; A patch area determination module, used to determine the key patch deployment area and the patch center according to the model focus heat map, the image size and resolution, and the image area with the largest contribution, and perform patch deployment operations accordingly, the patch area determination module is connected to the target model analysis module; An RGB disc initialization module is used to initialize an RGB disc in each of the patch deployment key areas, wherein the RGB disc is located at the center of the pixel point in the applicable patch, and to randomly assign an initial color value to each of the RGB discs, and to set a transparency range, and the RGB disc initialization module is connected to the patch area determination module; An adversarial sample concealment control module is used to adjust the color and transparency of the RGB disc through a preset evolutionary strategy, define a fitness function, and optimize the adversarial sample concealment in combination with the misclassification confidence and the adversarial sample concealment; wherein the adversarial sample concealment is controlled according to the transparency, and the adversarial sample concealment control module is connected to the RGB disc initialization module; An iterative optimization module, used for performing mutation operations, selection operations and recombination operations, and obtaining an optimal RGB wafer attribute combination through iterative optimization, wherein the iterative optimization module is connected to the adversarial sample concealment control module; An adversarial sample generation and verification module is used to apply the optimal RGB disc attribute combination to the original image to generate an adversarial sample and verify the misleading effect of the adversarial sample on the target deep learning model and its concealment to human vision. The adversarial sample generation and verification module is connected to the iterative optimization module.
Citation Information
Patent Citations
Adversarial sample detection method and device based on multilayer saliency features
CN113837244A
Intelligent system confrontation sample generation method and system based on diversified input strategies
CN115527084A
Cross-shaped patch attack resisting method
CN115795450A
Remote sensing image-oriented patch deployable attack resisting method, device and equipment
CN116844052A
Method and system for generating adversarial attacks on video recognition models
US11776241B1
