An adversarial example generation method under the constraint of limited ship texture generation area

Abstract

This invention discloses an adversarial example generation method under the constraint of limited ship texture generation area, comprising the following steps: 1) performing target detection on an image containing a target ship using a preset deep learning model; 2) extracting feature maps from deep convolutional layers as the analysis object; 3) the k-th feature map of the convolutional layer is A. k Calculate the weight coefficient a of the k-th feature map. k c 4) Construct a Grad-CAM heatmap; 5) Set a decision threshold based on the maximum pixel value of each heatmap, and binarize the heatmap according to the threshold to obtain the location information of regions with strong decision features; 6) Set adversarial textures at the locations of regions with strong decision features to form adversarial samples; 7) Use the binarized heatmap as a mask and overlay it with the adversarial samples generated in step 6) to obtain the final adversarial samples. This invention can improve the effectiveness of adversarial attacks while minimizing the area of ​​ship textures.

Description

Technical Field

[0001] This invention relates to deep learning technology, and more particularly to an adversarial example generation method under the condition of limited area for generating ship textures. Background Technology

[0002] Shipborne drones and unmanned surface vessels can use onboard deep learning-based target detection models to analyze target images captured by vision systems, quickly locate targets, and launch precise strikes. However, surface vessels, due to their large size, slow speed, and easily identifiable features, are readily detected, identified, and targeted. Research has found that adversarial attacks can induce deep neural network-based target detection models to output incorrect results by adding specific perturbations, thus avoiding detection and location. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an adversarial example generation method under the limitation of ship texture generation area, which addresses the shortcomings of the prior art.

[0004] The technical solution adopted by this invention to solve its technical problem is: a method for generating adversarial examples under the limitation of ship texture generation area, comprising the following steps:

[0005] 1) Using a pre-defined deep learning model, target detection is performed on images containing the target ship;

[0006] 2) Extract feature maps from deep convolutional layers as the analysis object; extract feature maps from the deepest convolutional layer of the model as the analysis object;

[0007] 3) The k-th feature map of the convolutional layer is A k Calculate the weight coefficient of the k-th feature map;

[0008] The gradient of the class score relative to the feature map is calculated. Assuming the true label of the input image x is c, the class score by which the model classifies the input image as c is defined as f. c (x), the k-th feature map of the convolutional layer is A. k The pixel value at position (i,j) in the k-th feature map is f c (x) for The gradient represents the feature map A. k The contribution of each pixel value in the map to the category score is determined, and then the gradient is normalized to obtain the weight coefficients of the k-th feature map.

[0009]

[0010] 4) Construct Grad-CAM heatmap;

[0011] The Grad-CAM model for a given input image is obtained by weighted summation of the feature maps. ReLU is used to perform a non-linear transformation on the accumulated result, discarding all negative gradients.

[0012]

[0013] in, For the final heatmap, The weights for each feature map;

[0014] 5) Set a decision threshold based on the maximum value of each Grad-CAM pixel, and then apply the threshold to... By performing binarization, the location information of regions with strong decisive features can be obtained;

[0015] By setting a hyperparameter δ, and combining the maximum value of each Grad-CAM image... Set the threshold θ as follows:

[0016]

[0017] The initial setting value is 0.5;

[0018] 6) Set adversarial textures in regions with strong decisive features to form adversarial samples;

[0019] 7) Using the binarized data As a mask, it is superimposed on the adversarial sample generated in step 6) to obtain the final adversarial sample:

[0020]

[0021] in, ε represents the Hadamard product operation of the matrix, where ε is the adversarial texture sample and ε' is the superimposed adversarial sample.

[0022] According to the above scheme, in step 6), the anti-texture is an anti-camouflage pattern based on Thiessen polygons.

[0023] According to the above scheme, the method for countering camouflage pattern generation is as follows:

[0024] Use Thiessen polygons to generate polygons composed of several camouflage colors; break up the edges of the polygons by color sampling.

[0025] According to the above scheme, the method for countering camouflage pattern generation is as follows:

[0026] 6.1) The Thiessen polygon is approximated by a differentiable function to obtain an approximate Thiessen polygon;

[0027] 6.2) Assume the camouflage pattern uses N C There are several different camouflage colors, including N P One control point:

[0028]

[0029] For each color, its coordinates on a two-dimensional plane are represented as follows:

[0030] For each pixel x on the 2D texture, its color is c. k probability As shown in the formula:

[0031]

[0032] in,

[0033] It is expressed as follows:

[0034]

[0035] Wherein, parameter α represents the smooth radius of the approximate Thiessen polygon;

[0036] By optimizing the coordinates of the control points {b ij}, N is calculated using the above formula. C Probability distribution diagram of size ×H×W

[0037] 6.3) According to the probability distribution diagram Color the camouflage pattern.

[0038] According to the above scheme, in step 6.3), based on the color probability distribution... Color pixel x.

[0039]

[0040] Among them, c (x) For the color of pixel x, g i Let Gumbel be a random variable. For pixels x and color c i The probability distribution, where τ is the temperature parameter;

[0041] g i = -log(-logu).

[0042] According to the above scheme, in step 6.3), based on the color probability distribution... Color pixel x.

[0043]

[0044] Among them, c (x) For the color of pixel x, g i 'For improved Gumbel random variables, For pixels x and color c i The probability distribution, where τ is the temperature parameter;

[0045]

[0046] in, For the fitted variables, It remains unchanged during the optimization process, while It is an optimizable variable, and the hyperparameter λ controls the proportion of optimizable variables.

[0047] According to the above scheme, in step 6), when printing adversarial texture patterns, the RGB values ​​in the input digital world are converted by introducing structural similarity loss (SSIM).

[0048] The beneficial effects of this invention are:

[0049] 1. This invention finds the location of the effective features of the target being detected in the image in physical space, and detects the location of the effective features by attacking the target, which can improve the effectiveness of anti-attack while minimizing the area of ​​the texture.

[0050] 2. This invention proposes an adversarial camouflage generation method based on Thiessen polygons to combat the visual salience problem of patches. Attached Figure Description

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings:

[0052] Figure 1 This is a flowchart of a method according to an embodiment of the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0054] like Figure 1 As shown, an adversarial example generation method under the constraint of limited ship texture generation area includes the following steps:

[0055] 1) Using a pre-defined deep learning model, target detection is performed on images containing the target ship;

[0056] 2) Extract feature maps from deep convolutional layers as the analysis object; extract feature maps from the deepest convolutional layer of the model as the analysis object;

[0057] 3) The k-th feature map of the convolutional layer is A k Calculate the weight coefficient of the k-th feature map;

[0058] The gradient of the class score relative to the feature map is calculated. Assuming the true label of the input image x is c, the class score by which the model classifies the input image as c is defined as f. c (x), the k-th feature map of the convolutional layer is A. k The pixel value at position (i,j) in the k-th feature map is f c (x) for The gradient represents the feature map A. k The contribution of each pixel value in the map to the category score is determined, and then the gradient is normalized to obtain the weight coefficients of the k-th feature map.

[0059]

[0060] 4) Construct Grad-CAM heatmap;

[0061] The Grad-CAM model for a given input image is obtained by weighted summation of the feature maps. ReLU is used to perform a non-linear transformation on the accumulated result, discarding all negative gradients.

[0062]

[0063] in, For the final heatmap, The weights for each feature map;

[0064] 5) In Grad-CAM, the value of each pixel directly reflects the contribution of the corresponding region of the image to the model's classification decision. However, not all regions with numerical values ​​represent highly decisive feature regions in the image. Some regions with smaller values ​​may only have a weak impact on the model's decision. If these weakly decisive regions are targeted, the generated adversarial examples may not be effective in interfering with the model's prediction. Therefore, it is necessary to set a threshold to locate important feature regions in the image.

[0065] A decision threshold is set based on the maximum value of each Grad-CAM pixel, and the decision is based on the threshold. By performing binarization, the location information of regions with strong decisive features can be obtained;

[0066] By setting a hyperparameter δ, and combining the maximum value of each Grad-CAM image... Set the threshold θ as follows:

[0067]

[0068] The initial setting value is 0.5.

[0069] After binarization As a mask, it is superimposed on the subsequently generated adversarial examples to obtain the final adversarial example:

[0070]

[0071] in, This represents the Hadamard product operation of matrices.

[0072] 6) Set adversarial texture patterns in locations with strong decisive features to form adversarial samples;

[0073] The methods to counter camouflage pattern generation are as follows:

[0074] Use Thiessen polygons to generate polygons composed of several camouflage colors; break up the edges of the polygons by color sampling.

[0075] The methods to counter camouflage pattern generation are as follows:

[0076] 6.1) The Thiessen polygon is approximated by a differentiable function to obtain an approximate Thiessen polygon;

[0077] 6.2) Assume the camouflage pattern uses N C There are several different camouflage colors, including N P One control point:

[0078]

[0079] For each color, its coordinates on a two-dimensional plane are represented as follows:

[0080] For each pixel x on the 2D texture, its color is c. k probability As shown in the formula:

[0081]

[0082] in,

[0083] It is expressed as follows:

[0084]

[0085] Wherein, parameter α represents the smooth radius of the approximate Thiessen polygon;

[0086] By optimizing the coordinates of the control points {b ij}, N is calculated using the above formula. C Probability distribution diagram of size ×H×W

[0087] 6.3) According to the probability distribution diagram Color the camouflage pattern;

[0088] Based on color probability distribution Color pixel x.

[0089]

[0090] Among them, c (x) For the color of pixel x, g i Let Gumbel be a random variable. For pixels x and color c i The probability distribution, where τ is the temperature parameter;

[0091] To expand the optimizable parameter space and improve the stealth effect of camouflage textures, we use an optimizable parameter space.

[0092] The variable g of the transformation i Replace random variable g i Then, according to the color probability distribution Color pixel x.

[0093]

[0094] Among them, c (x) For the color of pixel x, g i 'For improved Gumbel random variables, For pixels x and color c i The probability distribution, where τ is the temperature parameter;

[0095]

[0096] in, For the fitted variables, It remains unchanged during the optimization process, while It is an optimizable variable, and the hyperparameter λ controls the proportion of optimizable variables.

[0097] When printing adversarial texture patterns, SSIM (Structural Similarity Loss) is used to transform the RGB values ​​in the input digital world.

[0098] SSIM loss compares the similarity of two images along three dimensions: brightness, contrast, and structure. The calculation methods for each dimension are shown in the following equations:

[0099]

[0100] In the formula, c(x,y) represents the comparison of contrast, l(x,y) represents the comparison of brightness, and s(x,y) represents the comparison of structure; μ x μ y It is the average value of all pixels in the image; σ x , σ y C1, C2, and C3 are constants with very small values, used to avoid a denominator of 0.

[0101] σ xy The calculation formula is shown below:

[0102]

[0103] The SSIM loss is calculated as the product of the three dimensions mentioned above, as shown in the following formula:

[0104] SSIM(x,y)=[l(x,y)] α *[c(x,y)] β *[s(x,y)] γ

[0105] To further improve the accuracy of the fitting, a fitting method is used to simulate the color differences caused by the migration process. Compared to polynomials, a three-layer fully connected neural network with higher fitting accuracy is more suitable for fitting. The specific calculation method for color transformation is as follows:

[0106] [R′ G′ B′]=C([RGB])

[0107] C(·) is a three-layer fully connected neural network that fits the color conversion. The input is the RGB value in the digital world, and the output is the RGB value after being applied to the physical world.

[0108] It should be understood that those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for generating adversarial examples under limited area for ship texture generation, characterized in that, Includes the following steps: 1) Using a pre-defined deep learning model, target detection is performed on images containing the target ship; 2) Extract feature maps from deep convolutional layers as the analysis object; extract feature maps from the deepest convolutional layer of the model as the analysis object; 3) The first convolutional layer Each feature map is Calculate the first Weight coefficients of each feature map ; 4) Construct Grad-CAM heatmap; The Grad-CAM model for a given input image is obtained by weighted summation of the feature maps. ReLU is used to perform a non-linear transformation on the accumulated result, discarding all negative gradients. in, For the final heatmap, The weights of the feature map; 5) Set a decision threshold based on the maximum value of each Grad-CAM pixel, and then apply the threshold to... Binarization is performed to obtain the location information of regions with strong decisive features; 6) Set adversarial textures in regions with strong decisive features to form adversarial samples; In step 6), the adversarial texture is an adversarial camouflage pattern based on Thiessen polygons; The methods to counter camouflage pattern generation are as follows: 6.1) The Thiessen polygon is approximated by a differentiable function to obtain an approximate Thiessen polygon; 6.2) Camouflage pattern usage Different camouflage colors, including One control point: ， For each color, its coordinates on a two-dimensional plane are represented as follows: For each pixel on the two-dimensional texture Its color is probability As shown in the formula: ； in, It is expressed as follows: Among them, parameters The smooth radius represents the approximate Thiessen polygon. By optimizing the coordinates of the control points The above formula is used to calculate the result. Probability distribution diagram of size ; 6.3) According to the probability distribution diagram Color the camouflage pattern; 7) Using the binarized result As a mask, it is superimposed on the adversarial sample generated in step 6) to obtain the final adversarial sample.

2. The adversarial example generation method under the limited area of ​​ship texture generation according to claim 1, characterized in that, Calculate the gradient of the category score relative to the feature map, assuming the input image... The real label is The model will classify the input image as... The category score is defined as The first convolutional layer Each feature map is , No. The pixel value at position (x, y) in the feature map is , for The gradient represents the feature map. The contribution of each pixel value to the category score is determined, and then the gradient is normalized to obtain the result. Weight coefficients of each feature map : 。 3. The adversarial example generation method under the limited area of ​​ship texture generation according to claim 1, characterized in that, In step 5), the decision threshold is set as follows: By setting a hyperparameter Combined with the maximum value of each Grad-CAM image Set threshold The size is as follows: The initial setting value is 0.

5.

4. The adversarial example generation method under the limited area of ​​ship texture generation according to claim 1, characterized in that, In step 6.3), based on the color probability distribution Give pixels Coloring, in, For pixels The color, Let Gumbel be a random variable. For pixels The color is The probability distribution, It is a temperature parameter; 。 5. The adversarial example generation method under the limited area of ​​ship texture generation according to claim 1, characterized in that, In step 6.3), based on the color probability distribution Give pixels Coloring, in, For pixels The color, For improved Gumbel random variables, For pixels The color is The probability distribution, It is a temperature parameter; in, For the fitted variables, It remains unchanged during the optimization process, while These are optimizable variables, hyperparameters. The proportion of optimizable variables was controlled.

6. An electronic device, characterized in that, include: One or more processors; as well as Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 5.

Images