Adversarial sample generation method under limited ship texture generation area

Abstract

The invention discloses an adversarial sample generation method under a limited ship texture generation area, and the method comprises the following steps: 1), carrying out the target detection of an image containing a target ship through a preset deep learning model; 2) extracting a deep convolutional layer feature map as an analysis object; 3) calculating a weight coefficient akc of the kth feature map when the kth feature map of the convolutional layer is Ak; (4) constructing a Grad-CAM thermodynamic diagram; 5) setting a judgment threshold value according to the maximum value of the pixel value of each thermodynamic image, and performing binarization processing on the thermodynamic image according to the threshold value to obtain position information of a feature region with strong judgment; 6) setting an adversarial texture at the position of the region with the strong decision feature to form an adversarial sample; and 7) taking the binarized thermodynamic diagram as a mask, and superposing the binarized thermodynamic diagram with the adversarial sample generated in the step 6) to obtain a final adversarial sample. The method can improve the anti-attack effect under the condition of reducing the ship texture area as much as possible.

Description

Technical Field

[0001] The present invention relates to deep learning technology, and in particular to a method for generating adversarial samples under a limited ship texture generation area. Background Art

[0002] Ship-borne drones, surface unmanned boats and other equipment can use the onboard deep learning-based target detection model to analyze the target images collected by the visual system, quickly locate the target and carry out precise strikes. However, due to their large size, slow maneuvering speed and obvious target features, surface ships are easily detected and identified and struck. Research has found that adversarial sample attacks can induce target detection models based on deep neural networks to output erroneous results by adding special disturbances to avoid detection and positioning. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide an adversarial sample generation method under a limited ship texture generation area in view of the defects in the prior art.

[0004] The technical solution adopted by the present invention to solve the technical problem is: a method for generating adversarial samples under a limited ship texture generation area, comprising the following steps:

[0005] 1) Perform target detection on images containing target ships through a preset deep learning model;

[0006] 2) Extract the feature map of the deep convolution layer as the analysis object; Extract the feature map of the deepest convolution layer of the model as the analysis object;

[0007] 3) The kth feature map of the convolutional layer is A k , calculate the weight coefficient of the kth feature map;

[0008] Calculate the gradient of the category score relative to the feature map. Assuming that the true label of the input image x is c, the model defines the category score of the input image as c as f c (x), the kth feature map of the convolutional layer is A k , the pixel value at position (i,j) in the kth feature map is f c (x) For The gradient represents the feature map A k The contribution of all pixel values ​​at all positions in the kth feature map to the category score is calculated, and then the gradient is normalized to obtain the weight coefficient of the kth feature map:

[0009]

[0010] 4) Construct Grad-CAM heat map;

[0011] The model performs weighted summation on the feature maps to obtain the Grad-CAM for a certain input image, uses ReLu to perform nonlinear transformation on the accumulated results, and discards all negative gradients;

[0012]

[0013] in, For the final heat map, is the weight of each feature map;

[0014] 5) Set the decision threshold according to the maximum value of each Grad-CAM pixel value, and By performing binarization processing, the location information of the area with strong decisive features can be obtained;

[0015] By setting a hyperparameter δ, combining the maximum value of each Grad-CAM Set the size of the threshold θ, as follows:

[0016]

[0017] The initial setting value of is 0.5;

[0018] 6) Setting adversarial textures at locations with strong deterministic feature regions to form adversarial samples;

[0019] 7) After binary processing As a mask, it is superimposed with the adversarial sample generated in step 6) to obtain the final adversarial sample:

[0020]

[0021] in, represents the Hadamard product operation of the matrix, ε is the adversarial texture sample, and ε' is the superimposed adversarial sample.

[0022] According to the above scheme, in step 6), the adversarial texture is an adversarial camouflage pattern based on Thiessen polygons.

[0023] According to the above scheme, the method for generating the anti-camouflage pattern is as follows:

[0024] Use Thiessen polygons to generate polygons composed of several camouflage colors; fragment the edges of the polygons by color sampling.

[0025] According to the above scheme, the method for generating the anti-camouflage pattern is as follows:

[0026] 6.1) Use differentiable functions to approximate Thiessen polygons to obtain approximate Thiessen polygons;

[0027] 6.2) Assume that the camouflage pattern uses N C Different camouflage colors, including N P Control points:

[0028]

[0029] For each color, its coordinates on the two-dimensional plane are expressed as

[0030] For each pixel x on the two-dimensional texture, its color is c k Probability Such as:

[0031]

[0032] in,

[0033] It is expressed as follows:

[0034]

[0035] Among them, the parameter α represents the smoothing radius of the approximate Thiessen polygon;

[0036] By optimizing the coordinates of the control points {b ij}, use the above formula to calculate N C ×H×W size probability distribution diagram

[0037] 6.3) According to the probability distribution diagram Color the camouflage pattern.

[0038] According to the above scheme, in step 6.3), according to the color probability distribution Color pixel x,

[0039]

[0040] Among them, c (x) is the color of pixel x, g i is a Gumbel random variable, For pixel x, color is c i The probability distribution of , τ is the temperature parameter;

[0041] g i =-log(-logu).

[0042] According to the above scheme, in step 6.3), according to the color probability distribution Color pixel x,

[0043]

[0044] Among them, c (x) is the color of pixel x, g i ' is the improved Gumbel random variable, For pixel x, color is c i The probability distribution of , τ is the temperature parameter;

[0045]

[0046] in, is the fitting variable, remains unchanged during the optimization process, while is an optimizable variable, and the hyperparameter λ controls the proportion of optimizable variables.

[0047] According to the above scheme, in step 6), when printing the adversarial texture pattern, the RGB value in the input digital world is converted by introducing the structural similarity loss SSIM.

[0048] The beneficial effects produced by the present invention are:

[0049] 1. The present invention can improve the effect of countering attack while minimizing the area of ​​texture by finding the position of the effective features detected by the target in the image in the physical space and attacking the position of the effective features detected by the target.

[0050] 2. The present invention proposes an adversarial camouflage generation method based on Thiessen polygons to combat the visual saliency problem of patches. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0052] Figure 1 It is a method flow chart of an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0054] like Figure 1 As shown, a method for generating adversarial samples under a limited ship texture generation area includes the following steps:

[0055] 1) Perform target detection on images containing target ships through a preset deep learning model;

[0056] 2) Extract the feature map of the deep convolution layer as the analysis object; Extract the feature map of the deepest convolution layer of the model as the analysis object;

[0057] 3) The kth feature map of the convolutional layer is A k , calculate the weight coefficient of the kth feature map;

[0058] Calculate the gradient of the category score relative to the feature map. Assuming that the true label of the input image x is c, the model defines the category score of the input image as c as f c (x), the kth feature map of the convolutional layer is A k , the pixel value at position (i,j) in the kth feature map is f c (x) For The gradient represents the feature map A k The contribution of all pixel values ​​at all positions in the kth feature map to the category score is calculated, and then the gradient is normalized to obtain the weight coefficient of the kth feature map:

[0059]

[0060] 4) Construct Grad-CAM heat map;

[0061] The model performs weighted summation on the feature maps to obtain the Grad-CAM for a certain input image, uses ReLu to perform nonlinear transformation on the accumulated results, and discards all negative gradients;

[0062]

[0063] in, For the final heat map, is the weight of each feature map;

[0064] 5) The size of each pixel value in Grad-CAM directly reflects the contribution of the corresponding area of ​​the image to the model classification decision. However, not all areas with numerical values ​​represent the feature areas with strong decision-making in the image. Some areas with small numerical values ​​may only have a weak impact on the model decision. If these weak decision areas are used as attack targets, the generated adversarial samples may not be effective in interfering with model predictions. Therefore, it is necessary to set a threshold to locate the important feature areas in the image.

[0065] The decision threshold is set according to the maximum value of each Grad-CAM pixel value. By performing binarization processing, the location information of the area with strong decisive features can be obtained;

[0066] By setting a hyperparameter δ, combining the maximum value of each Grad-CAM Set the size of the threshold θ, as follows:

[0067]

[0068] The initial setting value of is 0.5.

[0069] After binarization As a mask, it is superimposed with the subsequently generated adversarial sample to obtain the final adversarial sample:

[0070]

[0071] in, Represents the Hadamard product operation of matrices.

[0072] 6) Setting adversarial texture patterns at locations with strong deterministic feature regions to form adversarial samples;

[0073] The method to counter camouflage pattern generation is as follows:

[0074] Use Thiessen polygons to generate polygons composed of several camouflage colors; fragment the edges of the polygons by color sampling.

[0075] The method to counter camouflage pattern generation is as follows:

[0076] 6.1) Use differentiable functions to approximate Thiessen polygons to obtain approximate Thiessen polygons;

[0077] 6.2) Assume that the camouflage pattern uses N C Different camouflage colors, including N P Control points:

[0078]

[0079] For each color, its coordinates on the two-dimensional plane are expressed as

[0080] For each pixel x on the two-dimensional texture, its color is c k Probability Such as:

[0081]

[0082] in,

[0083] It is expressed as follows:

[0084]

[0085] Among them, the parameter α represents the smoothing radius of the approximate Thiessen polygon;

[0086] By optimizing the coordinates of the control points {b ij}, use the above formula to calculate N C ×H×W size probability distribution diagram

[0087] 6.3) According to the probability distribution diagram Color the camouflage pattern;

[0088] According to color probability distribution Color pixel x,

[0089]

[0090] Among them, c (x) is the color of pixel x, g i is a Gumbel random variable, For pixel x, color is c i The probability distribution of , τ is the temperature parameter;

[0091] In order to expand the parameter space that can be optimized and improve the stealth effect of camouflage texture, we use an

[0092] The variable g i Instead of the random variable g i , then according to the color probability distribution Color pixel x,

[0093]

[0094] Among them, c (x) is the color of pixel x, g i ' is the improved Gumbel random variable, For pixel x, color is c i The probability distribution of , τ is the temperature parameter;

[0095]

[0096] in, is the fitting variable, remains unchanged during the optimization process, while is an optimizable variable, and the hyperparameter λ controls the proportion of optimizable variables.

[0097] When printing adversarial texture patterns, the structural similarity loss SSIM is introduced to convert the RGB values ​​in the input digital world.

[0098] SSIM loss compares the similarity of two images based on brightness, contrast, and structure. The calculation methods of the three dimensions are as follows:

[0099]

[0100] Where c(x,y) is the comparison of contrast, l(x,y) is the comparison of brightness, s(x,y) is the comparison part of structure; μ x , μ y is the average value of all pixels in the image; σ x , σ y is the variance of all pixel values ​​in the image; C1, C2, C3 are small constants used to avoid the denominator being zero.

[0101] σ xy The calculation formula is as follows:

[0102]

[0103] The calculation method of SSIM loss is the product of the above three dimensions, as follows:

[0104] SSIM(x,y)=[l(x,y)] α *[c(x,y)] β *[s(x,y)] γ

[0105] In order to further improve the accuracy of fitting, the color difference caused by the migration process is simulated by fitting. Compared with polynomials, a three-layer fully connected neural network with higher fitting accuracy is more suitable for fitting. The specific calculation method of color conversion is as follows:

[0106] [R′ G′ B′]=C([RGB])

[0107] Among them, C(·) is a three-layer fully connected neural network that fits color conversion. The input is the RGB value in the digital world, and the output is the RGB value after being applied to the physical world.

[0108] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A method for generating adversarial samples under a limited ship texture generation area, characterized in that: The following steps are involved: 1) Perform target detection on images containing target ships through a preset deep learning model; 2) Extract the feature map of the deep convolution layer as the analysis object; Extract the feature map of the deepest convolution layer of the model as the analysis object; 3) The kth feature map of the convolutional layer is A k , calculate the weight coefficient of the kth feature map 4) Construct Grad-CAM heat map; The model performs weighted summation on the feature maps to obtain the Grad-CAM for a certain input image, uses ReLu to perform nonlinear transformation on the accumulated results, and discards all negative gradients; in, For the final heat map, is the weight of the feature map; 5) Set the decision threshold according to the maximum value of each Grad-CAM pixel value, and Perform binarization processing to obtain the location information of the area with strong decisive features; 6) Setting adversarial textures at locations with strong deterministic feature regions to form adversarial samples; 7) After binary processing As a mask, it is superimposed with the adversarial sample generated in step 6) to obtain the final adversarial sample.

2. The adversarial sample generation method under limited ship texture generation area according to claim 1 is characterized in that: Calculate the gradient of the category score relative to the feature map. Assuming that the true label of the input image x is c, the model defines the category score of the input image as c as f c (x), the kth feature map of the convolutional layer is A k , the pixel value at position (i,j) in the kth feature map is f c (x) For The gradient represents the feature map A k The contribution of all pixel values ​​at all positions in the kth feature map to the category score is then normalized to obtain the weight coefficient of the kth feature map. 。 3. The adversarial sample generation method under limited ship texture generation area according to claim 1 is characterized in that: In step 5), the decision threshold is set as follows: By setting a hyperparameter δ, combining the maximum value of each Grad-CAM Set the size of the threshold θ, as follows: The initial setting value of δ is 0.

5.

4. The adversarial sample generation method under limited ship texture generation area according to claim 1, characterized in that: In the step 6), the adversarial texture is an adversarial camouflage pattern based on Thiessen polygons.

5. The method for generating adversarial samples under limited ship texture generation area according to claim 4, characterized in that: The method for generating the anti-camouflage pattern is as follows: Use Thiessen polygons to generate polygons composed of several camouflage colors; fragment the edges of the polygons by color sampling.

6. The method for generating adversarial samples under limited ship texture generation area according to claim 1, characterized in that: The method for generating the anti-camouflage pattern is as follows: 6.1) Use differentiable functions to approximate Thiessen polygons to obtain approximate Thiessen polygons; 6.2) Assume that the camouflage pattern uses N C Different camouflage colors, including N P Control points: For each color, its coordinates on the two-dimensional plane are expressed as For each pixel x on the two-dimensional texture, its color is c k Probability Such as: in, It is expressed as follows: Among them, the parameter α represents the smoothing radius of the approximate Thiessen polygon; By optimizing the coordinates of the control points {b ij }, use the above formula to calculate N C ×H×W size probability distribution diagram 6.3) According to the probability distribution diagram Color the camouflage pattern.

7. The method for generating adversarial samples under limited ship texture generation area according to claim 6, characterized in that: In step 6.3), according to the color probability distribution Color pixel x, Among them, c (x) is the color of pixel x, g i is a Gumbel random variable, For pixel x, color is c i The probability distribution of , τ is the temperature parameter; Mr i =-log(-logu).

8. The method for generating adversarial samples under limited ship texture generation area according to claim 6, characterized in that: In step 6.3), according to the color probability distribution Color pixel x, Among them, c (x) is the color of pixel x, g i ' is the improved Gumbel random variable, For pixel x, color is c i The probability distribution of , τ is the temperature parameter; in, is the fitting variable, remains unchanged during the optimization process, while is an optimizable variable, and the hyperparameter λ controls the proportion of optimizable variables.

9. An electronic device, characterized in that: include: one or more processors; as well as a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are caused to perform the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images