SAR image-oriented camouflage image generation method

Abstract

The invention discloses a camouflage image generation method for an SAR (Synthetic Aperture Radar) image. The camouflage image generation method comprises the following steps of describing a process of generating the SAR image by a 3D (Three-Dimensional) model as follows: initializing C by using # imgabs0 # and # imgabs1 #; starting iteration; updating a mask vector and # imgabs2 # to carry out post-processing on the C to obtain a two-dimensional SAR image; inputting the image into an automatic target identification classification network to obtain a prediction label; calculating a loss function; and updating # imgabs3 # by using a gradient backward propagation algorithm, and after iteration is finished, generating a final disguised image according to the mapping model. The invention provides a camouflage image generation method based on an SAR imaging mechanism for the first time, and a camouflage image conforming to SAR imaging characteristics can be generated; a RaySAR-based physical countermeasure attack pipeline is realized, and the effectiveness of disguise is proved by testing.

Description

Technical Field

[0001] The invention belongs to the technical field of SAR images, and in particular relates to a camouflage image generation method for SAR images. Background Art

[0002] Deep convolutional neural networks (DCNNs) have been widely used in the field of remote sensing image recognition, including optical remote sensing image target recognition and SAR automatic target recognition (SAR ATR). However, the existence of camouflage greatly limits the application of SAR ATR. Through this carefully designed small perturbation that is difficult for the human eye to recognize, DCNN can be misled to produce wrong prediction results. The camouflage problem has received widespread attention from academia and industry, and has become a focus of attention in the field of artificial intelligence security.

[0003] At present, there are many camouflage practices in the field of remote sensing. In the field of SAR ATR, researchers are also conducting research on camouflage attacks, most of which are digital attacks: directly adding perturbation camouflage to the pixels of the image, and the perturbation is in the digital space. For example, FGSM and BIM are applied to the attack of SAR ATR classifiers, or universal adversarial perturbations are used to deceive SAR ATR classifiers. The perturbations generated by these attack methods are hidden, making them difficult for observers to detect, which poses a security threat to the widespread application of SAR ATR.

[0004] However, most of these digital attack methods are based on the principle of optical image attack, without considering the difference between the imaging mechanism of SAR and optical images. The pixel information of SAR images is actually backscattered information related to factors such as radar system parameters and surface characteristics of objects, while the imaging of visible light is mainly affected by the color and surface characteristics of objects. Different colors and materials absorb and reflect light of different wavelengths. Therefore, the images generated based on the principle of optical image attack may not conform to the imaging characteristics of SAR. At the same time, considering the possibility of real-world implementation of adversarial attacks, this application focuses on physical attacks: adding disturbances to the real-world environment or physical simulators, and the disturbances are in physical space. Compared with digital attacks, physical attacks pose a greater potential threat to society. In addition, we believe that the camouflaged images generated by simulators based on the SAR imaging mechanism are more in line with the imaging characteristics of SAR, which will make the attack more concealed.

[0005] Physical adversarial attacks have received extensive attention in the field of optical image recognition, such as patch-based attacks and camouflage-based attacks. These methods essentially achieve attacks by modifying the pattern on the surface of an object. However, these methods cannot be directly transferred to the SAR field because of the difference in imaging mechanisms. SAR images essentially record the backscattering information of ground objects. Therefore, simply modifying the pattern on the surface of an object cannot change the pixel value of the SAR image. It is necessary to change the material properties of the object surface, such as scattering coefficient, surface roughness and other parameters, so as to change the radar echo signal and then change the pixel value of the SAR image.

[0006] At the same time, in order to simulate the SAR imaging process, an end-to-end SAR simulation renderer should be used. In previous studies, the SAR simulation program RaySAR based on ray tracing has been widely used as a simulator for SAR imaging, such as in the analysis, design and verification of SAR systems, auxiliary SAR image interpretation, SAR image processing algorithm verification, etc. However, since RaySAR is a non-differentiable system, physical adversarial perturbations cannot be directly obtained by optimization, and a physical adversarial attack pipeline based on RaySAR must be proposed. Summary of the invention

[0007] To solve the above problems, the present application proposes a pipeline for SAR physical adversarial attack (SAR-PAA). First, in order to connect the digital world with the physical world, the camouflage of the present application includes a digital part and a physical part. The digital camouflage of the present application acts on the SAR geometric output data, that is, the scattering point and scattering intensity information generated by the ray tracer. The physical camouflage of the present application acts on the material parameters of the 3D object, and uses RaySAR to generate a two-dimensional SAR image. Then, by designing an effective loss function, the camouflage is updated through back propagation. Finally, the final camouflage is mapped back to the material information of the 3D model through a mapping model, thereby realizing the update of the physical camouflage.

[0008] To achieve the above-mentioned purpose, the present application discloses a camouflage image generation method for SAR images, the method comprising the following steps:

[0009] The process of generating SAR images from 3D models is described as: C is the SAR geometric output data, M is the grid information, T is the material information, θ c is the imaging parameter, R is the ray tracing renderer;

[0010] use Initialize C, is the final adversarial SAR geometric output data;

[0011] Start iteration. In each iteration, perform the following steps:

[0012] Update the mask vector and

[0013] Post-processing of C Get a two-dimensional SAR image I;

[0014] Input image I into the SAR automatic target recognition classification network Get predicted labels

[0015] Calculate the loss function;

[0016] Update using the gradient backpropagation algorithm

[0017] After the iteration, according to the mapping model Generate the final camouflaged image.

[0018] Preferably, the grid information includes vertex coordinates and information of triangular face elements, the material information includes reflection coefficient, scattering coefficient and surface roughness; the imaging parameter θ c Including radar signal source coordinates, receiving antenna coordinates and pointing position; the ray tracing renderer An approximate physical optics model is used to simulate the specular reflection and diffuse reflection of the radar signal, and the SAR geometric output data C is recorded, which includes the reflection direction and ray path of the echo position in the image.

[0019] Preferably, the process of obtaining the two-dimensional SAR image I is expressed as Indicates that, where θ p are post-processing parameters, including the maximum and minimum pitch angles, the maximum and minimum range angles, the spacing, and the maximum number of ray ejections. H and W are the height and width of the image, respectively.

[0020] Preferably, the perturbation to C is defined as digital camouflage, and the perturbation generation is regarded as an optimization problem, whose objective function is as follows:

[0021]

[0022] In the formula, C adv As the adversarial SAR geometry output data is optimized in iterations, is the final adversarial SAR geometric output data, is the loss function, y gt is the true label.

[0023] Preferably, the loss function is as follows:

[0024]

[0025] Where ω is the parameter to be optimized, x is the intensity value in the SAR geometric output data, c is a constant used to balance the classification loss and limit the perturbation intensity loss, and H is the cross entropy loss function.

[0026] Preferably, in order to generate effective two-dimensional SAR images, the range of intensity values ​​is limited to [0,1], and for the perturbation vector δ, the goal is to make 0≤x i +δ i ≤1,x i is a single intensity value in the SAR geometric output data, δ i is the perturbation value applied to a single intensity value. Since the range of the sigmoid function is (0,1), let δ i =sigmoid(ω i )―x i , so that 0≤sigmoid(ω i )=x i +δ i ≤1; in the initialization parameter ω i When , use the inverse function of the sigmoid function To ensure the similarity between the perturbation and the original intensity value vector; the L2 norm is used to limit the perturbation vector δ, that is, ||δ||2 is minimized.

[0027] Preferably, a sparse attack strategy is adopted to limit the range of the disturbance scattering points, that is, to limit the number of disturbance scattering points, specifically including:

[0028] Initialize the mask vector, which has the same shape as the perturbation vector and all its elements are 1; multiply the generated perturbation by the mask vector element by element; arrange the elements in the perturbation vector in order of size, and if the element is less than a certain threshold, set the element with the corresponding index in the mask vector to 0, that is, eliminate the perturbations in the perturbation vector that are less than a certain threshold, so that the 0 elements in the mask vector continue to increase during iteration.

[0029] Preferably, the mapping model Determine 3 transformation relationships:

[0030] Coordinate transformation: from the coordinate system of the scattering points in C to the coordinate system of the 3D model;

[0031] Points correspond to triangular surfaces;

[0032] From intensity values ​​to material parameters: Changes in the intensity values ​​in the SAR geometry output data correspond to changes in the material parameters in the 3D model.

[0033] Preferably, for coordinate transformation, a ray tracing renderer Record the coordinates of each scattering point in the coordinate system of the 3D model, so as to realize the conversion from the coordinate system of the scattering point to the coordinate system of the 3D model; for the points corresponding to the triangular surface element, select the scattering point with the largest change in intensity value before and after the disturbance as the benchmark, and calculate the material parameters of the triangular surface element;

[0034] For the intensity value to material parameters, a single variable optimizer is used to perform bounded optimization on the scattering coefficient of the triangle face corresponding to each scattering point; specifically, a function is defined that inputs the scattering coefficient and uses the same imaging parameters as before through a ray tracing renderer. The absolute value of the difference between the output intensity value and the target intensity value is obtained, and the Brent algorithm is used to find the local minimum of this function in the interval [0,1] to obtain the scattering coefficient corresponding to the target intensity value on the triangular face.

[0035] The beneficial effects of this application are as follows:

[0036] For the first time, a new SAR ATR camouflage image generation method based on SAR imaging mechanism is proposed, which can generate camouflage images that conform to SAR imaging characteristics.

[0037] A RaySAR-based physical adversarial attack pipeline was implemented and tested in both the digital and physical worlds using three different 3D models, demonstrating the effectiveness of the camouflage.

[0038] The present application is an end-to-end physical camouflage that is possible to implement in the real world and has certain mobility in different network structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 The framework diagram of the present invention;

[0040] Figure 2 Schematic diagram of mapping scattering points to triangular facets;

[0041] Figure 3 Renderings of the 3D models used;

[0042] Figure 4 The SAR camouflage image generated by the embodiment of the present application is compared with the real SAR image of the MSTAR dataset, the upper part is the real SAR image, and the lower part is the SAR camouflage image;

[0043] Figure 5 Schematic diagram of the position of the physical adversarial perturbation generated by an embodiment of the present application on a 3D model. DETAILED DESCRIPTION

[0044] The present invention is further described below in conjunction with the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention belong to the protection scope of the present invention.

[0045] Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0046] Before introducing the embodiments of the present application, some terms involved in the present application are first explained.

[0047] 1.SAR Auto Target Recognition Researchers tend to design specialized CNNs. Xu et al. proposed a deep CNN model SARnet for SAR ATR with two convolution-pooling layers and two fully connected layers. It achieved an accuracy of 95.68% on the MSTAR dataset. Zhai et al. proposed MF-SarNet for SAR ATR. The Fire module in the network is used to extract features and implement fewer parameters. MF-SarNet consists of 18 convolutional layers, 2 fully connected layers, and 8 Fire modules. The dataset was expanded 360 times using clockwise rotation data enhancement, achieving an accuracy of 98.53% on MSATR. Shang et al. proposed M-Net to solve the overfitting problem caused by insufficient samples. M-Net uses information recorders to store spatial features and uses spatial similarity to predict labels for unknown samples.

[0048] 2. Adversarial Attacks on SAR ATR Attack methods designed for SAR ATR. These methods usually take into account the special mechanism of SAR imaging and have a high correlation with SAR imaging.

[0049] 3. SARSimulationRaySAR is a SAR simulator based on ray tracing method developed by Auer et al. It is an extension of the open source ray tracing rendering software POV-Ray, which is designed to simulate radar signals in 3D, namely azimuth, distance and elevation. RaySAR uses an approximate physical optical model to simulate the specular reflection component and diffuse reflection component of the radar signal, and uses ray tracing method to record the reflection direction and reflection component. The ray path determines the position of the echo in the image.

[0050] The technical solutions provided in the embodiments of the present application involve technologies such as machine learning and natural language processing of artificial intelligence, which are specifically introduced and explained through the following embodiments.

[0051] First, the forward process of generating SAR images from a 3D model is defined as follows: given a 3D model, it has grid information M and material information T, the grid information includes vertex coordinates and triangular face information, and the material information includes reflection coefficient, scattering coefficient and surface roughness, etc. Imaging parameter θ c Including the radar signal source coordinates, receiving antenna coordinates and pointing position, etc. Using a ray tracing renderer R, an approximate physical optical model is used to simulate the specular reflection and diffuse reflection of the radar signal, and the reflection direction and ray path C that determine the echo position in the image are recorded. This process can be expressed by the formula In order to generate a two-dimensional SAR image I, C needs to be post-processed This process can be expressed as Indicates that, where θ p are post-processing parameters, including the maximum / minimum value of the pitch angle, the maximum / minimum value and spacing of the range, the maximum number of ray ejections, etc. Input image I into the SAR automatic target recognition classification network And get the predicted label

[0052] The camouflage method of this application will be implemented in digital space and physical space in turn. First, adversarial perturbations are generated for SAR geometric output data C in digital space, and then the perturbations are further mapped to material information T in physical space. Therefore, this application defines perturbations to C as digital attacks, and perturbation generation is regarded as an optimization problem, whose objective function is as follows:

[0053]

[0054] In the formula, is the final adversarial SAR geometric output data, is the loss function, y gt is the true label. By solving the optimization problem described in Formula 1, the adversarial SAR geometric output data is obtained.

[0055] After perturbing C, we can further map Mapped to the physical space, we define this process as a physical attack, which can be expressed as

[0056]

[0057] In the formula, is the final adversarial physical material information, and the final physical adversarial perturbation can be obtained through this mapping. The symbols used in this paper are summarized in Table 1.

[0058] Table 1 Symbol table

[0059]

[0060] In order to generate SAR physical adversarial perturbations, this application proposes a framework for SAR physical adversarial attacks, including a differentiable SAR geometric output data post-processor. Figure 1 shown.

[0061] Figure 1 In this paper, an optimization method is first used to generate adversarial perturbations in SAR geometric output data, and the designed loss function is used to update the perturbations through back propagation to obtain the final perturbations. Finally, this perturbation is mapped to the material information of the 3D model through a mapping model.

[0062] This application calls the process of generating disturbances in SAR geometric output data digital camouflage, because the disturbance is still in the digital space. In this part, an optimization method is used to generate adversarial disturbances by calculating the loss and performing gradient back propagation. Therefore, the loss function is very important in this process, which determines the final attack effect. The loss function design will be discussed in detail later.

[0063] Correspondingly, this application calls the process of mapping the perturbations generated in the SAR geometric output data to the 3D model physical camouflage, because this step is actually mapping the perturbations in the digital space to the physical space. In this section, the design of the mapping model and how to implement this mapping will be discussed in detail. Generating adversarial perturbation in Digital domain (GaPD)

[0064] In the digital camouflage part, an optimization method is used to generate adversarial perturbations in the SAR geometric output data C. When designing the loss function, the following three factors are mainly considered:

[0065] (1) Objective function. Our strategy is to use an untargeted attack method to automatically find the wrong target label. Our optimization goal is to make the predicted value of the adversarial sample approach the target label, and finally make the model produce a wrong classification. Specifically, we use the classification loss function of the model, that is, the cross entropy loss, as the objective function, and select the label with the highest prediction probability other than the correct label as the target label to minimize the loss function and find the adversarial sample. However, using only this loss function cannot make the adversarial sample effective, because there is no limit on the scope of perturbation optimization, which may make the adversarial sample inconsistent with the imaging principle of SAR or violate certain characteristics of SAR images.

[0066] (2) Intensity value range restriction. Since the optimization target is the intensity value contributed by the scattering points in the SAR geometric output data, in order to generate effective two-dimensional SAR images, it is necessary to limit the range of intensity values ​​to [0, 1]. Let the perturbation vector be δ, and the goal is to make 0≤x i +δ i ≤1. Since the range of the sigmoid function is (0,1), let δ i =sigmoid(ω i )―x i , so that 0≤sigmoid(ω i )=x i +δ i ≤1. Therefore, the sigmoid function is used to limit the range of optimization parameters to ensure that the generated disturbance is reasonable. i When , use the inverse function of the sigmoid function To ensure the similarity of the perturbation to the original intensity value vector.

[0067] (3) Limitation of perturbation amplitude. In order to increase the authenticity of the generated adversarial samples, and because we need to map the perturbation of the intensity value to the material parameters of the 3D model in the subsequent physical attack, the smaller the perturbation amplitude, the more conducive it is to the subsequent scattering coefficient matching process. The L2 norm is used to limit the perturbation vector δ, that is, to minimize ||δ||2. The final loss function is as follows:

[0068]

[0069] Where ω is the parameter to be optimized, x is the intensity value in the SAR geometric output data, c is a constant that can be used to balance the classification loss and limit the perturbation intensity loss, and H is the cross entropy loss function.

[0070] In addition, since the material information of the triangular facets of the 3D model corresponding to each scattering point needs to be modified in the next step of physical camouflage, the present application adopts a sparse attack strategy to limit the range of the perturbed scattering points, that is, to limit the number of perturbed scattering points. The iterative algorithm in the L0 attack proposed by Carlini and Wagner is used to implement the sparse attack. In each iteration, some pixels that have little effect on the classifier output are fixed until a minimum subset of perturbed pixels is determined. Specifically, first, a mask vector is initialized, which has the same shape as the perturbation vector, and all the elements are 1; the generated perturbation is element-by-element multiplied (Hadamard product) with the mask vector, so to limit the scope of the perturbation, it is only necessary to limit the number of 1 elements in the mask vector; the elements in the perturbation vector are arranged in order of size, and if the element is less than a certain threshold, the element with the corresponding index in the mask vector is set to 0, that is, the perturbation in the perturbation vector that is less than a certain threshold is eliminated, and the 0 elements in the mask vector are allowed to increase continuously during iteration. Mapping adversarial perturbation from Digital domain to Physical domain (MaPP)

[0071] In order to map the disturbance of SAR geometric output data C in digital space to physical space, a mapping model is defined This process is used to generate the final adversarial material information and is called physical camouflage.

[0072] In this mapping model, three transformation relationships are determined:

[0073] 1. Coordinate transformation: from the coordinate system of the scattered points in C to the coordinate system of the 3D model

[0074] 2. Points correspond to triangular surfaces: The basic unit of the intensity value in C is the scattering point, while the basic unit of the 3D model material parameter is the triangular surface element. A corresponding relationship needs to be established.

[0075] 3. From intensity values ​​to material parameters: Changes in the intensity values ​​in the SAR geometric output data correspond to changes in the material parameters in the 3D model

[0076] For coordinate conversion, the ray tracing renderer R can be used to record the coordinates of each scattering point in the coordinate system of the 3D model, thereby realizing the conversion of the scattering point coordinate system to the 3D model coordinate system; for the correspondence of points to triangular faces, since the areas of the triangular faces that make up the 3D model are inconsistent, according to the triangular faces of different areas, our experiments show that a triangular face with a smaller area may correspond to one scattering point, while a triangular face with a larger area may correspond to multiple scattering points. Our goal is to make the basic unit of the 3D model material parameter action - the triangular face and the basic unit of the intensity value in C - the scattering point correspond one to one, so that the change of the scattering point intensity value can be simulated by changing the material parameters of the triangular face. Therefore, considering the different effects of multiple scattering points on the same triangular face, we select the scattering point with the largest change in intensity value before and after the disturbance as the benchmark to calculate the material parameters of the triangular face.

[0077] Figure 2 When the scattering points are mapped to triangular face elements, since the triangular face elements have different sizes, a triangular face element with a larger area may correspond to multiple scattering points. However, the present application can only select one scattering point as a reference to calculate the material parameters of the triangular face element.

[0078] For intensity values ​​to material parameters, due to the ray tracing renderer The calculation of intensity values ​​needs to consider many factors, such as the angle between the triangle and the light source, so it is difficult to directly obtain the material parameters of the corresponding triangle through a certain intensity value. In addition, since radar signals are usually scattered on the surface of objects rather than specularly reflected in SAR imaging, the influence of the scattering coefficient on the intensity value dominates among all material parameters. We use a single variable optimizer to perform bounded optimization on the scattering coefficient of the triangle corresponding to each scattering point. Specifically, we define a function that inputs the scattering coefficient and uses the same imaging parameters as before through a ray tracing renderer. The absolute value of the difference between the output intensity value and the target intensity value is obtained, and the Brent algorithm is used to find the local minimum of this function in the interval [0,1] to obtain the scattering coefficient corresponding to the target intensity value on the triangular face.

[0079] The pseudo code of the attack method can be expressed as Algorithm 1.

[0080]

[0081] There is currently no physical dataset for SAR simulation research (i.e., a dataset containing multiple scene information and 3D models), so a new dataset was constructed by expanding the MSTAR dataset, which will be described in detail in the next section. Due to the differences in datasets and the fact that the principles of the attack methods in this application are different from the existing attack methods, the adversarial perturbations in this application are not directly added to the image space, whether digital or physical, so our experiments did not select a baseline.

[0082] To this end, the experiment will mainly verify whether the attack method we proposed is feasible, rather than whether the attack effect is strong. First, the effects of the proposed digital attack and physical attack will be verified respectively, then their migration in different network architectures will be verified respectively, and finally the influence of different pitch angles on the attack effect will be verified.

[0083] In the experiments, we used 3D models of three different vehicles: sedan, SUV, and van. The renderings of these 3D models are shown in Figure 2. Figure 3 As shown in Table 2, the number of their vertices and triangles are shown in Table 2. We use these 3D models to generate 2D SAR images, and use them to expand the MSTAR dataset to form a classification dataset with 13 categories. Specifically, using parameters similar to the MSTAR dataset, with the same pitch angle, SAR images of size 172x172 pixels are generated at 360 degrees with a spacing of 1 degree. The final dataset has 13 categories: 2S1, BMP2, BRDM_2, BTR60, D7, SLICY, T62, T72, ZIL131, ZSU_23_4, CAR, JEEP, VAN, and the number of images in each category is between 428 and 720. The specific distribution is shown in Table 3.

[0084] Table 2 3D models and comparison of their number of vertices and triangles.

[0085] 3D Model CAR JEEP VAN vertices 116929 50651 76214 triangles 207830 93927 139097

[0086] Table 3 shows all categories and details of the training and test sets after the expansion of the MSTAR dataset.

[0087] Class Training set Test set 2S1 233 274 BMP2 233 195 BRDM_2 233 274 BTR60 233 195 D7 233 274 SLICY 233 274 T62 233 273 T72 232 196 ZIL131 233 274 ZSU_23_4 233 274 CAR 360 360 JEEP 360 360 VAN 360 360

[0088] Figure 4 Comparison between the SAR simulation image generated for this application and the real SAR image of the MSTAR dataset. The upper one is the real SAR image and the lower one is the SAR simulation image.

[0089] This application uses the fooling rate as an indicator to evaluate the attack effect. The fooling rate is defined as follows:

[0090]

[0091] Among them, N total is the total number of samples participating in the attack, F is the classifier network, x adv is the generated adversarial sample, y gt are their true labels. The fooling rate is the ratio of the number of samples misclassified by the classifier network to the total number of samples involved in the attack.

[0092] Six different target classification and recognition networks were selected to test the camouflaged images generated by this application, including three mainstream target classification and recognition networks and three lightweight target classification and recognition networks: ResNet-50, VGG-16, DenseNet-121, Mobilenet V2, SqueezeNet and ShuffleNet V2. All networks used the official implementation of PyTorch. These classifier networks were trained separately, using the ImageNet pre-trained model, and the hyperparameters were set as follows: the optimizer used SGD with momentum, the learning rate was 0.001, the momentum was 0.9, the batch size was 32, and the maximum epoch was 80. The classification accuracy of the trained classifier network is shown in Table 4.

[0093] Table 4: The accuracy of all trained classifier networks on the test set.

[0094]

[0095] For the adversarial attack method of this application, the hyperparameters are set as follows: the optimizer uses Adam, the learning rate is 0.05, the coefficient is 1, and the maximum number of iterations is 50. The experiments are conducted on a workstation equipped with an NVIDIA GTX 1080Ti 11GB GPU.

[0096] We conducted attack experiments on the test set and ensured that the images were correctly classified before the attack. The results are divided into two parts: the attack effects in the digital domain and their corresponding attack effects in the physical domain.

[0097] In the digital field, the fooling rates of adversarial samples generated for 3 targets on 6 networks were statistically analyzed, as shown in Table 5. The results show that the attack method of the present application has a certain fooling rate for both general network structures and lightweight networks, among which ShuffleNetV2 has the highest fooling rate and ResNet-50 has the lowest. For different attack targets, the fooling rate of sedan targets is the highest, reaching an average of 99.02%, and the off-road vehicle target has the lowest average fooling rate of 77.14%. This shows that the attack method of the present application is effective in fooling the SAR target automatic recognition network. The present application provides some examples of generated adversarial samples, and after adding adversarial perturbations, they are all misclassified into other categories. The attack method of the present application can deceive the classifier with a high degree of confidence on the basis of having concealment that is difficult for the human eye to recognize.

[0098] Table 5 The fooling rates of digital attacks with different targets under 6 classification networks and their average fooling rates.

[0099]

[0100] The adversarial samples generated in this application are generated on the ResNet-50 network.

[0101] The adversarial perturbations generated in the digital domain are transferred to the physical domain, and the fooling rates of the corresponding adversarial samples are statistically analyzed, as shown in Table 6. The fooling rate of the generated adversarial perturbations in the physical world is somewhat lower than that in the digital world. The highest fooling rate is still ShuffleNetV2, and the lowest is VGG-16. For different attack targets, the average fooling rate of the sedan target is still the highest, at 97.87%, and the lowest average fooling rate is 67.02% for the off-road vehicle target. A schematic diagram of the generated adversarial perturbations on the target is provided, as shown in Figure 5 As shown, the triangular faces with adversarial scattering coefficients are displayed in light colors. The figure includes three 3D models, among which the triangular faces with adversarial scattering coefficients are marked in light colors, and the number of triangular face elements is marked below the image. It can be seen from the figure that the triangular face elements after perturbation have different area sizes. We found that in the physical world, the factors that affect the fooling rate are related to the number and area size of the triangular face elements of the 3D model. For example, in the off-road vehicle target, the number of triangular face elements is the least, and the attack of this application shows the lowest average fooling rate in this target. In the van target, in Figure 5 It can be seen from the figure that the area of ​​the triangular face after the disturbance is large, and the average fooling rate of the attack of the present application in this target has dropped the most. In the car target, the number of triangular face elements is the largest, and the area is relatively small, and the average fooling rate of the attack of the present application in this target is the highest.

[0102] Table 6 Physical attack fooling rates of different targets under 6 classification networks and their average fooling rates, and their change levels are marked in the table.

[0103]

[0104] In this section, we first tested the transferability of digital attacks between different network architectures, as shown in Table 7, by generating adversarial samples on the proxy model and using them to attack the black-box target model. Among them, the black-box attack has an average fooling rate of 80.11% on ResNet-50, while the adversarial samples made based on VGG-16 have an average fooling rate of 77.5% on the other five black-box models. As can be seen from the table, the transferability of adversarial attacks between different networks varies, which means that their robustness against black-box adversarial attacks is different. For example, the VGG-16 network and the ShuffleNetV2 network have low fooling rates against attacks from all sources, indicating that they have high robustness against black-box adversarial attacks. In addition, we also found that network characteristics affect transferability, and different network architectures have different sensitivities to attacks. For example, ResNet-50 as a source network has an attack fooling rate of 72.73% on SqueezeNet, while the fooling rate on ShuffleNetV2 is only 3.79%. In summary, our results show that the threat of physical adversarial attacks still exists even in the face of black-box network architectures.

[0105] Table 7 The transferability of digital domain adversarial samples generated under 6 different classifier networks between 6 different network architectures, and their fooling rates are marked in the table.

[0106]

[0107]

[0108] At the same time, the transferability of physical attacks was also tested, as shown in Table 8. The results show that the transferability of physical attacks is similar to that of digital attacks. Since physical attacks are essentially the mapping of digital attacks in the physical world, the results obtained are in line with expectations.

[0109] Table 8 shows the transferability of physical domain adversarial samples generated under 6 different classifier networks between 6 different network architectures, and their fooling rates are marked in the table.

[0110]

[0111] Since under realistic conditions, changes in the pitch angle of SAR satellite imaging will affect changes in viewing angle during imaging, in order to simulate the impact of such changes on attacks, the pitch angle of imaging is changed by changing the positions of the camera and the ground objects in the simulated scene, thereby proving the robustness of the adversarial perturbation of the present application at different pitch angles. The changes in the fooling rate at different pitch angles with an interval of 2 degrees from 19 to 27 degrees were tested under the ResNet-50 network. As shown in Table 9, the results show that the changes in the fooling rates of the three targets at different pitch angles are not obvious, among which the average fooling rates of sedan targets and van targets at different pitch angles are higher, and the average fooling rate of off-road vehicle targets is the lowest. This shows that the attack method of the present application is robust at different pitch angles.

[0112] Table 9 The fooling rate of adversarial samples generated at different pitch angles under the ResNet-50 network.

[0113]

[0114]

[0115] The beneficial effects of this application are as follows:

[0116] This application proposes a new physical adversarial attack method SAR-PAA for SAR ATR, which can migrate adversarial perturbations that are difficult for the human eye to recognize in the digital space to the physical space and cause misclassification of SAR ATR. The conversion from the physical world to the digital world is realized through a SAR simulation renderer RaySAR, and the digital perturbation is updated through back propagation by designing an effective loss function. Finally, the final digital perturbation is mapped back to the material information of the 3D model through a mapping model to achieve the final physical perturbation. Experimental results show that this application can effectively implement SAR adversarial attacks in the physical world, achieving an average fooling rate of up to 99.02% in the digital world and an average fooling rate of up to 97.87% in the physical world. Compared with existing methods, this application bridges the gap between digital attacks and physical attacks and has the possibility of being realized in the real world.

[0117] As used herein, the word "preferred" is intended to be used as an example, instance, or illustration. Any aspect or design described herein as "preferred" is not necessarily to be construed as being more advantageous than other aspects or designs. On the contrary, the use of the word "preferred" is intended to present concepts in a specific way. The term "or" as used in this application is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless otherwise specified or clear from the context, "X uses A or B" means any one of the naturally included permutations. That is, if X uses A; X uses B; or X uses both A and B, then "X uses A or B" is satisfied in any of the foregoing examples.

[0118] Moreover, although the present disclosure has been shown and described with respect to one or implementations, those skilled in the art will think of equivalent variations and modifications based on the reading and understanding of this specification and the accompanying drawings. The present disclosure includes all such modifications and variations, and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the above-mentioned components (such as elements, etc.), the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (such as it is functionally equivalent), even if the structure is not equivalent to the disclosed structure of the function in the exemplary implementation of the present disclosure shown herein. In addition, although the specific features of the present disclosure have been disclosed with respect to only one of several implementations, such features can be combined with one or other features of other implementations that may be desired and advantageous for a given or specific application. Moreover, insofar as the terms "including", "having", "containing" or their variations are used in specific embodiments or claims, such terms are intended to be included in a manner similar to the term "comprising".

[0119] The functional units in the embodiments of the present invention may be integrated into a processing module, or each unit may exist physically separately, or multiple or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc. The above-mentioned devices or systems may execute the storage method in the corresponding method embodiment.

[0120] To sum up, the above embodiment is an implementation mode of the present invention, but the implementation mode of the present invention is not limited by the embodiment. Any other changes, modifications, substitutions, combinations, and simplifications that deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A camouflage image generation method for SAR images, characterized in that: The following steps are involved: The process of generating SAR images from 3D models is described as: C is the SAR geometric output data, M is the grid information, T is the material information, θ c is the imaging parameter, R is the ray tracing renderer; use Initialize C, is the final adversarial SAR geometric output data; Start iteration. In each iteration, perform the following steps: Update the mask vector and Post-processing of C Get a two-dimensional SAR image I; Input image I into the SAR automatic target recognition classification network Get predicted labels Calculate the loss function; Update using the gradient backpropagation algorithm After the iteration, according to the mapping model Generate the final camouflaged image.

2. The camouflage image generation method for SAR images according to claim 1, characterized in that: The grid information includes vertex coordinates and triangular face information, the material information includes reflection coefficient, scattering coefficient and surface roughness; the imaging parameter θ c Including radar signal source coordinates, receiving antenna coordinates and pointing position; the ray tracing renderer An approximate physical optics model is used to simulate the specular reflection and diffuse reflection of the radar signal, and the SAR geometric output data C is recorded, which includes the reflection direction and ray path of the echo position in the image.

3. The method for generating a camouflaged image for SAR images according to claim 2, characterized in that: The process of obtaining the two-dimensional SAR image I is expressed as Indicates that, where θ p are post-processing parameters, including the maximum and minimum pitch angles, the maximum and minimum range angles, the spacing, and the maximum number of ray ejections. H and W are the height and width of the image, respectively.

4. The method for generating a camouflaged image for a SAR image according to claim 3, characterized in that: The perturbation to C is defined as digital camouflage, and the perturbation generation is regarded as an optimization problem, whose objective function is as follows: In the formula, C adv As the adversarial SAR geometry output data is optimized in iterations, is the final adversarial SAR geometric output data, is the loss function, y gt is the true label.

5. The method for generating a camouflaged image for a SAR image according to claim 4, characterized in that: The loss function is as follows: Where ω is the parameter to be optimized, x is the intensity value in the SAR geometric output data, c is a constant used to balance the classification loss and limit the perturbation intensity loss, and H is the cross entropy loss function.

6. The method for generating a camouflaged image for a SAR image according to claim 5, characterized in that: In order to generate effective two-dimensional SAR images, the range of intensity values ​​is limited to [0,1]. For the perturbation vector δ, the goal is to make 0≤x i +δ i ≤1,x i is a single intensity value in the SAR geometric output data, δ i is the perturbation value applied to a single intensity value. Since the range of the sigmoid function is (0,1), let δ i =sigmoid(ω i )―x i , so that 0≤sigmoid(ω i )=x i +δ i ≤1; in the initialization parameter ω i When , use the inverse function of the sigmoid function To ensure the similarity between the perturbation and the original intensity value vector; use the L2 norm to limit the perturbation vector δ, that is, ∥δ∥2 is minimized.

7. The method for generating a camouflaged image for a SAR image according to claim 6, characterized in that: The sparse attack strategy is adopted to limit the range of the disturbance scattering points, that is, to limit the number of disturbance scattering points, including: Initialize the mask vector, which has the same shape as the perturbation vector and all its elements are 1; multiply the generated perturbation by the mask vector element by element; arrange the elements in the perturbation vector in order of size, and if the element is less than a certain threshold, set the element with the corresponding index in the mask vector to 0, that is, eliminate the perturbations in the perturbation vector that are less than a certain threshold, so that the 0 elements in the mask vector continue to increase during iteration.

8. The method for generating a camouflaged image for SAR images according to claim 7, characterized in that: The mapping model Determine 3 transformation relationships: Coordinate transformation: from the coordinate system of the scattering points in C to the coordinate system of the 3D model; Points correspond to triangular surfaces; From intensity values ​​to material parameters: Changes in the intensity values ​​in the SAR geometry output data correspond to changes in the material parameters in the 3D model.

9. The method for generating a camouflaged image for SAR images according to claim 8, characterized in that: For coordinate transformation, the ray tracing renderer Record the coordinates of each scattering point in the coordinate system of the 3D model, so as to realize the conversion from the scattering point coordinate system to the coordinate system of the 3D model; For the points corresponding to the triangular surface element, the scattering point with the largest change in intensity value before and after the disturbance is selected as the benchmark to calculate the material parameters of the triangular surface element; For the intensity value to material parameters, a single variable optimizer is used to perform bounded optimization on the scattering coefficient of the triangle face corresponding to each scattering point; specifically, a function is defined that inputs the scattering coefficient and uses the same imaging parameters as before through a ray tracing renderer. The absolute value of the difference between the output intensity value and the target intensity value is obtained, and the Brent algorithm is used to find the local minimum of this function in the interval [0,1] to obtain the scattering coefficient corresponding to the target intensity value on the triangular face.

Images