A method for generating a camouflage image for a SAR image

Abstract

The application discloses a camouflage image generation method for SAR images, and comprises the following steps: describing the process of generating a SAR image from a 3D model as: initializing C; starting iteration: updating a mask vector and post-processing C to obtain a two-dimensional SAR image; inputting the image into an automatic target recognition classification network to obtain a predicted label; calculating a loss function; and after updating iteration using a gradient back propagation algorithm, generating a final camouflage image according to a mapping model. The application first proposes a camouflage image generation method based on a SAR imaging mechanism, can generate a camouflage image conforming to the characteristics of SAR imaging, and realizes a physical adversarial attack pipeline based on RaySAR, and test proves the effectiveness of the camouflage.

Description

Technical Field

[0001] The present invention belongs to the technical field of SAR images, and in particular relates to a camouflage image generation method for SAR images. Background Art

[0002] Deep convolutional neural networks (DCNNs) have been widely used in remote sensing image recognition, including target recognition in optical remote sensing images and SAR automatic target recognition (SAR ATR). However, the use of camouflage significantly limits SAR ATR's applications. These subtle perturbations, carefully designed to be difficult for the human eye to detect, can mislead DCNNs into producing erroneous predictions. This camouflage problem has garnered widespread attention from both academia and industry, becoming a key focus in the field of AI security.

[0003] Currently, numerous camouflage techniques have been implemented in the field of remote sensing. Researchers in the field of SAR ATR are also conducting research on camouflage attacks, most of which are digital attacks: perturbations are directly added to image pixels, with the perturbations occurring in the digital space. For example, FGSM and BIM are used to attack SAR ATR classifiers, or universal adversarial perturbations are used to deceive SAR ATR classifiers. The perturbations generated by these attack methods are concealed, making them difficult for observers to detect, posing a security threat to the widespread application of SAR ATR.

[0004] However, most of these digital attack methods are based on the principle of optical image attack, without considering the difference between the imaging mechanism of SAR and optical images. The pixel information of SAR images is actually backscattered information related to factors such as radar system parameters and surface characteristics of objects, while the imaging of visible light is mainly affected by the color and surface characteristics of the object. Different colors and materials absorb and reflect light of different wavelengths. Therefore, the images generated based on the principle of optical image attack may not conform to the imaging characteristics of SAR. At the same time, considering the possibility of real-world implementation of countermeasure attacks, this application focuses on physical attacks: adding disturbances to the real-world environment or physical simulators, and the disturbances are in physical space. Compared with digital attacks, physical attacks pose a greater potential threat to society. In addition, we believe that the camouflaged images generated by simulators based on the SAR imaging mechanism are more consistent with the imaging characteristics of SAR, which will make the attack more covert.

[0005] Physical adversarial attacks have garnered widespread attention in the field of optical image recognition, such as patch-based and camouflage-based attacks. These methods essentially achieve their attacks by modifying the surface pattern of an object. However, these methods cannot be directly transferred to the SAR field due to differences in imaging mechanisms. SAR images essentially record backscattered information from ground objects. Therefore, simply modifying the surface pattern of an object will not change the pixel values ​​in the SAR image. Instead, the surface material properties of the object, such as the scattering coefficient and surface roughness, must be modified to alter the radar echo signal and, in turn, the pixel values ​​in the SAR image.

[0006] Furthermore, to simulate the SAR imaging process, an end-to-end SAR simulation renderer should be used. In previous research, the ray-tracing-based SAR simulation program RaySAR has been widely used as a SAR imaging simulator, for example in the analysis, design, and verification of SAR systems, assisting in SAR image interpretation, and validating SAR image processing algorithms. However, since RaySAR is a non-differentiable system, physical adversarial perturbations cannot be directly obtained through optimization. Therefore, a physical adversarial attack pipeline based on RaySAR must be proposed. Summary of the Invention

[0007] To solve the above problems, the present application proposes a pipeline for SAR physical adversarial attack (SAR-PAA). First, in order to connect the digital world and the physical world, the camouflage of the present application includes a digital part and a physical part. The digital camouflage of the present application acts on the SAR geometric output data, that is, the scattering points and scattering intensity information generated by the ray tracer. The physical camouflage of the present application acts on the material parameters of the 3D object and uses RaySAR to generate a two-dimensional SAR image. Then, by designing an effective loss function, the camouflage is updated through back propagation. Finally, the final camouflage is mapped back to the material information of the 3D model through a mapping model, thereby realizing the update of the physical camouflage.

[0008] To achieve the above objectives, the present application discloses a camouflage image generation method for SAR images, the method comprising the following steps:

[0009] The process of generating SAR images from 3D models is described as: C is the SAR geometric output data, M is the grid information, T is the material information, θ c is the imaging parameter, R is the ray tracing renderer;

[0010] use Initialize C, is the final adversarial SAR geometric output data;

[0011] Start iteration. In each iteration, perform the following steps:

[0012] Update the mask vector and

[0013] Post-processing of C Get a two-dimensional SAR image I;

[0014] Input image I into the SAR automatic target recognition classification network Get predicted labels

[0015] Calculate the loss function;

[0016] Update using the gradient backpropagation algorithm

[0017] After the iteration, according to the mapping model Generate the final camouflaged image.

[0018] Preferably, the grid information includes vertex coordinates and triangular facet information, the material information includes reflection coefficient, scattering coefficient and surface roughness; the imaging parameter θ c Including radar signal source coordinates, receiving antenna coordinates and pointing position; the ray tracing renderer An approximate physical optics model is used to simulate the specular and diffuse reflections of radar signals, and the SAR geometric output data C is recorded, which includes the reflection direction and ray path of the echo position in the image.

[0019] Preferably, the process of obtaining the two-dimensional SAR image I is performed using the formula Indicates that, where θ p are post-processing parameters, including the maximum and minimum pitch angles, the maximum and minimum range angles, the spacing, and the maximum number of ray ejections. H and W are the height and width of the image, respectively.

[0020] Preferably, the perturbation to C is defined as digital camouflage, and the perturbation generation is regarded as an optimization problem with the following objective function:

[0021]

[0022] Where C adv As the adversarial SAR geometric output data is optimized in iterations, is the final adversarial SAR geometric output data, is the loss function, y gt is the true label.

[0023] Preferably, the loss function is as follows:

[0024]

[0025] where ω is the parameter to be optimized, x is the intensity value in the SAR geometry output data, c is a constant to balance the classification loss and the restriction perturbation intensity loss, H is the cross-entropy loss function.

[0026] Preferably, in order to generate valid 2D SAR images, the range of intensity values is limited between [0, 1], and the goal is to make 0 ≤ x i + δ i ≤ 1, x i is a single intensity value in the SAR geometry output data, δ i is the perturbation value imposed on the single intensity value, since the value range of the sigmoid function is (0, 1), let δ i = sigmoid(ω i )―x i , so that 0 ≤ sigmoid(ω i ) = x i + δ i ≤ 1; when initializing the parameter ω i , the inverse function of the sigmoid function is used to ensure the similarity of the perturbation and the original intensity value vector; the L2 norm is used to restrict the perturbation vector δ, i.e., to make ||δ||2minimum.

[0027] Preferably, a sparse attack strategy is adopted to limit the range of perturbed scattering points, i.e., to limit the number of perturbed scattering points, which specifically includes:

[0028] Initialize a mask vector, which has the same shape as the perturbation vector, and all elements are 1; the generated perturbation is multiplied element by element with the mask vector; arrange the elements in the perturbation vector in order of size, and if the element is less than a certain threshold, set the corresponding element in the mask vector to 0, i.e., eliminate the perturbation in the perturbation vector that is less than a certain threshold, so that the 0 elements in the mask vector increase in the iteration.

[0029] Preferably, the mapping model determines 3 transformation relationships:

[0030] Coordinate conversion: from the coordinate system of the scattering points in C to the coordinate system of the 3D model;

[0031] Point correspondence to triangular face element;

[0032] From intensity value to material parameter: change the intensity value in the SAR geometry output data to change the material parameter in the 3D model.

[0033] Preferably, for coordinate conversion, the light ray tracing renderer Record the coordinates of each scattering point in the 3D model's coordinate system to achieve the conversion from the scattering point coordinate system to the 3D model's coordinate system; for the points corresponding to the triangular surface element, select the scattering point with the largest change in intensity value before and after the disturbance as the benchmark to calculate the material parameters of the triangular surface element;

[0034] For the transition from intensity values ​​to material parameters, a single variable optimizer is used to perform bounded optimization on the scattering coefficient of the triangle face corresponding to each scattering point; specifically, a function is defined that inputs the scattering coefficient and uses the same imaging parameters as before to render the scattering coefficient through a ray tracing renderer. The absolute value of the difference between the output intensity value and the target intensity value is obtained, and the Brent algorithm is used to find the local minimum of this function in the interval [0,1] to obtain the scattering coefficient corresponding to the target intensity value on the triangle surface.

[0035] The beneficial effects of this application are as follows:

[0036] For the first time, a new SAR ATR camouflage image generation method based on SAR imaging mechanism is proposed, which can generate camouflage images that conform to SAR imaging characteristics.

[0037] A physical adversarial attack pipeline based on RaySAR was implemented and tested in the digital and physical worlds using three different 3D models, demonstrating the effectiveness of camouflage.

[0038] This application is an end-to-end physical camouflage that is feasible in the real world and has certain mobility in different network structures. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A framework diagram of the present invention;

[0040] Figure 2 Schematic diagram of mapping scattering points to triangular surface elements;

[0041] Figure 3 Renderings of the 3D models used;

[0042] Figure 4 Comparison of the SAR camouflage image generated by the embodiment of the present application with the real SAR image of the MSTAR dataset. The upper part is the real SAR image, and the lower part is the SAR camouflage image.

[0043] Figure 5 Schematic diagram of the position of the physical adversarial perturbation generated by an embodiment of the present application on a 3D model. DETAILED DESCRIPTION

[0044] The present invention will be further described below with reference to the accompanying drawings, but the present invention is not limited in any way. Any changes or substitutions made based on the teachings of the present invention fall within the scope of protection of the present invention.

[0045] Artificial intelligence (AI) refers to the theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also involves studying the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0046] Before introducing the embodiments of the present application, some terms involved in the present application are explained.

[0047] 1. SAR Auto Target Recognition researchers tend to design specialized CNNs. Xu et al. proposed a deep CNN model, SARnet, for SAR ATR. This model features two convolutional-pooling layers and two fully connected layers, achieving an accuracy of 95.68% on the MSTAR dataset. Zhai et al. proposed MF-SarNet for SAR ATR. The Fire module in this network is used to extract features and achieve fewer parameters. MF-SarNet consists of 18 convolutional layers, 2 fully connected layers, and 8 Fire modules. Using clockwise rotation-based data augmentation to expand the dataset 360 times, it achieved an accuracy of 98.53% on MSTAR. Shang et al. proposed M-Net to address the overfitting problem caused by insufficient samples. M-Net uses information recorders to store spatial features and leverages spatial similarity to predict labels for unknown samples.

[0048] 2. Adversarial Attacks on SAR ATR Attack methods designed for SAR ATR usually take into account the special mechanisms of SAR imaging and are highly relevant to SAR imaging.

[0049] 3. SAR Simulation RaySAR is a ray-tracing-based SAR simulator developed by Auer et al. It is an extension of the open-source ray-tracing rendering software POV-Ray. It is designed to simulate radar signals in 3D, including azimuth, range, and elevation. RaySAR uses an approximate physical optics model to simulate the specular and diffuse reflection components of radar signals. RaySAR uses ray tracing to record the reflection direction and component. The ray path determines the position of the echo in the image.

[0050] The technical solutions provided in the embodiments of this application involve technologies such as machine learning and natural language processing of artificial intelligence, and are specifically introduced and explained through the following embodiments.

[0051] First, this application defines the forward process of generating SAR images from a 3D model as follows: given a 3D model, it has grid information M and material information T. The grid information includes vertex coordinates and triangular face information, and the material information includes reflection coefficient, scattering coefficient, and surface roughness. Imaging parameter θ is c Including the radar signal source coordinates, receiving antenna coordinates and pointing position, etc. Using a ray tracing renderer R, an approximate physical optics model is used to simulate the specular reflection and diffuse reflection of the radar signal, and record the reflection direction and ray path C that determine the echo position in the image. This process can be expressed by the formula In order to generate a two-dimensional SAR image I, C needs to be post-processed. This process can be expressed as Indicates that, where θ p It is the post-processing parameter, including the maximum / minimum value of the pitch angle, the maximum / minimum value and spacing of the range direction, the maximum number of ray ejections, etc. Input the image I into the SAR automatic target recognition and classification network And get the predicted label

[0052] The camouflage method of this application will be implemented in digital space and physical space in turn. First, in the digital space, adversarial perturbations are generated for the SAR geometric output data C, and then the perturbations are further mapped to the material information T in the physical space. Therefore, this application defines the perturbation to C as a digital attack and regards the perturbation generation as an optimization problem, whose objective function is as follows:

[0053]

[0054] Where, is the final adversarial SAR geometric output data, is the loss function, y gt is the true label. By solving the optimization problem described in Formula 1, the adversarial SAR geometric output data is obtained.

[0055] After the perturbation of C is realized, further, through a mapping Mapped to the physical space, we define this process as a physical attack, which can be expressed as

[0056]

[0057] Where, is the final adversarial physical material information, through this mapping we can get the final physical adversarial perturbation. The notations used in this paper are summarized in Table 1.

[0058] Table 1 Notations

[0059]

[0060] To generate SAR physical adversarial perturbation, this paper proposes a framework of SAR physical adversarial attack, which contains a differentiable SAR geometry output data post-processor, the framework is shown as Figure 1

[0061] Figure 1 In the paper, first, we generate adversarial perturbation in SAR geometry output data using optimization method, update the perturbation by backpropagation using the designed loss function, and finally map this perturbation to the material information of the 3D model through a mapping model.

[0062] This paper calls the process of generating perturbation in SAR geometry output data as digital camouflage, because the perturbation is still in the digital space. In this part, we use optimization method to generate adversarial perturbation by computing loss and gradient backpropagation, therefore, the loss function is very important in this process, which determines the final attack effect, the design of loss function will be discussed in detail later.

[0063] Correspondingly, this paper calls the process of mapping the perturbation generated in SAR geometry output data to the 3D model as physical camouflage, because this step is actually mapping the perturbation in digital space to physical space. In this part, we will discuss in detail the design of mapping model and how to realize this mapping.

[0064] In the digital camouflage part, we use optimization method to generate adversarial perturbation in SAR geometry output data C, when designing the loss function, we mainly consider the following 3 factors:

[0065] ​(1) Objective function. Our strategy is to use an untargeted attack method to automatically find the wrong target label. Our optimization goal is to make the predicted value of the adversarial sample approach the target label continuously, and eventually cause the model to produce an incorrect classification. Specifically, we use the model's classification loss function, namely the cross entropy loss, as the objective function, and select the label with the highest predicted probability other than the correct label as the target label to minimize the loss function and find the adversarial sample. However, using only this loss function cannot make the adversarial sample effective, because there is no limit on the scope of perturbation optimization, which may make the adversarial sample inconsistent with the SAR imaging principle or violate certain characteristics of SAR images.

[0066] (2) Intensity value range restriction. Since the optimization target is the intensity value contributed by the scattering points in the SAR geometric output data, in order to generate an effective two-dimensional SAR image, the intensity value range needs to be restricted to [0, 1]. Let the perturbation vector be δ, and the goal is to make 0≤x i +δ i ≤1. Since the range of the sigmoid function is (0,1), let δ i =sigmoid(ω i )―x i , so that 0≤sigmoid(ω i )=x i +δ i ≤1. Therefore, the sigmoid function is used to limit the range of optimization parameters to ensure that the generated disturbance is reasonable. i When , use the inverse function of the sigmoid function To ensure the similarity of the perturbation to the original intensity value vector.

[0067] (3) Perturbation amplitude limit. In order to increase the authenticity of the generated adversarial samples, and because we need to map the perturbation of the intensity value to the 3D model material parameters in the subsequent physical attack, the smaller the perturbation amplitude, the more conducive it is to the subsequent scattering coefficient matching process. The L2 norm is used to limit the perturbation vector δ, that is, to minimize ||δ||2. The final loss function is as follows

[0068]

[0069] Where ω is the parameter to be optimized, x is the intensity value in the SAR geometric output data, c is a constant that can be used to balance the classification loss and limit the perturbation intensity loss, and H is the cross entropy loss function.

[0070] In addition, since the material information of the triangular facets of the 3D model corresponding to each scattering point needs to be modified in the next step of physical camouflage, this application adopts a sparse attack strategy to limit the range of the perturbed scattering points, that is, to limit the number of perturbed scattering points. The iterative algorithm in the L0 attack proposed by Carlini and Wagner is used to implement the sparse attack. In each iteration, some pixels that have little effect on the classifier output are fixed until a minimum subset of perturbed pixels is determined. Specifically, first, a mask vector is initialized, which has the same shape as the perturbation vector and all its elements are 1; the generated perturbation is element-wise multiplied (Hadamard product) with the mask vector, so to limit the scope of the perturbation, it is only necessary to limit the number of 1 elements in the mask vector; the elements in the perturbation vector are arranged in order of size, and if the element is less than a certain threshold, the element with the corresponding index in the mask vector is set to 0, that is, the perturbation in the perturbation vector that is less than a certain threshold is eliminated, and the 0 elements in the mask vector are allowed to increase continuously during the iteration. Mapping adversarial perturbation from Digital domain to Physical domain (MaPP)

[0071] In order to map the disturbance of SAR geometric output data C in digital space to physical space, a mapping model is defined This process is used to generate the final adversarial material information and is called physical camouflage.

[0072] In this mapping model, three transformation relationships are determined:

[0073] 1. Coordinate transformation: from the coordinate system of the scattered points in C to the coordinate system of the 3D model

[0074] 2. Point correspondence to triangular surface element: The basic unit of intensity value in C is the scattering point, while the basic unit of 3D model material parameter is the triangular surface element, so a corresponding relationship needs to be established.

[0075] 3. From intensity values ​​to material parameters: Changes in the intensity values ​​in the SAR geometric output data correspond to changes in the material parameters in the 3D model

[0076] For coordinate conversion, the ray tracing renderer R can be used to record the coordinates of each scattering point in the 3D model's coordinate system, thereby achieving the conversion from the scattering point coordinate system to the 3D model's coordinate system. For point correspondence to triangular surfaces, since the areas of the triangular surfaces that make up the 3D model are inconsistent, our experiments show that triangular surfaces with smaller areas may correspond to one scattering point, while triangular surfaces with larger areas may correspond to multiple scattering points. Our goal is to make the basic unit of 3D model material parameters—triangular surfaces—correspond one-to-one with the basic unit of intensity value in C—scattering points. This way, we can simulate the change in scattering point intensity value by changing the material parameters of the triangle surface. Therefore, considering that multiple scattering points have different effects on the same triangle surface, we select the scattering point with the largest change in intensity value before and after the perturbation as the benchmark to calculate the material parameters of the triangle surface.

[0077] Figure 2 When mapping scattering points to triangular surfaces, since triangular surfaces have different sizes, a triangular surface with a larger area may correspond to multiple scattering points. However, this application can only select one scattering point as a reference to calculate the material parameters of the triangular surface.

[0078] For intensity values ​​to material parameters, due to the ray tracing renderer The calculation of the intensity value needs to consider many factors, such as the angle between the triangle surface and the light source, so it is difficult to directly obtain the material parameters of the corresponding triangle surface through a certain intensity value. In addition, since the radar signal is usually scattered on the surface of the object rather than reflected by a mirror in SAR imaging, the influence of the scattering coefficient on the intensity value is dominant among all material parameters. We use a single variable optimizer to perform bounded optimization on the scattering coefficient of the triangle surface corresponding to each scattering point. Specifically, we define a function that inputs the scattering coefficient and uses the same imaging parameters as before through a ray tracing renderer The absolute value of the difference between the output intensity value and the target intensity value is obtained, and the Brent algorithm is used to find the local minimum of this function in the interval [0,1] to obtain the scattering coefficient corresponding to the target intensity value on the triangle surface.

[0079] The pseudo code of the attack method can be expressed as Algorithm 1.

[0080]

[0081] Currently, there are no physical datasets (i.e., datasets containing various scene information and 3D models) for SAR simulation research. Therefore, we constructed a new dataset by expanding the MSTAR dataset, which will be described in detail in the next section. Due to the differences in datasets and the fact that the attack methods used in this application differ from existing attack methods, our adversarial perturbations, whether digital or physical, are not directly applied to the image space, so we did not select a baseline for our experiments.

[0082] To this end, the experiment will primarily verify the feasibility of our proposed attack method, rather than its effectiveness. First, we will verify the effectiveness of the proposed digital and physical attacks, then verify their portability across different network architectures. Finally, we will examine the impact of different pitch angles on the attack effectiveness.

[0083] In the experiment, we used 3D models of three different vehicles: sedan, SUV and van. The renderings of these 3D models are shown in Figure 2. Figure 3 The number of vertices and triangles is shown in Table 2. We used these 3D models to generate 2D SAR images and used them to augment the MSTAR dataset, creating a classification dataset with 13 categories. Specifically, we used parameters similar to those used in the MSTAR dataset, generating SAR images of 172x172 pixels at the same pitch angle and 360 degrees with a pitch of 1 degree. The resulting dataset contains 13 categories: 2S1, BMP2, BRDM_2, BTR60, D7, SLICY, T62, T72, ZIL131, ZSU_23_4, CAR, JEEP, and VAN. The number of images in each category ranges from 428 to 720, as shown in Table 3.

[0084] Table 2. Comparison of 3D models and their number of vertices and triangles.

[0085] 3D Model CAR JEEP VAN vertices 116929 50651 76214 triangles 207830 93927 139097

[0086] Table 3 shows all categories and details of the training and test sets after the expansion of the MSTAR dataset.

[0087] Class Training set Test set 2S1 233 274 BMP2 233 195 BRDM_2 233 274 BTR60 233 195 D7 233 274 SLICY 233 274 T62 233 273 T72 232 196 ZIL131 233 274 ZSU_23_4 233 274 CAR 360 360 JEEP 360 360 VAN 360 360

[0088] Figure 4 Comparison of the SAR simulation image generated for this application with the real SAR image of the MSTAR dataset. The upper part is the real SAR image, and the lower part is the SAR simulation image.

[0089] This application uses the fooling rate as an indicator to evaluate the attack effect. The fooling rate is defined as follows:

[0090]

[0091] Among them, N total is the total number of samples participating in the attack, F is the classifier network, x adv is the generated adversarial sample, y gt are their true labels. The fooling rate is the ratio of the number of samples misclassified by the classifier network to the total number of samples participating in the attack.

[0092] Six different object classification and recognition networks were selected for testing on the disguised images generated in this application. These included three mainstream and three lightweight object classification and recognition networks: ResNet-50, VGG-16, DenseNet-121, MobileNet V2, SqueezeNet, and ShuffleNet V2. All networks used the official PyTorch implementation. These classifier networks were trained using the ImageNet pre-trained model with the following hyperparameter settings: SGD with momentum, a learning rate of 0.001, a momentum of 0.9, a batch size of 32, and a maximum epoch of 80. The classification accuracy of the trained classifier networks is shown in Table 4.

[0093] Table 4 Accuracy of all trained classifier networks on the test set.

[0094]

[0095] For the adversarial attack method proposed in this application, the hyperparameters are set as follows: the optimizer uses Adam, the learning rate is 0.05, the coefficient is 1, and the maximum number of iterations is 50. The experiments are conducted on a workstation equipped with an NVIDIA GTX 1080Ti 11GB GPU.

[0096] We conducted attack experiments on the test set and ensured that the images were correctly classified before the attack. The results are divided into two parts: the attack effects in the digital domain and their corresponding attack effects in the physical domain.

[0097] In the digital field, the fooling rates of adversarial samples generated for 3 targets on 6 networks were statistically analyzed, as shown in Table 5. The results show that the attack method of the present application has a certain fooling rate for both general network structures and lightweight networks, among which ShuffleNetV2 has the highest fooling rate and ResNet-50 has the lowest. For different attack targets, the fooling rate of sedan targets is the highest, reaching an average of 99.02%, and the off-road vehicle target has the lowest average fooling rate, reaching 77.14%. This shows that the attack method of the present application is effective in fooling SAR target automatic recognition networks. This application provides some examples of generated adversarial samples, and after adding adversarial perturbations, they are all misclassified into other categories. The attack method of the present application can deceive the classifier with a high degree of confidence on the basis of having a concealment that is difficult for the human eye to recognize.

[0098] Table 5 The fooling rates of digital attacks with different targets under 6 classification networks and their average fooling rates.

[0099]

[0100] The adversarial samples generated in this application are generated on the ResNet-50 network.

[0101] We transferred the adversarial perturbations generated in the digital domain to the physical domain, and calculated the fooling rates of the corresponding adversarial examples, as shown in Table 6. The fooling rates of the generated adversarial perturbations in the physical world decreased somewhat compared to those in the digital world. ShuffleNetV2 still had the highest fooling rate, while VGG-16 had the lowest. Across different attack targets, sedans still had the highest average fooling rate, at 97.87%, while SUVs had the lowest average fooling rate, at 67.02%. A schematic diagram of the generated adversarial perturbations on the targets is provided, as shown in the figure below. Figure 5 As shown, the triangular faces with adversarial scattering coefficients are displayed in light colors. The figure includes three 3D models, among which the triangular faces with adversarial scattering coefficients are marked in light colors, and the number of triangular face elements is marked below the image. It can be seen from the figure that the triangular face elements after perturbation have different area sizes. We found that in the physical world, the factors affecting the fooling rate are related to the number and area size of the triangular face elements of the 3D model. For example, in the off-road vehicle target, the number of triangular face elements is the least, and the attack of this application shows the lowest average fooling rate in this target. In the van target, Figure 5 As can be seen in the figure, the perturbed triangles have a larger area, and the attack presented in this application shows the largest decrease in average fooling rate for this target. In contrast, the sedan target has the largest number of triangles and a relatively small area, and the attack presented in this application shows the highest average fooling rate for this target.

[0102] Table 6 The fooling rates of physical attacks on different targets under 6 classification networks and their average fooling rates, and their variation levels are marked in the table.

[0103]

[0104] In this part, we first test the transferability of digital attacks between different network architectures, as shown in Table 7, by generating adversarial samples on the proxy model and using them to attack the black-box target model. Among them, the average fooling rate of black-box attack on ResNet-50 is 80.11%, while the average fooling rate of adversarial samples based on VGG-16 on the other five black-box models is 77.5%. From the table, we can see that the transferability of adversarial attacks between different networks is different, which means that their robustness to black-box adversarial attacks is different. For example, the fooling rates of VGG-16 network and ShuffleNetV2 network to all source attacks are low, which indicates that they have high robustness to black-box adversarial attacks. In addition, we also found that network characteristics affect transferability, and different network architectures have different sensitivity to attacks. For example, the attack fooling rate of ResNet-50 as the source network on SqueezeNet reached 72.73%, while the fooling rate on ShuffleNetV2 was only 3.79%. In summary, our results show that there is still a threat of physical adversarial attacks even when facing black-box network architectures.

[0105] Table 7 Transferability of digital domain adversarial samples generated under 6 different classifier networks between 6 different network architectures, respectively, and their fooling rates are marked in the table.

[0106]

[0107]

[0108] At the same time, the transferability of physical attacks is also tested, as shown in Table 8. The results show that the transferability of physical attacks is similar to that of digital attacks. Since physical attacks are essentially a mapping of digital attacks in the physical world, the results obtained are as expected.

[0109] Table 8 Transferability of physical domain adversarial samples generated under 6 different classifier networks between 6 different network architectures, respectively, and their fooling rates are marked in the table.

[0110]

[0111] Since under real conditions, the change in the pitch angle of SAR satellite imaging will affect the change in the viewing angle during imaging, in order to simulate the impact of this change on the attack, the imaging pitch angle is changed by changing the position of the camera and the ground object in the simulated scene, thereby proving the robustness of the adversarial perturbation of this application at different pitch angles. The change in the fooling rate at different pitch angles of 2 degrees from 19 to 27 degrees was tested under the ResNet-50 network. As shown in Table 9, the results show that the change in the fooling rate of the three targets at different pitch angles is not obvious, among which the average fooling rate of the sedan target and the van target at different pitch angles is higher, and the average fooling rate of the off-road vehicle target is the lowest. This shows that the attack method of this application is robust at different pitch angles.

[0112] Table 9. The fooling rate of adversarial samples generated at different pitch angles under the ResNet-50 network.

[0113]

[0114]

[0115] The beneficial effects of this application are as follows:

[0116] This application proposes a new physical adversarial attack method SAR-PAA for SAR ATR, which can migrate adversarial perturbations that are difficult for the human eye to recognize in the digital space to the physical space and cause misclassification of SAR ATR. A SAR simulation renderer RaySAR is used to realize the conversion from the physical world to the digital world, and the digital perturbations are updated through back propagation by designing an effective loss function. Finally, the final digital perturbation is mapped back to the material information of the 3D model through a mapping model to achieve the final physical perturbation. Experimental results show that this application can effectively realize SAR adversarial attacks in the physical world, achieving an average fooling rate of up to 99.02% in the digital world and an average fooling rate of up to 97.87% in the physical world. Compared with existing methods, this application bridges the gap between digital attacks and physical attacks and has the possibility of being realized in the real world.

[0117] As used herein, the word "preferred" is intended to serve as an example, instance, or illustration. Any aspect or design described herein as "preferred" is not necessarily to be construed as advantageous over other aspects or designs. Rather, the use of the word "preferred" is intended to present concepts in a concrete manner. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X employs A or B" is intended to mean any of the naturally inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, then "X employs A or B" is satisfied in any of the foregoing examples.

[0118] Moreover, although the present disclosure has been shown and described with respect to one or implementation, those skilled in the art will think of equivalent variations and modifications based on reading and understanding of this specification and the accompanying drawings. The present disclosure includes all such modifications and variations and is limited only by the scope of the appended claims. In particular, with respect to the various functions performed by the above-mentioned components (such as elements, etc.), the terms used to describe such components are intended to correspond to any component (unless otherwise indicated) that performs the specified function of the component (such as it is functionally equivalent), even if structurally different from the disclosed structure that performs the function in the exemplary implementation of the present disclosure shown herein. In addition, although the specific features of the present disclosure have been disclosed with respect to only one of several implementations, such features can be combined with one or other features of other implementations that can be desired and advantageous for a given or specific application. Moreover, insofar as the terms "including", "having", "containing" or their variations are used in specific embodiments or claims, such terms are intended to be included in a manner similar to the term "comprising".

[0119] The functional units in the embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or multiple or more units may be integrated into a single module. The aforementioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The aforementioned storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc. The aforementioned devices or systems may execute the storage method in the corresponding method embodiment.

[0120] In summary, the above embodiment is one implementation method of the present invention, but the implementation method of the present invention is not limited to the described embodiment. Any other changes, modifications, substitutions, combinations, and simplifications that deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A camouflage image generation method for SAR images, characterized in that: The following steps are involved: The process of generating SAR images from 3D models is described as: , is the SAR geometry output data, is the grid information, is the material information, is the imaging parameter, for ray tracing renderers; use initialization , is the final adversarial SAR geometric output data; Start iteration. In each iteration, perform the following steps: Update the mask vector and ; right Perform post-processing , get a two-dimensional SAR image ; The image Input SAR automatic target recognition and classification network , get the predicted label ; Calculate the loss function; Update using the gradient backpropagation algorithm ; After the iteration, according to the mapping model Generate the final camouflaged image; Among them, the two-dimensional SAR image is obtained The process is expressed as Indicates that are post-processing parameters, including the maximum and minimum pitch angles, the maximum and minimum range values, the spacing, and the maximum number of ray ejections. and are the height and width of the image respectively; will be The perturbation of is defined as digital camouflage, and the perturbation generation is regarded as an optimization problem with the following objective function: Where, As the adversarial SAR geometric output data is optimized in iterations, is the final adversarial SAR geometric output data, is the loss function, is the true label; In order to generate effective two-dimensional SAR images, the range of intensity values ​​is limited to [0, 1]. , the goal is to make , is a single intensity value in the SAR geometric output data, is the perturbation value applied to a single intensity value. Since the range of the sigmoid function is (0,1), let , making ; In the initialization parameters When , use the inverse function of the sigmoid function To ensure the similarity between the perturbation and the original intensity value vector; use the L2 norm to perturb the vector To restrict, even if smallest; The mapping model Determine three transformation relationships: Coordinate transformation: From The coordinate system of the scattering points in the image is converted to the coordinate system of the 3D model; Points correspond to triangles; From intensity values ​​to material parameters: Changes in the intensity values ​​in the SAR geometry output data correspond to changes in the material parameters in the 3D model; For coordinate transformation, the ray tracing renderer Record the coordinates of each scattering point in the 3D model's coordinate system to achieve the conversion from the scattering point coordinate system to the 3D model's coordinate system; for the points corresponding to the triangular surface element, select the scattering point with the largest change in intensity value before and after the disturbance as the benchmark to calculate the material parameters of the triangular surface element; For the transition from intensity values ​​to material parameters, a single variable optimizer is used to perform bounded optimization on the scattering coefficient of the triangle face corresponding to each scattering point; specifically, a function is defined that inputs the scattering coefficient and uses the same imaging parameters as before to render the scattering coefficient through a ray tracing renderer. The absolute value of the difference between the output intensity value and the target intensity value is obtained, and the Brent algorithm is used to find the local minimum of this function in the interval [0,1] to obtain the scattering coefficient corresponding to the target intensity value on the triangle surface.

2. The camouflage image generation method for SAR images according to claim 1, characterized in that: The grid information includes vertex coordinates and triangular face information, and the material information includes reflection coefficient, scattering coefficient and surface roughness; the imaging parameters Including radar signal source coordinates, receiving antenna coordinates and pointing position; the ray tracing renderer , using an approximate physical optics model to simulate the specular and diffuse reflection of radar signals and record SAR geometric output data , which includes the reflection direction and ray path of the echo position in the image.

3. The camouflage image generation method for SAR images according to claim 2, characterized in that: The loss function is as follows: Where, is the parameter that needs to be optimized. is the intensity value in the SAR geometric output data, is a constant used to balance the classification loss and limit the perturbation strength loss, is the cross entropy loss function.

4. The camouflage image generation method for SAR images according to claim 3, characterized in that: The sparse attack strategy is used to limit the range of the perturbation scattering points, that is, to limit the number of perturbation scattering points. Specifically, the following are included: Initialize the mask vector, which has the same shape as the perturbation vector and contains all 1s. Multiply the generated perturbation by the mask vector element by element. Arrange the elements in the perturbation vector in order of size. If an element is less than a certain threshold, set the element with the corresponding index in the mask vector to 0. This means that perturbations in the perturbation vector that are less than a certain threshold are eliminated, so that the number of 0 elements in the mask vector increases continuously during iteration.

Images