SAR (Synthetic Aperture Radar) remote sensing image migration simulation method fusing ground object scattering characteristics

Abstract

The invention relates to an SAR (Synthetic Aperture Radar) remote sensing image migration simulation method fusing ground object scattering characteristics, which is characterized in that an optical to SAR remote sensing image migration simulation network fusing the ground object scattering characteristics is designed based on a conditional generative adversarial network Pix2Pix, and the authenticity improvement of local scattering details in a simulated SAR remote sensing image and the interpretability enhancement of the migration network are realized. The invention provides an SAR remote sensing image migration simulation method aiming at the problems of low confidence coefficient and poor interpretability when an SAR remote sensing image is generated by an existing migration network technology by fully utilizing a heterogeneous ground feature microwave scattering mechanism. According to the method, by designing a multi-scale fine scattering discriminator integrated with ground object strong and weak scattering slice discrimination and a loss function driven by scattering characteristics, local scattering details are enhanced, and the overall simulation capability is improved, so that the purpose of performing high-quality simulation on an SAR remote sensing image is achieved. Experiments show that a large number of SAR remote sensing images more conforming to real ground object scattering characteristics can be generated, and data can be provided for training and verification of an optical and SAR bimodal remote sensing image intelligent processing algorithm.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology and deep learning technology, and specifically to a SAR remote sensing image migration simulation method integrating the scattering characteristics of ground objects. Background Art

[0002] In recent years, the rapid development of SAR imaging technology has led to advances in SAR remote sensing image processing, particularly in image segmentation, target detection, and recognition. Currently, SAR remote sensing image processing algorithms have entered a new era of intelligent processing algorithms based on deep neural networks. The massive data-driven nature of deep neural networks leads to a significant demand for SAR remote sensing image data. Without sufficient data, models struggle to fully train the network to extract effective features. Consequently, the need for SAR remote sensing image simulation has arisen.

[0003] Traditional SAR remote sensing image simulation is achieved by calculating the target's backscatter coefficient. This requires three-dimensional geometric modeling of the simulated target. Simulation accuracy is largely dependent on the accuracy of the modeling, and it is difficult to quickly and massively generate target samples, which cannot meet the data volume requirements of deep neural networks. A growing number of researchers are attempting to combine deep neural networks (DNNs) with the calculation of ground object scattering properties, achieving rapid simulation generation of high-quality SAR images while ensuring network interpretability. While these methods have successfully integrated deep neural networks into the calculation of ground object electromagnetic scattering properties, they only replace a specific physical computational step and rely heavily on specific datasets (such as the MSTAR dataset) for analysis and verification. The transferability of these models still needs to be further improved. Summary of the Invention

[0004] In order to solve the problems of poor interpretability and transferability of deep neural network models, this paper proposes a SAR remote sensing image migration simulation method that integrates the scattering characteristics of ground objects based on the conditional generative adversarial network model.

[0005] The present invention proposes a SAR remote sensing image migration simulation method that integrates the scattering characteristics of ground objects. The method enhances local scattering details and improves the overall simulation capability by designing a multi-scale fine scattering discriminator that incorporates the discrimination of strong and weak scattering slices of ground objects and a loss function driven by scattering characteristics, ultimately achieving high-quality simulation of SAR remote sensing images.

[0006] Furthermore, the specific process of the simulation method is:

[0007] Step 1: Input the measured optical image into the U-net-based generator, which extracts the features of the input optical image and uses the extracted image features to preliminarily generate a simulated SAR remote sensing image;

[0008] Step 2: Input the measured SAR remote sensing image and the simulated SAR remote sensing image obtained in Step 1 into the ground object scattering slice generation module, and at the same time input the dataset label mask map containing the optical image semantic information of the ground object categories. The ground object scattering slice generation module classifies the homogeneous ground objects with similar electromagnetic scattering characteristics in the image to obtain the strong and weak scattering slices of the measured and simulated SAR for ground objects. The scattering slices are additionally input into the network discriminator, and these scattering slices are also used for the pixel-level L1 loss calculation of the network;

[0009] Step 3: Using the measured optical image as a condition, use the multi-scale fine scattering discriminator to discriminate the authenticity of the entire simulated SAR remote sensing image, ensuring that the network can learn the complex mapping relationship between the optical image and the overall SAR remote sensing image. The multi-scale fine scattering discriminator takes the strong and weak scattering slices of the ground objects generated in Step 2 as the discrimination object, and separately optimizes the features of different scattering levels of the simulated SAR remote sensing image, and fully pays attention to and analyzes different types of scattering characteristics during discrimination;

[0010] Step 4: According to the scattering characteristics of the ground objects concerned in the simulation task, drive the calculation of the network loss function based on the scattering characteristics, and constrain the network to realize the migration generation of high-quality SAR remote sensing images.

[0011] Further, in Step 2, the ground object scattering slice generation module classifies the homogeneous ground objects with similar electromagnetic scattering characteristics in the image to obtain the strong and weak scattering slices of the measured and simulated SAR for ground objects specifically as follows:

[0012] First, based on the attributes and electromagnetic scattering characteristics of the ground objects contained in the paired dataset of optical and SAR remote sensing images and the optical image semantic information of the ground object categories contained in the dataset label mask map, determine the number of levels of the ground object scattering slices of the generated SAR remote sensing image and the initial threshold;

[0013] Then, perform real number encoding on the initial threshold, and randomly generate a number of initial individuals. The length of the encoded chromosome string satisfies L = Th1 + Th2 + Th3 + Th4, where Th1, Th2, Th3, and Th4 represent the thresholds for generating each level of scattering slices, and Th1 < Th2 < Th3 < Th4;

[0014] Then, use the fitness function for calculation, where S x(i) is the ground object scattering slices of each level generated by this threshold individual, P i is each ground object region of the SAR remote sensing image generated based on the optical semantic information, m is the number of levels of the ground object scattering slices, and Num(·) is the function to calculate the number of non-zero pixels in the image;

[0015] Then, divide the roulette wheel area according to the fitness value size, and use the roulette wheel operator to select the individuals of the next generation. Individuals with larger fitness values have a higher probability of being selected. Pair the selected individuals in pairs and use the arithmetic crossover method to generate new individuals. Set the mutation probability, traverse the genes of all individuals, and adjust the gene values according to the current generation number, random number, and constant when mutations occur to increase the population diversity;

[0016] Finally, stop when t > T or the fitness reaches the required error.

[0017] Further, in step 3, the multi-scale fine scattering discriminator is specifically:

[0018] Use the multi-scale fine scattering discriminator to discriminate the authenticity of the SAR remote sensing image under optical conditions and the authenticity of each level of scattering slices. On the one hand, the discriminator discriminates the authenticity of the entire simulated SAR remote sensing image based on the measured optical image; on the other hand, the discriminator takes the strong and weak scattering slices generated in step 2 as the discrimination objects, and separately optimizes the characteristics of different scattering levels of the simulated SAR remote sensing image, effectively avoiding the mutual interference between characteristics.

[0019] Further, in step 4, the loss function based on the scattering characteristic drive network specifically includes:

[0020] Conditional adversarial generation loss l cGAN Scattering characteristic-driven adversarial generation loss l GAN-SCD Scattering characteristic-driven L1 loss It consists of, and the expression is as follows:

[0021]

[0022] In the formula, G and D respectively represent the generator and discriminator of the network, x and y represent the optical and SAR remote sensing images, y i G(x) i respectively represent the i-th level slice of the measured SAR remote sensing image and the i-th level slice of the generated false SAR remote sensing image, and E(·) represents calculating the expected value; λ i represents the loss weight of the i-th level slice, which can be set according to the task scenario of the data set and the scattering characteristics of the ground objects concerned in the simulated SAR remote sensing image; P data i (y i ) is the distribution of the data of the i-th level slice of the measured image in the SAR domain, and P data (x) is the distribution of the image data in the optical domain;

[0023] The final loss function is as follows:

[0024]

[0025] In the formula, the parameter α is l cGAN The weight of the loss, which controls the adversarial generation loss l driven by the scattering characteristics GAN-SCD and l cGAN The relative importance of the loss; γ is The weight of the loss, which controls the adversarial generation loss l driven by the scattering characteristics GAN-SCD and The relative importance of the loss.

[0026] Compared with the prior art, the present invention adopts the above technical solutions and has the following technical effects:

[0027] The present invention is applicable to generating a large number of high-quality SAR remote sensing images. On the premise of ensuring the interpretability and transferability of the deep neural network model, it fuses the features of strong and weak scattering regions of SAR remote sensing images, and further improves the authenticity and detail restoration degree of key ground objects in SAR remote sensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic diagram of the overall process of the SAR remote sensing image migration simulation method of the present invention;

[0029] Figure 2 is a schematic diagram of the process of the ground object scattering slice generation module of the present invention;

[0030] Figure 3 is a schematic diagram of the network structure of the multi-scale fine scattering discriminator of the present invention;

[0031] Figure 4 is a graph of the comparative experimental results of the application of the present invention on the vehicle dataset. DETAILED DESCRIPTION OF THE INVENTION

[0032] The technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings:

[0033] The present invention proposes a SAR remote sensing image migration simulation method integrating ground object scattering characteristics. This method makes full use of the microwave scattering mechanism of heterogeneous ground objects. Based on the problems of difficult feature extraction and inaccurate scattering characteristic simulation in SAR remote sensing image simulation in actual engineering, scattering slices representing the scattering characteristics of homogeneous ground objects are introduced, and a multi-scale fine scattering discriminator integrating the discrimination of strong and weak scattering slices of ground objects is designed. On the basis of maintaining the understanding of the global structure and overall features of the image, the features of different scattering levels are separately optimized, and a scattering characteristic-driven loss function is designed to enhance local scattering details and improve the overall simulation ability, ultimately achieving the purpose of high-quality simulation of SAR remote sensing images. The present invention proposes a SAR remote sensing image migration simulation method integrating ground object scattering characteristics. The method enhances local scattering details and improves the overall simulation ability by designing a multi-scale fine scattering discriminator integrating the discrimination of strong and weak scattering slices of ground objects and a scattering characteristic-driven loss function, and finally realizes high-quality simulation of SAR remote sensing images.

[0034] The SAR remote sensing image migration simulation method integrating ground object scattering characteristics provided in this application is specifically as follows:

[0035] Step 1: Input the measured optical image into the generator based on U-net. The generator extracts the features of the input optical image and preliminarily generates a SAR remote sensing image using the extracted image features.

[0036] Step 2: Input the measured SAR remote sensing image and the simulated SAR remote sensing image obtained in Step 1 into the ground object scattering slice generation module, and at the same time input the dataset label mask image containing the semantic information of the optical image ground object category. The ground object scattering slice generation module classifies the homogeneous ground objects with similar electromagnetic scattering characteristics in the image to obtain the strong and weak scattering slices of the ground objects of the measured and simulated SARs. The scattering slices are additionally input into the network discriminator, and the scattering slices are also used for the pixel-level L1 loss calculation of the network.

[0037] Step 3: Using the measured optical image as a condition, use the multi-scale fine scattering discriminator to discriminate the authenticity of the entire simulated SAR remote sensing image, ensuring that the network can learn the complex mapping relationship between the optical image and the overall SAR remote sensing image. The multi-scale fine scattering discriminator takes the strong and weak scattering slices of the ground objects generated in Step 2 as the discrimination object, separately optimizes the features of different scattering levels of the simulated SAR remote sensing image, and fully focuses on and analyzes different types of scattering characteristics during discrimination.

[0038] Step 4: According to the scattering characteristics of the ground objects concerned in the simulation task, drive the calculation of the network loss function to constrain the network to achieve the migration generation of high-quality SAR remote sensing images.

[0039] The specific example of the SAR remote sensing image migration simulation integrating ground object scattering characteristics is as follows:

[0040] Application Step 1: Input the measured optical image into the U-net based generator. The detailed convolutional structure parameters of the generator network are shown in Table 1. The encoding part performs downsampling by setting the stride to 2 in the convolutional layer. Each feature map in the decoding layer is connected to the feature map with the same resolution size in the encoding layer in terms of channels, and the resolution of the original image is gradually restored through deconvolution, ensuring that the resolution of the output and input images is the same, thus preliminarily completing the generation from optical to SAR remote sensing images;

[0041] Table 1 Specific parameters of the generator network structure

[0042]

[0043] Furthermore, Application Step 2: Use the ground object scattering slice generation module to generate the strong and weak scattering slices of the ground objects in the SAR remote sensing image by using the genetic algorithm that fuses the optical semantic information of the dataset label mask map, and classify the homogeneous ground objects with similar electromagnetic scattering characteristics in the measured and simulated SAR remote sensing images obtained in Step 1; In Step 2, the ground object scattering slice generation module classifies the homogeneous ground objects with similar electromagnetic scattering characteristics in the image to obtain the strong and weak scattering slices of the ground objects in the measured and simulated SAR, specifically as follows:

[0044] First, based on the attributes and electromagnetic scattering characteristics of the ground objects contained in the paired dataset of optical and SAR remote sensing images and the optical image semantic information of the ground object categories contained in the dataset label mask map, determine the number of levels of the ground object scattering slices generated in the SAR remote sensing image and the initial threshold;

[0045] Then, perform real number encoding on the initial threshold, randomly generate several initial individuals, and the length of the encoded chromosome string satisfies L = Th1 + Th2 + Th3 + Th4, where Th1, Th2, Th3, and Th4 represent the thresholds for generating each level of scattering slices, and Th1 < Th2 < Th3 < Th4;

[0046] Then, adopt the fitness function for calculation, where S x(i) is the ground object scattering slices of each level generated by this threshold individual, P i is each ground object region in the SAR remote sensing image generated based on the optical semantic information, m is the number of levels of the ground object scattering slices, and Num(·) is the function to calculate the number of non-zero pixels in the image;

[0047] Then, divide the roulette wheel area according to the fitness value size, and use the roulette wheel operator to select the individuals of the next generation. Individuals with larger fitness values have a higher probability of being selected. Pair the selected individuals in pairs and use the arithmetic crossover method to generate new individuals. Set the mutation probability, traverse the genes of all individuals, and adjust the gene values according to the current generation, random number, and constant when mutation occurs to increase the population diversity;

[0048] Finally, stop when t > T or the fitness reaches the required error.

[0049] Furthermore, application step 3: Use the multi-scale fine scattering discriminator to discriminate the authenticity of the SAR remote sensing image under optical conditions and the authenticity of each level of scattering slices. On the one hand, the discriminator takes the optical image as a condition to discriminate the authenticity of the entire simulated SAR remote sensing image, ensuring that the network can learn the complex mapping relationship between the optical image and the overall SAR remote sensing image, and at the same time ensuring the discriminator's in-depth understanding of the global structure and overall characteristics of the SAR remote sensing image; on the other hand, the discriminator takes the strong and weak scattering slices generated in step 2 as the discrimination objects, and separately optimizes the characteristics of different scattering levels of the simulated SAR remote sensing image, effectively avoiding the mutual interference between features, and ensuring that different types of scattering characteristics can be fully concerned and analyzed during discrimination; in the said step 3, the multi-scale fine scattering discriminator is specifically:

[0050] Use the multi-scale fine scattering discriminator to discriminate the authenticity of the SAR remote sensing image under optical conditions and the authenticity of each level of scattering slices. On the one hand, the discriminator discriminates the authenticity of the entire simulated SAR remote sensing image based on the measured optical image; on the other hand, the discriminator takes the strong and weak scattering slices generated in step 2 as the discrimination objects, and separately optimizes the characteristics of different scattering levels of the simulated SAR remote sensing image, effectively avoiding the mutual interference between features.

[0051] Furthermore, application step 4: According to the scattering characteristics of the ground objects concerned in the simulation task, drive the calculation of the network loss function. The loss weights can be set according to the task scenario of the dataset and the scattering characteristics of the ground objects concerned in the simulated SAR remote sensing image. For the vehicle dataset used in this method, during training, the network model needs to pay more attention to the vehicle targets that appear as strong scattering regions. Therefore, the loss weights of the strong, medium, and weak level slices are set to 0.6, 0.2, and 0.2 respectively, and finally the network is constrained to achieve the migration generation of high-quality SAR remote sensing images.

[0052] In the said step 4, the driving of the network loss function based on the scattering characteristics specifically includes:

[0053] Conditional adversarial generation loss l cGAN 、Scattering characteristic-driven adversarial generation loss l GAN-SCD 、Scattering characteristic-driven L1 loss Composition, the expression is as follows:

[0054]

[0055] In the formula, G and D respectively represent the generator and discriminator of the network, x and y represent optical and SAR remote sensing images, and y i , G(x) i respectively represent the i-th level slice of the measured SAR remote sensing image and the i-th level slice of the generated fake SAR remote sensing image, and E(·) represents calculating the expected value; λ i represents the loss weight of the i-th level slice, which can be set according to the task scenario of the data set and the scattering characteristics of the features of interest in the simulated SAR remote sensing image; P data (y i ) is the distribution of the data of the i-th level slice of the measured image in the SAR domain, and P data (x) is the distribution of the image data in the optical domain;

[0056] The final loss function is as follows:

[0057]

[0058] In the formula, the parameter α is the weight of the l cGAN loss, which controls the relative importance of the scattering characteristic-driven adversarial generation loss l GAN-SCD and l cGAN loss; γ is the weight of the loss, which controls the relative importance of the scattering characteristic-driven adversarial generation loss l GAN-SCD and loss.

[0059] The comparison results of the simulation experiments on the high-resolution spaceborne optical and SAR vehicle data sets are as Figure 4 shown. By observing the rectangular areas boxed in the figure, it can be clearly seen that compared with other transfer generation methods based on deep neural networks, the scattering characteristics and texture features of the SAR vehicle targets generated by this method are closer to the measured SAR remote sensing images. In Figure 4 , for the vehicles in the bottom row, due to the influence of the surrounding trees in the optical image, the quality of the SAR remote sensing images generated by other methods is generally poor, and even target missing occurs. While this method can generate relatively clear vehicle contours and edge details, significantly improving the authenticity and detail retention of the simulation effect.

[0060] The specific embodiments described above further elaborate on the object, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only the specific embodiments of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A SAR remote sensing image migration simulation method integrating ground object scattering characteristics, characterized in that The method enhances local scattering details and improves the overall simulation ability by designing a multi-scale fine-grained scattering discriminator that incorporates the discrimination of strong and weak scattering slices of ground objects and a scattering characteristic-driven loss function, and finally realizes the high-quality simulation of SAR remote sensing images.

2. A method for SAR remote sensing image migration simulation integrating terrain scattering characteristics according to claim 1, characterized in that The specific process of the simulation method is as follows: Step 1: Input the measured optical image into the U-net-based generator. The generator extracts the features of the input optical image and uses the extracted image features to initially generate a simulated SAR remote sensing image. Step 2: Input the measured SAR remote sensing image and the simulated SAR remote sensing image obtained in Step 1 into the ground object scattering slice generation module, and at the same time input the dataset label mask map containing the semantic information of the optical image of the ground object category. The ground object scattering slice generation module classifies the homogeneous ground objects with similar electromagnetic scattering characteristics in the image to obtain the strong and weak scattering slices of the ground objects in the measured and simulated SARs. The scattering slices are additionally input into the network discriminator, and these scattering slices are also used for the pixel-level L1 loss calculation of the network. Step 3: Using the measured optical image as a condition, use the multi-scale fine-grained scattering discriminator to determine the authenticity of the entire simulated SAR remote sensing image, ensuring that the network can learn the complex mapping relationship between the optical image and the overall SAR remote sensing image. The multi-scale fine-grained scattering discriminator takes the strong and weak scattering slices of the ground objects generated in Step 2 as the discrimination object, and separately optimizes the features of different scattering levels of the simulated SAR remote sensing image, and fully pays attention to and analyzes different types of scattering characteristics during discrimination. Step 4: According to the scattering characteristics of the ground objects concerned in the simulation task, calculate the network loss function based on the scattering characteristics to constrain the network to realize the migration generation of high-quality SAR remote sensing images.

3. A method for SAR remote sensing image migration simulation integrating terrain scattering characteristics according to claim 2, characterized in that In Step 2, the specific process of the ground object scattering slice generation module classifying the homogeneous ground objects with similar electromagnetic scattering characteristics in the image to obtain the strong and weak scattering slices of the ground objects in the measured and simulated SARs is as follows: First, based on the attributes and electromagnetic scattering characteristics of the ground objects contained in the paired optical and SAR remote sensing image dataset and the semantic information of the optical image of the ground object category contained in the dataset label mask map, determine the number of levels of the ground object scattering slices of the generated SAR remote sensing image and the initial threshold. Then, perform real-number coding on the initial threshold and randomly generate several initial individuals. The length of the encoded chromosome string satisfies L = Th1 + Th2 + Th3 + Th4, where Th1, Th2, Th3, and Th4 represent the thresholds for generating scattering slices at each level, and Th1 < Th2 < Th3 < Th4. Then, the fitness function is adopted for calculation, where S x(i) is the scattering slices of each level of ground objects generated by the threshold individual, P i is each ground object region of the SAR remote sensing image generated based on optical semantic information, m is the number of levels of ground object scattering slices, and Num(·) is the number of non-zero pixels in the calculated image; Then, divide the roulette wheel area according to the fitness value, and use the roulette wheel operator to select the next-generation individuals. Individuals with larger fitness values have a higher probability of being selected; pair the selected individuals two by two and use the arithmetic crossover method to generate new individuals; set the mutation probability, traverse the genes of all individuals, and adjust the gene value according to the current generation, random number, and constant when mutation occurs to increase the population diversity. Finally, stop when t > T or the fitness reaches the required error.

4. A method for SAR remote sensing image migration simulation integrating terrain scattering characteristics according to claim 2, characterized in that In Step 3, the multi-scale fine-grained scattering discriminator is specifically: Use a multi-scale fine-grained scattering discriminator to distinguish the authenticity of SAR remote sensing images under optical conditions and the authenticity of scattering slices at all levels. On the one hand, the discriminator determines the authenticity of the entire simulated SAR remote sensing image based on the measured optical image; on the other hand, the discriminator takes the strong and weak scattering slices generated in step 2 as the discrimination objects, and separately optimizes the features of the simulated SAR remote sensing image at different scattering levels, effectively avoiding mutual interference between features.

5. A method for SAR remote sensing image migration simulation integrating ground object scattering characteristics according to claim 2, characterized in that, In step 4, the loss function based on the scattering characteristic driving network specifically includes: Conditional adversarial generation loss l cGAN and scattering characteristic-driven adversarial generation loss l GAN-SCD and scattering characteristic-driven L1 loss are combined, and the expression is as follows: where G and D represent the generator and discriminator of the network respectively, x and y represent optical and SAR remote sensing images, and y i , G(x) i represent the i-th level slice of the measured SAR remote sensing image and the i-th level slice of the generated fake SAR remote sensing image respectively, and E(·) represents the calculation of the expected value; λ i represents the loss weight of the i-th level slice, which can be set according to the task scenario of the dataset and the scattering characteristics of the features of interest in the simulated SAR remote sensing images; P data (y i ) is the distribution of the i-th level slice data of the SAR-domain measured image, and P data (x) is the distribution of the optical-domain image data; The final loss function is as follows: In the formula, the parameter α is l cGAN The weight of the loss, which controls the relative importance of the adversarial generation losses l GAN-SCD and l cGAN driven by the scattering characteristics; γ is The weight of the loss, which controls the relative importance of the adversarial generation losses l GAN-SCD and driven by the scattering characteristics.

Images