A SAR image simulation method based on multi-category geomorphologic style transfer

Abstract

The application discloses a SAR image simulation method based on multi-category landform style migration, relates to the technical field of aerial image processing, and comprises the following steps: S1, a plurality of remote sensing visible light images and SAR images are divided into a training set and a test set respectively, and pretreatment is performed; S2, an improved training semantic segmentation model is trained by using a semantic segmentation dataset; S3, an improved generative adversarial model is trained by using a visible light and SAR dataset; S4, a remote sensing image is segmented by using the trained semantic segmentation model; S5, the segmented visible light images of different landforms are processed by using the generative adversarial model, and SAR simulation images of different landforms are generated; S6, the generated SAR simulation images of different landforms are spliced to obtain a complete simulation image; S7, the spliced SAR simulation image is subjected to image fusion processing; and S8, noise simulation processing is performed on the SAR simulation image subjected to the fusion processing. By adopting the method, the accuracy of a detection result is improved, and the detection result has better robustness.

Description

Technical Field

[0001] This invention belongs to the field of aerial image processing technology, and in particular relates to a SAR image simulation method based on multi-category terrain style transfer. Background Technology

[0002] Compared to other remote sensing methods, Synthetic Aperture Radar (SAR) boasts superior capabilities and unparalleled advantages, leading to its widespread application. SAR image interpretation is crucial for SAR systems. SAR image generation technology not only serves for deception and jamming but also provides data for subsequent image processing research, making it a vital component in many SAR image application areas. However, SAR image interpretation often faces challenges such as insufficient datasets, and the quality of SAR datasets varies significantly due to noise interference. Traditional methods struggle to overcome interference and improve generated image quality; therefore, researching new methods for generating simulated SAR images is an urgent issue.

[0003] Existing algorithms for SAR image simulation from visible light images fall into two main categories: those based on traditional image processing and those based on deep learning. Traditional methods for supplementing SAR image datasets involve cropping and adding noise to existing images to expand the data, or segmenting and reconstructing images to expand the dataset. However, the reconstructed images obtained through segmentation and reconstruction are not of high quality and do not improve the overall dataset quality, thus not being truly effective expansion methods. With the development of deep learning in image processing, deep generative models can greatly assist in generating and processing SAR image datasets. Convolutional Neural Networks (CNNs) are commonly used and have become the foundation for building many complex generative models. Therefore, it is essential to apply neural networks to build targeted models to solve the difficulties in practical SAR image segmentation. Generative adversarial models are currently the most promising representative of deep generative models, enabling the transfer between different styles. In unsupervised classes, they are considered the most promising generative models, capable of obtaining near-realistic data with indistinguishable effects. Due to its powerful generation capabilities, stable training, fast convergence, and diverse generated samples, this type of model is highly effective in image generation for data augmentation, saving significant human and material resources. Therefore, introducing generative adversarial models into the field of SAR image generation is of great importance. Generative adversarial SAR image simulation methods perform well on datasets with relatively simple scenes, but perform poorly on datasets with complex terrain features of various types.

[0004] In SAR imaging, different terrain features produce diverse variations. Flat terrain typically exhibits uniform echo intensity because the reflected energy is relatively uniform. However, different types of landforms, such as water bodies and forests, have different microwave reflection characteristics. Water bodies usually show darker echoes because they absorb microwave energy, while forests exhibit complex patterns of scattered echoes due to the scattering effect of trees and branches. Surface roughness leads to different scattering effects; rough surfaces result in stronger scattering, leading to stronger echo signals, while smooth surfaces reduce scattering, resulting in weaker echo signals. Terrain variations, such as mountains and valleys, cause microwave signals to be reflected back to the radar from different directions, producing different phase differences and thus displaying distinct mottled patterns in SAR images. Furthermore, the right angles and edges of buildings and man-made structures appear as prominent bright or dark spots. However, SAR imaging is also affected by other factors, such as observation parameters, atmospheric conditions, and soil moisture. Considering all these factors, different types of terrain exhibit rich characteristics in SAR images. In summary, the characteristics of SAR imaging mean that the features of SAR images derived from visible light images in different scenarios often differ significantly. Directly using generative adversarial networks for processing often fails to take into account multiple and diverse terrain scenarios, resulting in distortion of SAR simulation images.

[0005] Therefore, it is necessary to provide a SAR image simulation method based on multi-category landform style transfer to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide a SAR image simulation method based on multi-category landform style transfer. Through accurate semantic segmentation and processing, it can better capture the details and features of different landforms in complex scenes. It can perform individual processing for different landform categories, thereby better simulating the SAR image corresponding to each landform. By stitching these processed images, a complete SAR simulation image corresponding to the visible light image can be reconstructed, thus providing more comprehensive and accurate data information.

[0007] To achieve the above objectives, this invention provides a SAR image simulation method based on multi-category terrain style transfer, comprising the following steps:

[0008] S1: Take several remote sensing visible light images and SAR images, divide them into training set and test set respectively, and perform preprocessing;

[0009] S2: Train the improved semantic segmentation model using the semantic segmentation dataset obtained in step S1;

[0010] S3: Training an improved generative adversarial model using visible light and SAR datasets;

[0011] S4: Use the trained semantic segmentation model to segment the remote sensing image to form several groups of images with different landforms;

[0012] S5: The visible light images of several groups of different landforms after segmentation are processed by a generative adversarial model to generate several corresponding SAR simulation images of different landforms.

[0013] S6: Stitch together the generated SAR simulation images of different landforms to obtain a complete simulation image;

[0014] S7: Perform a series of image fusion processes on the stitched SAR simulation image obtained in step S6 to obtain the fused SAR simulation image;

[0015] S8: Perform noise simulation processing on the fused SAR simulation image obtained in step S7 to obtain the final result.

[0016] Preferably, in step S1, the required dataset is based on UAV remote sensing data, which comes from visible light images and SAR images taken by the UAV from the same viewpoint, and specifically includes the following steps:

[0017] S11: Cut all images into squares of equal length and width, and copy the data twice;

[0018] S12: Take one portion of the data and perform semantic segmentation annotation on the visible light images in this portion of the data. Divide the visible light and SAR images into training and test sets in a 9:1 ratio to form a semantic segmentation dataset.

[0019] S13: Using the results of semantic segmentation annotation, the visible light images and SAR images in another dataset are segmented into different images according to different landform categories. Each image is then divided into a training set and a test set in a 9:1 ratio, serving as different generative adversarial datasets corresponding to different landform categories.

[0020] Preferably, step S2 specifically includes the following steps:

[0021] S21: An attention mechanism is introduced into the DeepLabV3+ model to strengthen the weight of the segmented edge parts during image processing, resulting in an improved semantic segmentation model;

[0022] S22: After the model is improved, input the training set from the semantic segmentation dataset obtained in S1 to start training;

[0023] S23: After training is complete, use the test set to evaluate the model's performance. Input the images in the test set into the trained model to obtain the model's image segmentation prediction results.

[0024] S24: After obtaining the prediction results, compare the prediction results with the true labels, and use evaluation metrics to evaluate the accuracy and generalization ability of the model.

[0025] S25: Optimize the model based on the test results.

[0026] Preferably, in step S3, the improved generative adversarial model is trained using the visible light and SAR datasets from S1, specifically including the following steps:

[0027] S31: By introducing the Wasserstein distance and gradient penalty term mechanism into the CycleGAN model, an improved generative adversarial model is obtained.

[0028] The improved model generator takes an image as input, first passes it through a residual network to extract image features, and then enters four branches. In the first branch, it undergoes reshape and transpose operations. In the second and third branches, it undergoes a reshape operation. The fourth branch does not perform any processing.

[0029] The outputs of the first and second branches are multiplied by matrix. The result is then multiplied by matrix with the output of the third branch after a softmax operation. After a reshape operation, the result is superimposed with the data from the fourth branch. Finally, the result is convolved to obtain the output image.

[0030] The optimization objective of the entire model is as follows:

[0031]

[0032]

[0033] Where x, y are the images input to the discriminator, and D A (·),D B (·) represents the discriminator output, G A (·), G B (·) represents the generator output, p r For the true data distribution, p g Generate data distribution for the network. For distribution p r and p g The Wasserstein distance between them, C, is the gradient of the discriminator network. The regularization term corresponding to the gradient of the discriminator network, with parameter λ. i It is the coefficient of the gradient penalty term: ~p k For the data sampling points along the line connecting the sampling points of the real data distribution and the network-generated data distribution, ·~pdt (·) represents the probability distribution of the input data ·;

[0034] S32: After the model is improved, input the training set from the generative adversarial dataset obtained in step one, and start training;

[0035] S33: The training time depends on the size of the dataset and the complexity of the model. After the model is trained, the performance of the model is evaluated using a test set. The test set contains visible light images. These images are input into the trained model, and the generator obtains simulated SAR images.

[0036] S34: Observe the simulated SAR images generated by the generator, compare the generated simulated SAR images with real SAR images, and evaluate the model's performance on the test set.

[0037] S35: If there is a significant difference or low quality between the generated image and the target domain image, readjust and retrain the model until the generated image is similar to the target domain image and of good quality, so as to realize the simulation of different landform types in SAR images.

[0038] Preferably, step S4 specifically includes the following steps:

[0039] S41: Load the trained, improved semantic segmentation model into the system;

[0040] S42: Input a square visible light image of equal length and width into the model for processing;

[0041] S43: The model predicts each pixel in the image and assigns it to the corresponding terrain category.

[0042] Preferably, step S5 specifically includes the following steps:

[0043] S51: Load the improved generative adversarial model trained in step S3 into the system;

[0044] S52: The segmented visible light images of each category obtained in step S4 are used as input to the generative adversarial model for image conversion;

[0045] S53: The generative adversarial model encodes the input visible light image, learns the feature representation in the image, converts these features into corresponding features in the SAR image, and generates a synthetic SAR image through a decoder.

[0046] Preferably, in step S6, for each category of SAR simulation region, the visible light image is semantically segmented according to the landform category based on the semantic segmentation model in step S4. The images of each landform category are then stitched together to obtain a complete image containing all the original landforms. The stitching formula is as follows:

[0047]

[0048] Where n is the number of different landform categories, is the image corresponding to the i-th landform category, and w i For the weight of the image corresponding to the i-th terrain category, l i This refers to the location information of the image corresponding to the i-th landform category within the entire image.

[0049] Preferably, in step S7, an improved lightweight filtering algorithm is used to handle the seam problem after splicing, specifically including the following steps:

[0050] S71: Perform radiometric correction on the stitched image. The formula for the radiometric correction filtering algorithm is shown below:

[0051]

[0052] Where g(x,y) is the gray value of the original image, f(x,y) is the gray value of the resulting image, and m g s is the grayscale mean of the original image. g m is the standard deviation of the original image. f s is the grayscale mean of the template image. f The standard deviation of the template image is a∈[0,1], which is adjusted according to the actual image conditions.

[0053] S72: Obtain the semantic segmentation annotations of each image involved in the stitching, and transform the annotations to obtain a grayscale image where black represents boundary lines and white represents the background;

[0054] S73: Expand the black line into an area with a width of 5 pixels, denoted as the area to be processed. The specific method is to draw a 5×5 pixel black rectangle around the points on the boundary line. The formula for this process is as follows:

[0055] R j →R b

[0056] Where j is a point on the boundary line, R j For a 5×5 pixel block centered at point j, R b Represents black pixel blocks of the same size;

[0057] S74: Smooths the seam joints. The specific formula is shown below:

[0058]

[0059] Where j is any point in the region to be processed, n is the number of pixels in a 3×3 pixel block centered at that point that overlaps with the region to be processed, and L i Let D be the gray value of the i-th point among these points. j The points are the filtered points; the algorithm introduces the tanh function.

[0060] Preferably, in step S8, SAR image-specific noise is added to the obtained SAR simulation image, and the distribution of the noise follows the formula below:

[0061] I′(x, y)=I(x, y)+N

[0062] Where I(x, y) are the pixel values ​​in the original image, I′(x, y) are the pixel values ​​in the image after adding noise, and N is the noise distribution function, the detailed expression of which is as follows:

[0063]

[0064] Among them, w i The weights of N for each distribution i For different noise distributions, n is the total number of distributions; for each N i The formula is as follows:

[0065]

[0066] Where u is the expected mean of this distribution, σ is the standard deviation, and ρ is the correlation coefficient between the two dimensions, with a value range of [-1, 1].

[0067] Therefore, the SAR image simulation method based on multi-category landform style transfer described above, as used in this invention, has the following beneficial effects:

[0068] (1) In the SAR image simulation task under various complex terrains, the present invention has high accuracy in the conversion from visible light remote sensing images to corresponding simulated SAR images.

[0069] (2) By incorporating a generative adversarial model, the present invention makes the generated simulated SAR images reliable.

[0070] (3) This invention processes image parts with different landform features separately, so that the corresponding locations of the generated simulated SAR images have corresponding features, thereby improving the closeness between the simulated images and the real images.

[0071] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0072] Figure 1 This is a flowchart of a SAR image simulation method based on multi-category landform style transfer according to the present invention;

[0073] Figure 2 This is a graph of the semantic segmentation dataset and the generative adversarial dataset in this invention;

[0074] Figure 3 This is a block diagram of the attention mechanism introduced in the improved semantic segmentation model of this invention;

[0075] Figure 4 This is a schematic diagram of the improved model generator in this invention;

[0076] Figure 5 This is an example diagram of the landform classification and segmentation in this invention;

[0077] Figure 6 These are sample images of various types of simulation images in this invention;

[0078] Figure 7 This is an example image of image stitching in this invention;

[0079] Figure 8 These are before and after images showing the removal of seams in this invention;

[0080] Figure 9 These are before-and-after images showing the effects of adding noise in this invention. Detailed Implementation

[0081] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0082] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0083] The terms "comprising" or "including" as used in this invention mean that the element preceding the term encompasses the element listed after the term, and do not exclude the possibility of encompassing other elements. Terms such as "inner," "outer," "upper," and "lower" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In this invention, unless otherwise explicitly specified and limited, the term "attached" and similar terms should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can refer to a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication of two elements or the interaction relationship between two elements. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0084] like Figure 1 As shown, this invention provides a SAR image simulation method based on multi-category terrain style transfer, comprising the following steps:

[0085] S1: Take several remote sensing visible light images and SAR images, divide them into training and test sets respectively, and perform preprocessing; In step S1, the required dataset is based on UAV remote sensing data, which comes from visible light images taken by the UAV and SAR images from the same viewpoint, specifically including the following steps:

[0086] S11: Cut all images into squares of equal length and width, and copy the data twice;

[0087] S12: Take one portion of the data and perform semantic segmentation annotation on the visible light images in this portion of the data. Divide the visible light and SAR images into training and test sets in a 9:1 ratio to form a semantic segmentation dataset.

[0088] S13: Using the semantic segmentation annotation results, the visible light images and SAR images in another dataset are segmented into different images according to different landform categories. These are then divided into training and test sets in a 9:1 ratio, serving as different generative adversarial datasets corresponding to different landform categories. Figure 2 As shown.

[0089] S2: Train the improved semantic segmentation model using the semantic segmentation dataset obtained in step S1; step S2 specifically includes the following steps:

[0090] S21: Introduce an attention mechanism to the DeepLabV3+ model, such as Figure 3As shown, by strengthening the weights of edge segmentation during image processing, an improved semantic segmentation model is obtained. In the encoder of this model, the feature vector obtained after dilated convolution of the input image enters multiple convolutional layers and also enters the attention mechanism module designed in this method. In the attention mechanism module, the input feature vector enters two branches. In the first branch, it undergoes global pooling, and in the second branch, it undergoes average pooling. The outputs of these two branches are multiplied by a matrix. Then, the resulting feature vector is multiplied by the input feature vector, and the result is named the intermediate feature vector. This intermediate feature vector is then subjected to max pooling and average pooling respectively. The results are then superimposed and convolved to reduce dimensionality. This result is then multiplied by the intermediate feature vector, and the result is the output of the attention mechanism module. The output of the attention mechanism module, along with the output of the pooling pyramid, is input into the final 1×1 convolutional module of the encoder.

[0091] S22: After the model is improved, input the training set from the semantic segmentation dataset obtained in S1 to start training;

[0092] S23: After training is complete, use the test set to evaluate the model's performance. Input the images in the test set into the trained model to obtain the model's image segmentation prediction results.

[0093] S24: After obtaining the prediction results, compare the prediction results with the true labels, and use evaluation metrics to evaluate the accuracy and generalization ability of the model.

[0094] S25: Based on the test results, optimize the model. This can be done by adjusting hyperparameter values, increasing the amount of training data, or applying data augmentation techniques to further improve model performance, thereby enhancing the accuracy and generalization ability of the semantic segmentation task.

[0095] The validated and optimized model can be applied to semantic segmentation of new remote-sensed visible light images. Ultimately, it enables the input of the remote-sensed visible light image to be segmented into the model to obtain the semantic segmentation results, thereby achieving accurate segmentation of different landform types in remote-sensed visible light images.

[0096] S3: Train the improved generative adversarial model using visible light and SAR datasets; In step S3, the improved generative adversarial model is trained using the visible light and SAR datasets from S1, specifically including the following steps:

[0097] S31: By introducing Wasserstein distance and gradient penalty terms into the CycleGAN model, an improved generative adversarial model is obtained. The specific structure of the improved model generator is as follows: Figure 4 As shown;

[0098] The improved model generator takes an image as input, first passes it through a residual network to extract image features, and then enters four branches. In the first branch, it undergoes reshape and transpose operations. In the second and third branches, it undergoes a reshape operation. The fourth branch does not perform any processing.

[0099] The outputs of the first and second branches are multiplied by matrix. The result is then multiplied by matrix with the output of the third branch after a softmax operation. After a reshape operation, the result is superimposed with the data from the fourth branch. Finally, the result is convolved to obtain the output image.

[0100] The optimization objective of the entire model is as follows:

[0101]

[0102] Where x, y are the images input to the discriminator, and D A (·),D B (·) represents the discriminator output, G A (·), G B (·) represents the generator output, p r For the true data distribution, p g Generate data distribution for the network. For distribution p r and p g The Wasserstein distance between them, C, is the gradient of the discriminator network. The regularization term corresponding to the gradient of the discriminator network, with parameter λ. i It is the coefficient of the gradient penalty term: ~p k For the data sampling points along the line connecting the sampling points of the real data distribution and the network-generated data distribution, ·~p dt (·) represents the probability distribution of the input data ·;

[0103] S32: After the model is improved, input the training set from the generative adversarial dataset obtained in step one, and start training;

[0104] S33: The training time depends on the size of the dataset and the complexity of the model. After the model is trained, the performance of the model is evaluated using a test set. The test set contains visible light images. These images are input into the trained model, and the generator obtains simulated SAR images.

[0105] S34: Observe the simulated SAR images generated by the generator, compare the generated simulated SAR images with real SAR images, and evaluate the model's performance on the test set.

[0106] S35: If there is a significant difference or low quality between the generated image and the target domain image, readjust and retrain the model until the generated image is similar to the target domain image and of good quality, so as to realize the simulation of different landform types in SAR images.

[0107] The optimized model can then be applied to generate images from new remotely sensed visible light images. Ultimately, it allows inputting the original remotely sensed visible light images into the model to obtain SAR simulation images for the corresponding landform categories, thereby enabling the simulation of different landform types within SAR images.

[0108] S4: Use the trained semantic segmentation model to segment the remote sensing image, forming several groups of images with different landforms; step S4 specifically includes the following steps:

[0109] S41: Load the trained, improved semantic segmentation model into the system; this model has learned the segmentation rules of different landform categories in remotely sensed visible light images through a large amount of training data.

[0110] S42: Input a square visible light image of equal length and width into the model for processing;

[0111] S43: The model predicts each pixel in the image and assigns it to a corresponding landform category, which can include mountains, plains, rivers, lakes, etc. By performing pixel-level classification of the image, a landform segmentation map can be obtained, clearly showing the boundaries and distribution of different landforms. Examples of landform classification and segmentation are shown below. Figure 5 As shown.

[0112] S5: The segmented visible light images of different terrains are processed using a generative adversarial model to generate corresponding SAR simulation images of different terrains. Step S5 specifically includes the following steps:

[0113] S51: Load the improved generative adversarial model trained in step S3 into the system;

[0114] S52: The segmented visible light images of each category obtained in step S4 are used as input to the generative adversarial model for image conversion;

[0115] S53: The generative adversarial model encodes the input visible light image, learns the feature representations in the image, converts these features into corresponding features in the SAR image, and generates a synthetic SAR image through a decoder. This yields a SAR simulation image corresponding to the category of the input visible light image. In this way, leveraging the learning capability of the generative adversarial model, ground feature or landscape features in the visible light image are converted into corresponding SAR representations. Examples of simulation images for each category are shown below. Figure 6 As shown.

[0116] S6: The generated SAR simulation images of different landforms are stitched together to obtain a complete simulation image. In step S6, the region boundaries of each SAR simulation area are defined by semantic segmentation of the visible light image according to the landform category, based on the semantic segmentation model in step S4. These landform categories can include mountains, plains, lakes, rivers, etc. The images of each landform category are stitched together to obtain a complete image containing all the original landforms. The stitching formula is as follows:

[0117]

[0118] Where n is the number of different landform categories, is the image corresponding to the i-th landform category, and w i For the weight of the image corresponding to the i-th terrain category, l i To provide the location information of the image corresponding to the i-th landform category within the entire image, the image stitching sample is as follows: Figure 7 As shown.

[0119] S7: Perform a series of image fusion processes on the stitched SAR simulation image obtained in step S6 to obtain the fused SAR simulation image;

[0120] In step S7, an improved lightweight filtering algorithm is used to address the seam problem after splicing, specifically including the following steps:

[0121] S71: Perform radiometric correction on the stitched image. The formula for the radiometric correction filtering algorithm is shown below:

[0122]

[0123] Where g(x,y) is the gray value of the original image, f(x,y) is the gray value of the resulting image, and m g s is the grayscale mean of the original image. g m is the standard deviation of the original image. f s is the grayscale mean of the template image. f The standard deviation of the template image is a∈[0,1], which is adjusted according to the actual image conditions.

[0124] S72: Obtain the semantic segmentation annotations of each image involved in the stitching, and transform the annotations to obtain a grayscale image where black represents boundary lines and white represents the background;

[0125] S73: Expand the black line into an area with a width of 5 pixels, denoted as the area to be processed. The specific method is to draw a 5×5 pixel black rectangle around the points on the boundary line. The formula for this process is as follows:

[0126] R j →R b

[0127] Where j is a point on the boundary line, R j For a 5×5 pixel block centered at point j, R b This represents black pixel blocks of the same size. Observation of the generated simulated SAR image reveals that the width of the seam is 2-3 pixels. Therefore, the area to be processed is a region with a width of 5 pixels, which can cover all areas containing seams and leave enough space for processing.

[0128] S74: Smooths the seam joints. The specific formula is shown below:

[0129]

[0130] Where j is any point in the region to be processed, n is the number of pixels in a 3×3 pixel block centered at that point that overlaps with the region to be processed, and L i Let D be the gray value of the i-th point among these points. j The points are after filtering; the algorithm introduces the tanh function to reduce the increase of the mean by larger values, making it more suitable for the task requirements. The effect before and after removing the seams is as follows. Figure 8 As shown.

[0131] S8: Perform noise simulation processing on the fused SAR simulation image obtained in step S7 to obtain the final result. In step S8, SAR-specific noise is added to the obtained SAR simulation image, and the noise distribution follows the formula below:

[0132] I′(x,y)=I(x,y)+N

[0133] Where I(x,y) are the pixel values ​​in the original image, I'(x,y) are the pixel values ​​in the image after noise has been added, and N is the noise distribution function, the detailed expression of which is as follows:

[0134]

[0135] Among them, w i The weights of N for each distribution i For different noise distributions, n is the total number of distributions; for each N i The formula is as follows:

[0136]

[0137] Where u is the expected mean of this distribution, σ is the standard deviation, and ρ is the correlation coefficient between the two dimensions, ranging from [-1, 1]. The effect before and after adding noise is as follows: Figure 8 As shown.

[0138] After all operations are completed, a complete SAR simulation image corresponding to the visible light image can be obtained. This synthesized image will provide rich information on ground features, including landform categories, object structures, and terrain distribution, and can be widely used in fields such as geological exploration, military reconnaissance, and disaster monitoring.

[0139] Therefore, this invention employs the aforementioned SAR image simulation method based on multi-category landform style transfer. Through accurate semantic segmentation and processing, it can better capture the details and features of different landforms in complex scenes. It can perform individual processing for different landform categories, thereby better simulating the SAR image corresponding to each landform. By stitching these processed images, a complete SAR simulation image corresponding to the visible light image can be reconstructed, thus providing more comprehensive and accurate data information.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A SAR image simulation method based on multi-category landform style transfer, characterized in that: Includes the following steps: S1: Take several remote sensing visible light images and SAR images, divide them into training set and test set respectively, and perform preprocessing; S2: Train the improved semantic segmentation model using the semantic segmentation dataset obtained in step S1; S3: Train the improved generative adversarial model using visible light and SAR datasets; In step S3, the improved generative adversarial model is trained using the visible light and SAR datasets from S1, specifically including the following steps: S31: By introducing the Wasserstein distance and gradient penalty term mechanism into the CycleGAN model, an improved generative adversarial model is obtained. The improved model generator takes an image as input, first passes it through a residual network to extract image features, and then enters four branches. In the first branch, it undergoes reshape and transpose operations. In the second and third branches, it undergoes a reshape operation. The fourth branch does not perform any processing. The outputs of the first and second branches are multiplied by matrix. The result is then multiplied by matrix with the output of the third branch after a softmax operation. After a reshape operation, the result is superimposed with the data from the fourth branch. Finally, the result is convolved to obtain the output image. The optimization objective of the entire model is as follows: in, The image is input to the discriminator. For the discriminator output results, Output the generator results. For the true data distribution, Generate data distribution for the network. For distribution and Wasserstein distance between them It is the gradient of the discriminator network. The regularization term corresponding to the gradient of the discriminator network, with parameters... It is the coefficient of the gradient penalty term. For data sampling points along the line connecting the sampling points of the real data distribution and the network-generated data distribution, For input data The probability distribution it follows; S32: After the model is improved, input the training set from the generative adversarial dataset obtained in step one, and start training; S33: The training time depends on the size of the dataset and the complexity of the model. After the model is trained, the performance of the model is evaluated using a test set. The test set contains visible light images. These images are input into the trained model, and the generator obtains simulated SAR images. S34: Observe the simulated SAR images generated by the generator, compare the generated simulated SAR images with real SAR images, and evaluate the model's performance on the test set. S35: If there is a significant difference or low quality between the generated image and the target domain image, readjust and train the model until the generated image is similar to the target domain image and of good quality, so as to realize the simulation of different landform types in SAR images. S4: Use the trained semantic segmentation model to segment the remote sensing image to form several groups of images with different landforms; S5: The visible light images of several groups of different landforms after segmentation are processed by a generative adversarial model to generate several corresponding SAR simulation images of different landforms. S6: Stitch together the generated SAR simulation images of different landforms to obtain a complete simulation image; S7: Perform a series of image fusion processes on the stitched SAR simulation image obtained in step S6 to obtain the fused SAR simulation image; S8: Perform noise simulation processing on the fused SAR simulation image obtained in step S7 to obtain the final result.

2. The SAR image simulation method based on multi-category landform style transfer according to claim 1, characterized in that: In step S1, the required dataset is based on UAV remote sensing data, which comes from visible light images and SAR images taken by the UAV from the same viewpoint. Specifically, it includes the following steps: S11: Cut all images into squares of equal length and width, and copy the data twice; S12: Take one portion of the data and perform semantic segmentation annotation on the visible light images in this portion of the data. Divide the visible light and SAR images into training and test sets in a 9:1 ratio to form a semantic segmentation dataset. S13: Using the results of semantic segmentation annotation, the visible light images and SAR images in another dataset are segmented into different images according to different landform categories. Each image is then divided into a training set and a test set in a 9:1 ratio, serving as different generative adversarial datasets corresponding to different landform categories.

3. The SAR image simulation method based on multi-category landform style transfer according to claim 2, characterized in that: Step S2 specifically includes the following steps: S21: An attention mechanism is introduced into the DeepLabV3+ model to strengthen the weight of the segmented edge parts during image processing, resulting in an improved semantic segmentation model; S22: After the model is improved, input the training set from the semantic segmentation dataset obtained in S1 to start training; S23: After training is complete, use the test set to evaluate the model's performance. Input the images in the test set into the trained model to obtain the model's image segmentation prediction results. S24: After obtaining the prediction results, compare the prediction results with the true labels, and use evaluation metrics to evaluate the accuracy and generalization ability of the model. S25: Optimize the model based on the test results.

4. The SAR image simulation method based on multi-category landform style transfer according to claim 3, characterized in that: Step S4 specifically includes the following steps: S41: Load the trained, improved semantic segmentation model into the system; S42: Input a square visible light image of equal length and width into the model for processing; S43: The model predicts each pixel in the image and assigns it to the corresponding terrain category.

5. The SAR image simulation method based on multi-category landform style transfer according to claim 4, characterized in that: Step S5 specifically includes the following steps: S51: Load the improved generative adversarial model trained in step S3 into the system; S52: The segmented visible light images of each category obtained in step S4 are used as input to the generative adversarial model for image conversion; S53: The generative adversarial model encodes the input visible light image, learns the feature representation in the image, converts these features into corresponding features in the SAR image, and generates a synthetic SAR image through a decoder.

6. The SAR image simulation method based on multi-category landform style transfer according to claim 5, characterized in that: In step S6, for each category of SAR simulation region, the visible light image is semantically segmented according to the landform category based on the semantic segmentation model in step S4. The images of each landform category are then stitched together to obtain a complete image containing all the original landforms. The stitching formula is as follows: Where n is the number of different landform categories, and is the image corresponding to the i-th landform category. The weights for the image corresponding to the i-th terrain category are... This refers to the location information of the image corresponding to the i-th landform category within the entire image.

7. The SAR image simulation method based on multi-category landform style transfer according to claim 6, characterized in that: In step S7, an improved lightweight filtering algorithm is used to address the seam problem after splicing, specifically including the following steps: S71: Perform radiometric correction on the stitched image. The formula for the radiometric correction filtering algorithm is shown below: in, The grayscale values ​​of the original image. The grayscale value of the resulting image. The grayscale mean of the original image. The standard deviation of the original image. The grayscale mean of the template image. The standard deviation of the template image. Adjust according to the actual situation of the image; S72: Obtain the semantic segmentation annotations of each image involved in the stitching, and transform the annotations to obtain a grayscale image where black represents boundary lines and white represents the background; S73: Expand the black line into an area with a width of 5 pixels, denoted as the area to be processed. The specific method is to draw a 5×5 pixel black rectangle around the points on the boundary line. The formula for this process is as follows: in, For points on the boundary line, For A 5x5 pixel block centered on the point. Represents black pixel blocks of the same size; S74: Smooths the seam joints. The specific formula is shown below: in, For any point in the region to be processed, This represents the number of pixels in a 3×3 pixel block centered at that point that overlap with the area to be processed. For these points, the first The grayscale value of each point The points are the filtered points; the algorithm introduces the tanh function.

8. The SAR image simulation method based on multi-category landform style transfer according to claim 7, characterized in that: In step S8, SAR-specific noise is added to the obtained SAR simulation image. The distribution of the noise follows the formula below: in, These are the pixel values ​​from the original image. The pixel values ​​in the image after noise has been added. The detailed expression for the added noise distribution function is as follows: in, Weights for each distribution, For different noise distributions, n is the total number of distributions; for each The formula is as follows: in, Let this be the expected mean of the distribution. Standard deviation, The correlation coefficient between the two dimensions, with a value range of [value range missing]. .

Images