Improved CycleGAN SAR satellite image simulation generation method and system

Abstract

The invention relates to the technical field of satellite image processing, and discloses an improved CycleGAN SAR satellite image simulation generation method and system, and the method comprises the steps: obtaining a real optical satellite image data set and a real SAR image data set, placing the real optical satellite image data set and the real SAR image data set in a source domain and a target domain respectively, carrying out the preprocessing of the two domain images, and dividing the two domain images into a training set, a verification set and a test set; constructing an improved CycleGAN model of a forward and reverse mapping relation between the optical image and the synthetic aperture radar image, and integrating the improved adversarial loss and cyclic consistency loss to generate a total loss function of the model; performing unsupervised adversarial training on the model based on the improved loss function, and alternately updating parameters of a generator and a discriminator until the model converges; and inputting an optical image to be converted into the trained improved cyclic generative adversarial network CycleGAN model, outputting a simulated SAR image and performing post-processing enhancement, thereby solving the contradiction between the limitation of SAR satellite image acquisition and abundant optical image resources, and providing support for the application depending on the SAR image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of satellite image processing, and particularly relates to a SAR satellite image simulation generation based on an improved CycleGAN and a system. BACKGROUND

[0002] Synthetic Aperture Radar (SAR) images have been widely used in military reconnaissance, topographic mapping, geological exploration, and ocean monitoring due to their all-weather, all-day, and strong penetration advantages. However, obtaining SAR images usually requires specialized radar equipment, which not only has high costs but also has certain limitations on data acquisition. In addition, when conducting reconnaissance and surveillance on targets, due to the limitations of resolution and possible countermeasures taken by the enemy, it becomes relatively difficult to obtain a large number of clear and complete target images.

[0003] In contrast, optical satellite images are relatively easy and inexpensive to obtain, and there are abundant data resources. Compared with SAR images, the collection and interpretation process of optical remote sensing images is simpler and more intuitive, which mainly relies on sunlight for passive imaging, so it needs to be carried out in sunny and clear weather. Although there are differences in imaging mechanisms between the two, both can reflect the geometric characteristics of the ground target. The technology of converting optical images into SAR images can to some extent make up for the lack of SAR image data, which has important significance for related research, algorithm development and application verification. Therefore, developing a simulation generation method that can convert optical images into SAR satellite images has important practical significance and wide application potential. However, traditional image conversion techniques often have difficulty in effectively simulating the complex characteristics of SAR images, such as radar echo characteristics and scattering mechanisms, when processing the conversion of optical images to SAR images. This leads to significant limitations in the quality and application value of the converted images, which cannot meet the needs of high-precision and high-reliability application scenarios.

[0004] With the continuous development of deep learning and the continuous improvement of computer power, image generation models based on deep learning have achieved remarkable results in the field of image. In recent years, many image-to-image translation methods based on deep learning have been proposed, and many scholars have made some progress in mapping between SAR data and optical data. In 2014, the generative adversarial network (GAN) was introduced. Due to its powerful ability and flexibility, it is widely used in image translation, super-resolution and style conversion. The existing patent CN118052702A discloses a SAR-to-optical image conversion system based on a multi-layer fusion generative adversarial network, which solves the problem that the generator structure using residual network in the existing SAR-to-optical image conversion method is relatively simple, resulting in imperfect feature extraction and weak correlation between features. Moreover, the coherent speckle noise in SAR images can affect the texture details in the images, weakening the effect of the generated optical images. The existing patent CN113487623A discloses an optical and SAR registration method and system based on cGANs and image conversion, which includes obtaining a pair of SAR images and optical images to be registered; using the trained conditional generative adversarial network to convert the SAR images into synthetic optical image patches; and splicing the synthetic optical image patches to obtain a synthetic optical image. The existing patent CN112819688A discloses a SAR image conversion method and system for converting optical images, which uses a generative adversarial network model and a convolutional neural network to realize denoising and coloring of SAR images, so that the SAR images can be converted into optical images. Through image scaling and slicing processing, the SAR images of large scenes can be quickly converted.

[0005] At present, although the generative adversarial network (GAN) has shown certain potential in image conversion, the conventional GAN still has defects such as unstable training and insufficient details of generated images. In addition, the GAN-based image conversion technology has made some progress in converting SAR images into optical images. Researchers have accumulated rich experience in model structure, loss function design and training strategy, which has continuously improved the conversion effect. At the same time, the conversion of optical images to SAR images is relatively less studied and the technology is not mature, still in the exploratory and development stage. SUMMARY

[0006] The purpose of the present application is to solve the above-mentioned shortcomings in the prior art, and to provide an improved SAR satellite image simulation generation and system based on CycleGAN. The improved CycleGAN is used to convert optical images into SAR satellite images, aiming to solve the contradiction between the limited acquisition of SAR satellite images and the rich resources of optical images, and to provide support for applications relying on SAR images through improved image conversion technology.

[0007] In one aspect, an improved SAR satellite image simulation generation of CycleGAN is provided, comprising the following steps: S1: obtaining real optical satellite image datasets and real SAR image datasets of various scenes, landforms and ground object types, and placing the obtained real optical satellite image datasets and real SAR image datasets in the source domain and the target domain , respectively; S2: constructing an improved CycleGAN model of forward and reverse mapping relationship between optical images and synthetic aperture radar images, and integrating the improved adversarial loss and cycle consistency loss to generate a total loss function of the model; S3: performing unsupervised adversarial training of the model on the training set based on the improved loss function, and updating the generator and discriminator parameters alternately until the model converges; S4: inputting the optical image to be converted into the trained improved CycleGAN model, outputting a simulated SAR image and performing post-processing enhancement Further, in step S2, the improved CycleGAN model includes a generator and and a discriminator and , wherein, the generator and are used for forward and reverse mapping conversion between the source domain and the target domain , so that the source domain image is infinitely approximated to the image in the target domain after mapping. The discriminator is combined for adversarial training to optimize the image conversion effect, and the contour information and content features of the source domain image retained by the generator are constrained through the cycle consistency loss.

[0008] Further, in step S2, the improvement of the adversarial loss includes: The Wasserstein distance is used in combination with the gradient penalty term GAN-GP to replace the traditional cross-entropy loss to calculate the adversarial loss, and for the generator and the discriminator , the adversarial loss is represented as: wherein, E represents the expected value of the distribution function, and D represents the target domain SAR images in Follows probability distribution , Represents the source domain Optical images in Follows probability distribution , To in real images and generating images Random interpolated images between them The weight coefficients of the gradient penalty term are used to adjust the proportion of the gradient penalty in the overall adversarial loss, so as to constrain the gradient of the discriminator to satisfy the Lipschitz continuity condition.

[0009] Preferably, in step S2, the improvement of the cycle consistency loss includes introducing perceptual loss and style loss, specifically including: The perceptual loss is computed using a pre-trained deep convolutional neural network for the source domain. image Convert to target domain image Then convert back to the source domain image The perception function is expressed as: in, For layers in a deep convolutional neural network, , and The first The number of channels, height, and width of the layer feature map. This is the feature extraction function for a deep convolutional neural network. Meanwhile, the style loss is calculated based on the pre-trained deep convolutional neural network, and is expressed as: in, This represents the Frobenius norm.

[0010] More preferably, in step S2, the integrated total loss function is expressed as: in, For generators and discriminator The method for calculating the counter-loss is the same as that for other purposes. , and To balance the importance of the super parameters of the adversarial loss and the cycle consistency loss, the performance of the model can be optimized by adjusting the super parameters, so that the generated images have better cycle consistency in perception and style while maintaining similarity with the target domain image distribution.

[0011] Further, in step S3, the unsupervised adversarial training includes: S31: configuring reasonable model training super parameters including learning rate and iteration number by Adam optimization algorithm; S32: initializing early stopping counter and setting patience value, comparing the calculated validation set loss with the recorded minimum validation set loss each time: If the calculated validation set loss is greater than or equal to the minimum validation set loss, the counter is accumulated; If the calculated validation set loss is less than the minimum validation set loss, the minimum validation set loss is updated to the calculated validation set loss, and the counter is reset; S33: stop training when the counter reaches the patience value or the training round number reaches the maximum limit, and save the current model parameters, and the model corresponding to the parameters is used as the final improved cycle generative adversarial network CycleGAN model.

[0012] Further, in step S4, the post-processing enhancement includes: A denoising algorithm based on non-local mean filtering is used for the generated SAR image to realize adaptive noise suppression by calculating the similarity weight between image blocks, and to realize speckle noise suppression. Adaptive Gamma correction is used to improve the dynamic range of the image for contrast enhancement.

[0013] On the other hand, an improved SAR satellite image simulation generation system of CycleGAN is provided, comprising: A data acquisition and preprocessing module is used to obtain real optical satellite image data sets and real SAR image data sets of various scenes, landforms and ground object types, and place the obtained real optical satellite image data sets and real SAR image data sets in the source domain and the target domain , pre-process the images of the two domains and divide them into training set, validation set and test set; A model construction module is used to construct an improved cycle generative adversarial network CycleGAN model of the forward and reverse mapping relationship between optical images and synthetic aperture radar images, and to integrate the improved adversarial loss and cycle consistency loss to generate the total loss function of the model. The training optimization module is configured to perform unsupervised adversarial training on the model based on the improved loss function on the training set by alternately updating the generator and discriminator parameters until the model converges. The image generation and enhancement module is configured to input the optical image to be converted into the trained improved CycleGAN model, output a simulated SAR image, and perform post-processing enhancement.

[0014] In addition, a computer readable storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the improved SAR satellite image simulation generation method of CycleGAN.

[0015] Meanwhile, an electronic device is provided, which includes: one or more processors; a storage device configured to store one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the improved SAR satellite image simulation generation method of CycleGAN.

[0016] Compared with the prior art, the present application has the following advantages: The present application uses a generative adversarial network (GAN) architecture through an improved CycleGAN, does not require paired "optical-SAR image pairs", generates scarce SAR images using rich optical image resources, can automatically learn image features, and improves data preparation efficiency by more than 60%; The present application improves the adversarial loss by introducing the Wasserstein distance combined with the gradient penalty term GAN-GP, introduces the perception loss and the style loss to optimize the cycle consistency loss, integrates the improved adversarial loss and the cycle consistency loss, effectively improves the training stability, avoids the gradient vanishing and explosion problem in traditional adversarial training, makes the model better converge, and enhances the feature consistency in the image conversion process, so that the generated image can still maintain good feature integrity after multiple conversions, thereby improving the performance and practicality of the optical image to SAR image conversion method. The present application prevents model overfitting through a dynamic early stopping mechanism, improves the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) of the generated SAR image through post-processing enhancement, and effectively simulates the scattering mechanism and texture characteristics specific to SAR. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, illustrate the application together with the embodiments thereof, and are used to explain the application, and do not constitute a limitation on the application. In the drawings: Figure 1A flow chart of a SAR satellite image simulation generation method of an improved CycleGAN of the present application; Figure 2 An improved CycleGAN model training framework schematic diagram of the present application; Figure 3 An image instance of an optical image of a certain region intercepted according to the present application; Figure 4 An image instance of a SAR image of a certain region intercepted according to the present application; Figure 5 An input optical image example of the present application; Figure 6 An improved CycleGAN of the present application; Figure 5 The SAR image converted from the input image. DETAILED DESCRIPTION

[0018] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative work fall within the scope of protection of the present application.

[0019] The present application obtains real optical satellite images and real SAR images of different styles, and places them in the source domain (X domain) and the target domain (Y domain) respectively. Then, based on the image style conversion idea of the generative adversarial network, a new generative adversarial loss function and a cycle consistency loss function are designed to construct and train an improved CycleGAN model. Then, through data training, the model can establish the forward and reverse mapping relationship between the optical image and the SAR image. Finally, the optical image to be processed is input into the trained improved CycleGAN model for conversion, so as to obtain the corresponding SAR conversion image.

[0020] The specific embodiments of the present application will be described below in combination with the drawings and examples.

[0021] Embodiment 1 The simulation experiment conditions and environment of the present embodiment are as follows: The simulation experiment hardware platform adopted is: the GPU is NVIDIA GeForce RTX 4090D, the size of the display memory is 24GB; the CPU is the 14th generation Intel i9-14900K, the main frequency is 3.2 GHz, and the memory size is 64GB. The simulation experiment software platform adopted is: the operating system is windows 10; the framework is Pytorch2.0.1; the Python version is python 3.8.

[0022] As Figure 1 shown, a technical scheme of a SAR satellite image simulation generation method of an improved CycleGAN provided by the embodiment, aiming at the problem of converting optical images into SAR images through an improved CycleGAN, first acquires real optical images and real SAR images, then designs new generative adversarial loss and cycle consistency loss functions to construct and train an improved CycleGAN model, then establishes the mapping relationship between optical and SAR images through data training, and finally inputs the optical image to be processed into the trained improved CycleGAN model to obtain a SAR converted image, specifically including the following steps: S1: acquiring real optical satellite image datasets and real SAR image datasets of various scenes, landforms and ground object types, and placing the acquired real optical satellite image datasets and real SAR image datasets in the source domain and the target domain , pre-processing the images of the two domains and dividing them into a training set, a validation set and a test set; S2: constructing an improved CycleGAN model of a forward and reverse mapping relationship between optical images and synthetic aperture radar images, and integrating the improved adversarial loss and cycle consistency loss to generate a total loss function of the model; S3: unsupervised adversarial training of the model based on the improved loss function on the training set, and updating the generator and discriminator parameters alternately until the model converges; S4: inputting the optical image to be converted into the trained improved CycleGAN model, outputting a simulated SAR image and performing post-processing enhancement.

[0023] Among them, step S1 includes image data acquisition, preprocessing and dataset making, that is, acquiring optical and SAR image data of various scenes, landforms and ground object types, and pre-processing the images, including cropping, normalization and denoising, to ensure uniform size and data distribution suitable for network training; finally, the pre-processed data is divided into a training set, a validation set and a test set.

[0024] Two cases are set in the simulation experiment. In the first case, five task points are scattered in a region with a size of 1000m*1000m, and the action distance of the task load is 300m. In the second case, five task points are scattered in a region with a size of 1000m*1000m, and the action distance of the task load is 500m.

[0025] In the embodiment, as shown in Figure 3 and 4 , the images are respectively optical images and SAR images of a certain region, and the size is 256*256. The two images are already registered heterogeneous images, and the data set includes training samples and test samples for cutting the above images.

[0026] Then, in step S2, the improved CycleGAN model includes generators and and discriminators and , wherein, the generators and are respectively used for forward and reverse mapping conversion between source domain and target domain , so that the source domain image is infinitely approximated to the image in the target domain after mapping. The discriminators are combined for adversarial training to optimize the image conversion effect, and the contour information and content features of the source domain image kept by the generator are constrained through the cycle consistency loss.

[0027] The CycleGAN model does not need to use a matched data set, i.e., the source domain and the target domain data, and can realize image style transfer without establishing a one-to-one mapping between the training data. The CycleGAN model is essentially composed of two mirror-symmetric networks, and unlike other one-way transfer GANs, the CycleGAN can realize ring-shaped cycle iterative training. The CycleGAN model needs to prepare two different style data sets when realizing image style transfer between two domains. Through the CycleGAN, the image is converted between the source domain (optical image domain) and the target domain (SAR image domain), and the CycleGAN model training framework is as shown in Figure 2 . Specifically, the forward mapping : represents generating a SAR image from an optical image, and the goal is to make the distribution generated by as close as possible to the real SAR image, so that the discriminator cannot distinguish; and the reverse mapping may be regarded as the inverse mapping of , used to generate optical images from SAR images, whose target is consistent with . With the forward mapping and the inverse mapping, the CycleGAN network can be ensured to be trained in a cyclic iteration, i.e. to satisfy the conditions and .

[0028] Specifically, as shown in Figure 2 , the CycleGAN model mainly includes generators and , discriminators and . In the figure, , are images in the domain and the domain respectively. The generator is used to map images from the domain to the domain, and conversely, the generator is used to convert domain images to the domain. Taking the uplink branch in Figure 2 as an example, by training the two generators and , the X domain image is made to infinitely approach the image in the domain after being mapped by , and then combined with the discriminator for adversarial training to further optimize the image conversion effect. Since there is no one-to-one correspondence between the domain and the domain image, the training process is randomly matched, so the CycleGAN uses the cycle consistency loss to constrain the generators to retain the contour information and content features of the source domain image, prevent the generators from contradicting each other, make the model training more stable, speed up the convergence of the model, and enhance the multi-domain image style conversion effect.

[0029] The improved CycleGAN model is implemented by improving the model training loss function. First, the traditional adversarial loss uses a cross-entropy loss function as the calculation method, and the adversarial loss is improved, specifically including: using the Wasserstein distance combined with the gradient penalty term GAN-GP to replace the traditional cross-entropy loss to calculate the adversarial loss, for the generator and the discriminator , the adversarial loss is represented as: wherein,​ denotes the expectation value of the distribution function, denotes the target domain of SAR images subject to a probability distribution , denotes the source domain of optical images subject to a probability distribution , is a random interpolation image between the real image and the generated image (e.g., can be obtained by , where is a random number uniformly distributed over the interval [0,1]), is a weight coefficient of the gradient penalty term, used to adjust the proportion of the gradient penalty in the overall adversarial loss, to constrain the gradient of the discriminator to satisfy the Lipschitz continuity condition.

[0030] The adversarial loss is improved by using the Wasserstein GAN-GP, which can effectively improve the training stability, avoid the gradient vanishing and explosion problem in traditional adversarial training, and make the model converge better.

[0031] Then the cycle consistency loss is optimized. The cycle consistency loss is to ensure that the input image can be infinitely close to the original image after the image conversion period, i.e. the forward cycle loop and the reverse cycle loop . The traditional cycle consistency loss uses the distance between the reconstructed image and the real image to calculate, which is described as follows: wherein, and are used to control the weight of the forward cycle loss and the backward cycle loss, respectively, and represent the reconstructed images of the forward cycle loop and the reverse cycle loop, respectively, denotes the 1-norm.

[0032] This traditional method is that the generator and the discriminator are updated alternately during the training process until the loss of the discriminator approaches 0.5, so as to reach the Nash equilibrium condition. The cycle consistency loss aims to prevent the generator and from overfitting, so as to avoid all optical images being mapped to the same SAR image.

[0033] CycleGAN network does not need to match the dataset in image style transfer, and performs well in texture and color conversion. However, the model collapse phenomenon is prone to occur in the training process. The present application improves the CycleGAN loss function to alleviate the occurrence of this situation, so that the finally generated SAR image has better effect.

[0034] The present application introduces perceptual loss and style loss for improved optimization of cycle consistency loss, so that the generated SAR image is closer to the real image in high-level semantic features, thereby improving the visual quality and semantic consistency. Specifically, it includes: The perceptual loss is calculated using a pre-trained deep convolutional neural network, and for the source domain Image converted to the target domain Image converted back to the source domain Image The perceptual function is represented as: wherein, is a layer of a deep convolutional neural network, , and are the number of channels, height and width of the first layer feature map, is a feature extraction function of a deep convolutional neural network; At the same time, the style loss is calculated based on a pre-trained deep convolutional neural network, and is represented as: wherein, represents the Frobenius norm.

[0035] The optimized cycle consistency loss enhances the feature consistency in the image conversion process, so that the generated image can still maintain good feature integrity after multiple conversions, thereby improving the performance and practicality of the optical image to SAR image conversion method.

[0036] Finally, the above improved losses are combined to obtain the total loss function, and the integrated total loss function is represented as: wherein, is the adversarial loss for the generator and the discriminator , and the calculation method is the same as , and To balance the importance of the adversarial loss and the cycle consistency loss, the performance of the model can be optimized by adjusting these hyperparameters, so that the generated images have better cycle consistency in perception and style while maintaining similarity to the target domain image distribution through the adversarial loss.

[0037] Then the unsupervised adversarial training of step S3 is performed, further comprising: S31: configure reasonable model training hyperparameters including learning rate and iteration number through Adam optimization algorithm; S32: initialize early stopping counter and set patience value, compare the calculated validation set loss with the recorded minimum validation set loss after each calculation: If the calculated validation set loss is greater than or equal to the minimum validation set loss, the counter is incremented; If the calculated validation set loss is less than the minimum validation set loss, update the minimum validation set loss to the calculated validation set loss and reset the counter; S33: stop training when the counter reaches the patience value or the training round number reaches the maximum limit, and save the current model parameters, and the model corresponding to the parameters is used as the final improved cycle generative adversarial network CycleGAN model.

[0038] Specifically, the improved CycleGAN is trained using the preprocessed training sample set. A suitable optimization algorithm (such as Adam) is used to update the network (generator and discriminator) parameters through the total loss function backpropagation. In the training process, reasonable learning rate and iteration number and other hyperparameters are set, and early stopping method (Early Stopping) is adopted to prevent overfitting, and the training is stopped when the validation set loss no longer decreases or reaches the preset round number. This step specifically includes: Step 1: Adam algorithm has several key parameters that need to be reasonably set, mainly including learning rate , and and .

[0039] Step 2: Early stopping method (Early Stopping) is used to prevent overfitting, and a patience value (patience) is set, for example, set to N, indicating that when the validation set loss does not decrease for N consecutive iterations, the training is stopped; At the same time, set the maximum training round number M. In the training process, after calculating the validation set loss each time, compare it with the recorded minimum validation set loss: if the current loss is less than the minimum value, update the minimum value and reset the counter; if the current loss is greater than or equal to the minimum value, the counter is incremented. When the counter reaches the patience value or the training round number reaches the maximum limit, stop training and save the current model parameters, and the model corresponding to these parameters is the final model.

[0040] Throughout the training process, close attention needs to be paid to the training set loss, validation set loss, and the quality of generated images, and the hyperparameters and other settings are flexibly adjusted according to the actual performance, so as to obtain a good performance of the improved CycleGAN model for conversion of optical images to SAR images.

[0041] Finally, step S4 is implemented, SAR image simulation generation, the optical image to be converted is input into the generator G of the trained improved CycleGAN model, and the simulated SAR image is obtained. Through the post-processing enhancement, the image quality is further improved, and the visual effect and application requirement of the real SAR image are closer, and the specific steps include: A denoising algorithm based on non-local mean filtering is used for the generated SAR image, the similarity weight between image blocks is calculated to realize adaptive noise suppression, and speckle noise suppression is realized. Adaptive Gamma correction is used to improve the image dynamic range and contrast enhancement.

[0042] Finally, the image conversion effect test is performed. In this embodiment, the existing image conversion network framework is compared with this embodiment, the pix2pix, GAN and this embodiment are trained using the SAR image dataset without preprocessing; multiple evaluation indexes such as peak signal-to-noise ratio (PSNR), structural similarity index (SSIM) and mean square error (MSE) are used, and the calculation formula is as follows: Wherein, MSE is the mean square error, is the number of columns of the image pixels, is the number of rows of the image pixels, is the first row column pixel of the image , and is the first row column pixel of the image , and the root mean square error (RMSE) is the square root of MSE.

[0043] Secondly, the calculation formula of PSNR is: Wherein, is the maximum value of the image pixel value, for example, the maximum value of the pixel of the 8bit image is 255.

[0044] And the calculation formula of SSIM is: Wherein, is a local region pixel mean value of the image is a local region pixel mean value of the image is a local region pixel mean value of the image is a local region pixel mean value of the image is a local region variance of the image is a local region variance of the image is a local region variance of the image is a local region variance of the image is a local region covariance of the image is a local region covariance of the image is a local region covariance of the image represents a luminance stability constant, represents a contrast stability constant.

[0045] A1, A2, and A3 are respectively recorded by using the same test set for evaluation. Meanwhile, the model of the embodiment is trained by using the preprocessed optical image and SAR image data set, and is evaluated by using the test set, and is recorded as A4. In the embodiment, PSNR>28dB and SSIM>0.85 indicate that the generated image is highly consistent with the real SAR image in the structure, texture, and luminance dimensions.

[0046] Figure 5 , Figure 6 respectively are the input optical image and the corresponding converted SAR image.

[0047] In addition, the embodiment also provides an improved SAR satellite image simulation generation system of CycleGAN, comprising: a data acquisition and preprocessing module, configured to acquire real optical satellite image data sets and real SAR image data sets of various scenes, landforms, and ground object types, and place the acquired real optical satellite image data sets and real SAR image data sets in a source domain and a target domain , pre-process the images in the two domains, and divide the images in the two domains into a training set, a verification set, and a test set; a model construction module, configured to construct an improved CycleGAN model of a forward mapping relationship and a reverse mapping relationship between an optical image and a synthetic aperture radar image, and integrate an improved adversarial loss and a cycle consistency loss to generate a total loss function of the model; a training optimization module, configured to perform unsupervised adversarial training on the model based on the improved loss function on the training set, and update parameters of a generator and a discriminator alternately until the model converges; an image generation and enhancement module, configured to input a to-be-converted optical image into the trained improved CycleGAN model, output a simulated SAR image, and perform post-processing and enhancement.

[0048] It should be noted that the steps of the improved CycleGAN SAR satellite image simulation generation method provided in the embodiments can be implemented based on the corresponding modules in the improved CycleGAN SAR satellite image simulation generation system, and those skilled in the art can refer to the technical solutions of the system to implement the step flow of the method, that is, the embodiments in the system can be understood as preferred examples of implementing the method, and details are not repeated here.

[0049] In addition to implementing the system provided by the present application and each device thereof in a pure computer readable program code manner, the system provided by the present application and each device thereof can also be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. by logically programming the method steps to achieve the same functions. Therefore, the system provided by the present application and each device thereof can be considered as a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component; the devices for implementing various functions can also be considered as both software modules implementing the method and structures within the hardware component.

[0050] Finally, it should be noted that: the above only describes the preferred embodiments of the present application, the protection scope of the present application is not limited to the above-mentioned embodiments, any technical solutions falling within the scope of the present application should be considered as the protection scope of the present application. It should be noted that for ordinary skilled in the art, some improvements and refinements without departing from the principles of the present application, these improvements and refinements should also be considered as the protection scope of the present application.

[0051] The technical features of the above-described embodiments can be combined arbitrarily, and in order to make the description concise, all possible combinations of the technical features in the above-described embodiments are not described, however, as long as the combination of the technical features does not exist contradictory, it should be considered as the scope of the present application.

Claims

1. An improved CycleGAN method for SAR satellite image simulation and generation, characterized in that, Includes the following steps: S1: Acquire real optical satellite image datasets and real SAR image datasets for various scenes, terrains, and land cover types. Place the acquired real optical satellite image datasets and real SAR image datasets into the source domain respectively. and target domain The two-domain images are preprocessed and divided into training, validation and test sets. S2: An improved CycleGAN model is constructed to establish the forward and backward mapping relationship between optical images and synthetic aperture radar images, and the improved adversarial loss and cycle consistency loss are integrated to generate the total loss function of the model. S3: Perform unsupervised adversarial training on the training set based on the improved loss function, and update the generator and discriminator parameters alternately until the model converges; S4: Input the optical image to be converted into the trained improved CycleGAN model, output simulated SAR image and perform post-processing enhancement.

2. The improved CycleGAN SAR satellite image simulation generation method according to claim 1, characterized in that, In step S2, the improved CycleGAN model includes a generator. and and the discriminator and ,in, The generator and Used respectively for the source domain and target domain The forward and reverse mapping transformations between them make the source domain After mapping, the image approximates the target domain infinitely. The image in; Adversarial training is performed in conjunction with the discriminator to optimize the image conversion effect, while the contour information and content features of the source domain image are constrained by the cycle consistency loss.

3. The improved CycleGAN SAR satellite image simulation generation method according to claim 2, characterized in that, In step S2, the improvement against loss includes: The adversarial loss is computed using Wasserstein distance combined with a gradient penalty term in GAN-GP, instead of the traditional cross-entropy loss, for the generator. and discriminator The adversarial loss is expressed as: in, This represents the expected value of the distribution function. Represents the target domain SAR images in Follows probability distribution , Represents the source domain Optical images in Follows probability distribution , To in real images and generating images Random interpolated images between them The weight coefficients of the gradient penalty term are used to adjust the proportion of the gradient penalty in the overall adversarial loss, so as to constrain the gradient of the discriminator to satisfy the Lipschitz continuity condition.

4. The improved CycleGAN SAR satellite image simulation generation method according to claim 3, characterized in that, In step S2, the improvement to the cycle consistency loss includes the introduction of perceptual loss and style loss, specifically including: The perceptual loss is computed using a pre-trained deep convolutional neural network for the source domain. image Convert to target domain image Then convert back to the source domain image The perception function is expressed as: in, For layers in a deep convolutional neural network, , and The first The number of channels, height, and width of the layer feature map. This is the feature extraction function for a deep convolutional neural network. Meanwhile, the style loss is calculated based on the pre-trained deep convolutional neural network, and is expressed as: in, This represents the Frobenius norm.

5. The improved CycleGAN SAR satellite image simulation generation method according to claim 4, characterized in that, In step S2, the integrated total loss function is expressed as: in, For generators and discriminator The method for calculating the counter-loss is the same as that for other purposes. , and These are hyperparameters used to balance the importance of adversarial loss and cycle consistency loss. By adjusting these hyperparameters, the performance of the model can be optimized so that the generated images maintain similarity to the distribution of the target domain images while having better cycle consistency with the original images in terms of perception and style.

6. The improved CycleGAN SAR satellite image simulation generation method according to claim 1, characterized in that, In step S3, the unsupervised adversarial training further includes: S31: Configure appropriate model training hyperparameters, including learning rate and number of iterations, using the Adam optimization algorithm; S32: Initialize the early stop counter, set the patience value, and compare it with the recorded minimum validation set loss after each calculation: If the calculated validation set loss is greater than or equal to the minimum validation set loss, increment the counter. If the calculated validation set loss is less than the minimum validation set loss, update the minimum validation set loss to the calculated validation set loss and reset the counter; S33: When the counter reaches the patience value or the number of training rounds reaches the maximum limit, training is stopped, and the current model parameters are saved. The model corresponding to these parameters is used as the final improved CycleGAN model.

7. The improved CycleGAN SAR satellite image simulation generation method according to claim 1, characterized in that, In step S4, the post-processing enhancement includes: A noise reduction algorithm based on nonlocal mean filtering is used for the generated SAR image. Adaptive noise suppression is achieved by calculating the similarity weight between image patches, thus realizing speckle noise suppression. Adaptive Gamma correction is used to enhance the dynamic range of the image and improve contrast.

8. An improved CycleGAN SAR satellite image simulation and generation system, characterized in that, include: The data acquisition and preprocessing module is used to acquire real optical satellite image datasets and real SAR image datasets for various scenes, landforms, and land cover types. The acquired real optical satellite image datasets and real SAR image datasets are then placed in the source domain. and target domain The two-domain images are preprocessed and divided into training, validation and test sets. The model building module is used to construct an improved CycleGAN model that establishes the forward and backward mapping relationship between optical images and synthetic aperture radar images, and integrates the improved adversarial loss and cycle consistency loss to generate the total loss function of the model. The training optimization module is used to perform unsupervised adversarial training on the training set based on the improved loss function, by alternately updating the generator and discriminator parameters until the model converges. The image generation and enhancement module is used to input the optical image to be converted into the trained improved CycleGAN model, output simulated SAR images, and perform post-processing enhancement.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the SAR satellite image simulation generation method of the improved CycleGAN as described in any one of claims 1-7.

10. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the improved CycleGAN SAR satellite image simulation generation method as described in any one of claims 1-7.

Images