SAR image generation model, method, device, system and program product

Abstract

The invention discloses an SAR image generation model, method, device, system and program product. The SAR image generation model comprises a generator and a discriminator. The generator comprises a feature extraction layer, a feature fusion layer and an image generation layer; the generator is used for extracting SAR image features from a first SAR input image and a second SAR input image which are input based on different azimuth angles, and generating an SAR target image of a specified azimuth angle according to an azimuth angle control variable; the discriminator comprises a similarity discriminator and an azimuth angle discriminator; and the discriminator is used for discriminating the SAR target image generated by the generator and the corresponding real SAR label image, calculating azimuth angle information of the generated image, and providing an optimization direction for the generator according to a discrimination result. The SAR image generation model provided by the invention can generate a high-quality and high-precision SAR image with a specified azimuth angle.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a SAR image generation model, method, device, system and program product. Background Art

[0002] In synthetic aperture radar (SAR) target recognition and analysis, comprehensive coverage of image data at all azimuths of the target is the key to improving recognition accuracy. However, the actual situation is that SAR images are insufficient, difficult to obtain, and resource-intensive. Even if there are some SAR images, they may be obtained through different imaging conditions (such as bands, platforms, azimuths, etc.). These SAR images lack sufficient target information and are difficult to meet the research needs in the SAR field.

[0003] In order to solve this problem, a variety of studies have been carried out in recent years. There are three main methods for acquiring SAR target images: 1) measured data acquisition, 2) darkroom measurement, and 3) electromagnetic simulation. First of all, measured data acquisition can obtain SAR target images in different actual scenes on different platforms. These acquired data are the most real and effective. Darkroom measurement is similar to it, and can obtain SAR target images with high signal-to-noise ratio and high accuracy in a controlled environment. However, this acquisition method consumes a lot of manpower, material resources and time resources, and the number of SAR target images obtained in each experiment is often limited, which makes it not an economical and effective method to obtain sufficient SAR data. The results of generating omnidirectional SAR images through electromagnetic simulation are related to the accuracy of the three-dimensional model and electromagnetic calculation method. However, the improvement of the accuracy of the three-dimensional model and electromagnetic calculation method is often accompanied by an exponential increase in the amount of calculation, resulting in reduced calculation efficiency. In addition, when each different radar parameter changes, the calculation of the electromagnetic simulation needs to start from scratch, and the prior knowledge of the existing simulation data cannot be used. Therefore, it is very difficult to obtain large-scale, multi-azimuth SAR image data sets using traditional methods.

[0004] Research on generating SAR images based on deep generative models has brought new possibilities. Compared with traditional methods, deep learning generative models can expand SAR / ISAR datasets from limited angles to all-round perspectives by adaptively learning the multi-angle scattering characteristics of targets, thereby solving problems such as incomplete data collection and limited simulation accuracy.

[0005] Generative adversarial networks (GANs) are a widely used generative model that can retain the key texture and detail characteristics of targets in generated images through game-style training between generators and discriminators. In practical applications, variant models such as conditional GANs (CGANs) and auxiliary classification GANs (ACGANs) perform well in controlling conditional variables such as the azimuth angle and target type of generated images, making the generated SAR images more consistent with the distribution of real data.

[0006] In 2017, Song et al. (Q. Song and F. Xu, “Zero-Shot Learning of SAR TargetFeature Space With Deep Generative Neural Networks,” IEEE Geosci. RemoteSens. Lett., vol. 14, no. 12, pp. 2245–2249, Dec. 2017, doi: 10.1109 / LGRS.2017.2758900.) proposed a deep learning-based method that maps direction-invariant features such as category features to two-dimensional space through a fully connected network and combines them with orientation features to input into the generative model. At the same time, an interpreter representing the inverse process of the generative model is trained to output the image as a combination of orientation-invariant features and orientation features. By optimizing the distance between the two-dimensional features input by the generator and output by the interpreter, a stable SAR image generation network is finally obtained. This model integrates category and azimuth features into SAR image GAN generation, and can generate SAR images of targets of specific categories and specific azimuths. At the same time, the use of the interpreter can map unknown images into known feature spaces, and estimate their categories and azimuths based on the distance from known target features. The prediction is still good in zero-sample or single-sample cases. Although the feature space can reflect the similarity between targets, its physical meaning is not intuitive, and the prediction accuracy of the direction of unseen targets is low, and the error distribution has a certain degree of discreteness. Finally, the trained interpreter can achieve an accuracy of 96.8% in the target recognition task, and the error between the estimated azimuth and the true azimuth is 16°.

[0007] In 2018, Zhang et al. (M. Zhang, Z. Cui, X. Wang, and Z. Cao, “DataAugmentation Method of SAR Image Dataset,” in IGARSS 2018 - 2018 IEEEInternational Geoscience and Remote Sensing Symposium, Valencia: IEEE, Jul.2018, pp. 5292–5295. doi: 10.1109 / IGARSS.2018.8518825.) proposed using DCGAN to generate SAR images with specific azimuths. The sparse model is used to linearly combine the missing azimuth images and use two SAR images with known azimuths to expand the data set. After the image is generated, the SAR image is segmented to obtain a binary image. The minimum circumscribed matrix azimuth of the binary image is calculated as the target azimuth, and the image that meets the expected azimuth is output. It assumes that the target features can be accurately reconstructed through the linear combination of sparse bases. However, SAR images have highly nonlinear scattering characteristics, and the image changes at different azimuths are complex. It is difficult to capture these changes using only sparse linear combinations, resulting in distortion of the generated image. At the same time, the model only uses the azimuth discriminator for output image screening, and does not fully utilize the directional information of the SAR image to guide the generator learning, so it is difficult to generate images with a specified azimuth.

[0008] In 2019, Wang et al. (J. Wang, J. Li, B. Sun, and Z. Zuo, “SAR imagesynthesis based on conditional generative adversarial networks,” J. Eng., vol. 2019, no. 21, pp. 8093–8097, Nov. 2019, doi: 10.1049 / joe.2019.0696.) proposed using electromagnetic simulation to assist the GAN model in generating SAR images of specified azimuth angles. It generates omnidirectional simulated images by modeling the real target, and uses the azimuth conditional labels as generator input to learn the relationship between the simulated image and the real image. Although it has a good generation effect, it has the highest computational complexity and is difficult to apply on a large scale because it requires electromagnetic simulation and cGAN training. In the same year, Zhai et al. (J. Zhai, X. Dang, F. Chen, X. Xie, Y. Zhu, and H. Yin, “SAR Image Generation Using Structural Bayesian Deep Generative Adversarial Network,” in 2019 Photonics & Electromagnetics Research Symposium- Fall (PIERS - Fall), Xiamen, China: IEEE, Dec. 2019, pp. 1386–1392. doi:10.1109 / PIERS-Fall48861.2019.9021403.) proposed using Bayesian networks to model the dependencies between continuous azimuths. By combining multiple SAR images of adjacent azimuths, global and local features are extracted from them, and then the generator uses the extracted features to generate images of subsequent azimuths. This method can more effectively capture the changing characteristics of continuous azimuths, but it has high requirements for the data set. If the data interval is too large or discontinuous, the generation effect may be reduced.

[0009] In 2021, Liang et al. (Liang Mohan and Pi Yiming, "Research on Target Data Augmentation of Multi-view SAR Images Based on Generative Adversarial Networks," Master, 10.27005 / d.cnki.gdzku.2021.000850, 2021. [Online]. Available: https: / / link.cnki.net / doi / 10.27005 / d.cnki.gdzku.2021.000850) improved the InfoGAN model to improve the controllability of the azimuth angle of generated SAR images: the least squares loss was used instead of the cross entropy loss, and the Jensen–Shannon (JS) divergence was replaced by the Pearson chi-square divergence. The generation results show that under different noise conditions, the azimuth angle values ​​of the images generated by the model are different, which indicates that the azimuth angle control potential code decoupled by the InfoGAN model can indeed control the azimuth angle of the generated image. However, since the model cannot clearly identify what kind of noise corresponds to the specific angle of the SAR image target, it cannot generate images at specific azimuth angles and still needs further improvement.

[0010] In 2022, Wang et al. (Wang Ruyi et al., "Research on the Construction Method of Multi-angle SAR Dataset of Aircraft Targets Based on Angle Interpolation Simulation," Journal of Radars, vol. 11, no. 04 %K Dataset Construction; Aircraft Target; Image Simulation; Generative Adversarial Network; Self-Attention, Art. no. 04 %K Dataset Construction; Aircraft Target; Image Simulation; Generative Adversarial Network; Self-Attention, 2022.) proposed a self-attention GAN (SAGAN) based on DCGAN for azimuth interpolation of SAR images. Compared with the traditional SAR image generation model, the sine and cosine of the azimuth are introduced as conditional labels in the generator and concatenated with random noise after upsampling; in the discriminator, the image and azimuth information are concatenated to determine whether the image is true or false and whether the azimuth matches. In the same year, Wang et al. (C. Wang et al., “SAR Target Image Generation Method Using Azimuth-Controllable Generative Adversarial Network,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 15, pp. 9381–9397, 2022, doi: 10.1109 / JSTARS.2022.3218369.) improved the GAN generation model, which extracts target features based on two input SAR images and generates a SAR image with an azimuth in the middle of the azimuth of the input image through fusion of multiple residual blocks. At the same time, a similarity discriminator and an azimuth discriminator are designed to distinguish between real and generated images and measure the azimuth difference between the generated image and the target image, respectively, to provide direction for subsequent generation optimization. Although these two methods can complete the data to a certain extent, they still require high-density initial data support, and too large an interval will lead to a decrease in generation quality.

[0011] In 2024, Peng et al. (G. Peng, M. Liu, S. Chen, M. Tao, Y. Li, and M. Xing, “A Directional Generation Algorithm for SAR Image based on Azimuth-GuidedStatistical Generative Adversarial Network,” IEEE Trans. Signal Process., pp.1–16, 2024, doi: 10.1109 / TSP.2024.3502454.) proposed a SAR image with specified azimuth angles and real statistical characteristics by dividing the azimuth angles into adaptive intervals and constructing noise distribution based on the statistical characteristics of actual SAR images. The generative network uses the sequence number of the adaptive azimuth angle interval as a control condition and combines random noise samples to generate images, thereby ensuring that the image is consistent with the specified azimuth angle and the real data characteristics. It introduces statistical characteristic modeling and azimuth adaptive division, and performs outstandingly in the authenticity of the generated image and the accuracy of azimuth control. However, its complexity and reliance on high-quality training data may limit its practical application scenarios, especially under conditions of data scarcity or limited resources.

[0012] In summary, existing research mainly focuses on the generation and attribute control of SAR images, but there are still deficiencies in the accurate generation of azimuth angles. In particular, when the azimuth angle distribution of the dataset is uneven, the SAR images generated by the current method tend to be concentrated in certain angles and lack wide adaptability.

[0013] In practical applications, the azimuth change of SAR images directly affects the scattering characteristics of the target. Therefore, developing a high-precision, azimuthally controllable SAR image generation method not only helps to improve the accuracy of SAR target recognition, but also has important significance for related research work. Summary of the invention

[0014] In view of this, the embodiments of the present invention provide a SAR image generation model, method, device, system and program product to solve the problems existing in the prior art.

[0015] Other features and advantages of the present invention will become apparent from the following detailed description, or may be learned in part by practice of the present invention.

[0016] In order to achieve the above purpose, the embodiment of the present invention provides the following technical solutions:

[0017] According to a first aspect of an embodiment of the present invention, a SAR image generation model is provided, wherein the SAR image generation model is a generative adversarial network (GAC-GAN) with controlled generation angle, and is used to generate a SAR image of a specified azimuth angle;

[0018] The SAR image generation model includes a generator and a discriminator;

[0019] The generator includes a feature extraction layer, a feature fusion layer, and an image generation layer;

[0020] The generator is used to extract SAR image features from a first SAR input image and a second SAR input image inputted at different azimuth angles, and generate a SAR target image at a specified azimuth angle according to an azimuth angle control variable;

[0021] The discriminator includes a similarity discriminator and an azimuth discriminator;

[0022] The discriminator is used to discriminate the SAR target image generated by the generator and the corresponding real SAR label image, calculate the azimuth information of the generated image, and provide an optimization direction for the generator according to the identification result.

[0023] Furthermore, the feature extraction layer is used to receive the first azimuth-based The first SAR input image , based on the second azimuth The second SAR input image and the azimuth control variable Input three parallel input modules , and , extract the first SAR input image The corresponding first target feature , the second SAR input image The corresponding second target feature and the azimuth control variable Corresponding azimuth control features , the formula is defined as: , , ;

[0024] The feature fusion layer is used to combine the first target feature The second target feature and the azimuth control feature Input to the information fusion module , in the first target feature The second target feature and the azimuth control feature Information mapping is performed between them, and the azimuth angle is adaptively obtained as Target mapping information corresponding to the SAR target image , the formula is defined as: ;

[0025] The image generation layer is used to utilize the mapping module The target mapping information Mapping from high-dimensional to two-dimensional image space, the generated azimuth is SAR target image , defined as .

[0026] Furthermore, the similarity discriminator is used to learn the distribution characteristics of the real SAR label image, judge the similarity between the SAR target image generated by the generator and the corresponding real SAR image, and optimize the generator according to the judgment result;

[0027] The similarity discriminator adopts a fully convolutional structure and combines batch normalization and LeakyReLU activation function to enhance feature extraction capability;

[0028] The azimuth discriminator is used to identify the corresponding relationship between the azimuth control variable and the SAR target image feature generated by the generator, and optimize the azimuth accuracy of the generator when generating the image according to the identification result;

[0029] The azimuth discriminator adopts a fully convolutional structure and combines batch normalization and LeakyReLU activation function, wherein the convolution layer adopts deformable convolution to enhance feature extraction capability;

[0030] The azimuth discriminator gradually adjusts the generator weights according to the deviation of the output predicted azimuth from the true azimuth.

[0031] Furthermore, the loss function of the generator and the similarity discriminator is Defined as:

[0032] ,in, is the preset expectation operator, represents the Euclidean distance, represents the real SAR label image, represents the azimuth of the real SAR label image, Representing images The distribution of is the similarity discriminator, is the azimuth discriminator, For the generator;

[0033] The loss function of the azimuth discriminator Defined as: ,in, is the azimuth discriminator, represents the real SAR label image, represents the azimuth of the real SAR label image, represents the Euclidean distance.

[0034] Further, in the feature extraction layer, before the SAR input image is input into the input module, the electromagnetic scattering features of the SAR input image are extracted using the attribute scattering center modulated convolution kernel;

[0035] The attribute scattering center modulated convolution kernel is used to enhance the scattering feature extraction capability of SAR images and improve the target generation accuracy;

[0036] The feature extraction layer adopts a residual connection mechanism.

[0037] Furthermore, the feature fusion layer adopts conditional batch normalization processing;

[0038] The feature fusion layer adopts a residual connection mechanism.

[0039] According to a second aspect of an embodiment of the present invention, a SAR image generation method is provided, the method comprising:

[0040] Training the SAR image generation model as described in any one of the above items using a training data set to obtain a converged SAR image generation model, wherein the training data set includes a first SAR input image and a second SAR input image based on different azimuth angles, an azimuth angle control variable, and a real SAR label image;

[0041] Based on the converged SAR image generation model, a SAR target image of a specified azimuth angle is generated according to the input azimuth angle control parameter.

[0042] According to a third aspect of an embodiment of the present invention, a SAR image generation device is provided, the device comprising: a processor and a memory;

[0043] The memory is used to store one or more program instructions;

[0044] The processor is used to run one or more program instructions to execute the steps of the SAR image generation method as described above.

[0045] According to a fourth aspect of an embodiment of the present invention, a SAR image generation system is provided, the system comprising:

[0046] A generation model training module, used to train the SAR image generation model as described in any one of the above items using a training data set to obtain a converged SAR image generation model through training, wherein the training data set includes a first SAR input image and a second SAR input image based on different azimuth angles, an azimuth angle control variable, and a real SAR label image;

[0047] The SAR image generation module is used to generate a SAR target image of a specified azimuth angle based on the converged SAR image generation model and according to the input azimuth angle control parameters.

[0048] According to a fifth aspect of an embodiment of the present invention, a computer program product is provided, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, and when the program instructions are executed by a computer, the computer implements the steps of a SAR image generation method as described in any one of the above items.

[0049] The embodiments of the present invention provide a SAR image generation model, method, device, system and program product, which use randomly selected image pairs from a data set as generator input, select images in the middle of the image azimuth as labels, reduce the requirements for the azimuth distribution of training data, and improve the model availability. By introducing an azimuth control variable to assist the dual-input SAR image generator, the generator is trained to learn the relationship between the target azimuth and the azimuth of the input image, and use real SAR images to generate images with specified azimuths, thereby generating target images with controllable azimuths and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0051] Figure 1 A schematic diagram of the structure of a SAR image generation model provided by an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of a flow chart of a SAR image generation method provided by an embodiment of the present invention;

[0053] Figure 3 This is a comparison chart of the SAR image generation effects provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0055] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0056] In order to generate a SAR image with a specified azimuth angle, an embodiment of the present invention provides a SAR image generation model, where the SAR image generation model is a generative adversarial network (GAC-GAN) with controlled generation angle, which is used to generate a SAR image with a specified azimuth angle.

[0057] refer to Figure 1 The above SAR image generation model includes a generator and a discriminator. The GAC-GAN model is based on an adversarial training mechanism. The generator and the discriminator compete with each other in game-based training, which makes the generated image gradually approach the real image distribution. The generator generates a SAR target image of the target distribution based on the input information; the discriminator receives the target image and determines whether the input image is generated by the generator.

[0058] Specifically, the generator includes a feature extraction layer, a feature fusion layer, and an image generation layer. The feature extraction layer is used to extract the information required to generate the target azimuth SAR image from the SAR image pair, and generate the azimuth control feature according to the azimuth control variable. Then the feature fusion layer receives the extracted features, fuses them and inputs them into the image generation layer, and finally generates the target azimuth image. The detailed steps are as follows:

[0059] In the feature extraction layer, the received first azimuth angle The first SAR input image , based on the second azimuth The second SAR input image and the azimuth control variable with a value range of [0,1] Input three parallel input modules , and , extract the first SAR input image The corresponding first target feature , the second SAR input image The corresponding second target feature and the azimuth control variable Corresponding azimuth control features , the formula is defined as: , , .

[0060] In addition, in order to better extract the scattering structure and edge features that are more in line with the characteristics of SAR images from the SAR images, the SAR input image is input to the input module. and Before, the electromagnetic scattering features of the SAR input image are extracted by modulating the convolution kernel with the attribute scattering center. At the same time, the residual connection mechanism is introduced in the feature extraction layer to skip some hidden layers in the middle and directly add the input to the output of the subsequent layer to ensure the transmission of key information and alleviate the gradient vanishing problem in the deep network.

[0061] In the feature fusion layer, the first target feature received , the second target feature and azimuth control features Input to the information fusion module , in the first target feature , the second target feature and azimuth control features Information mapping is performed between them, and the azimuth is adaptively obtained and retained as Target mapping information corresponding to the SAR target image , the formula is defined as: .

[0062] In addition, in order to more effectively utilize the azimuth control variables to guide the generation of the specified azimuth mapping information, the present invention uses conditional batch normalization technology in the feature fusion layer. Traditional batch normalization calculates the mean and variance of the features when normalizing the input data at each layer, and then uses these statistical data to standardize the input. Conditional batch normalization combines additional conditional information (such as labels or attributes) to affect the normalization process, so that the generated or output results can more accurately meet specific conditions. The embodiment of the present invention extends the standard batch normalization by introducing conditional variables to calculate the normalization parameters (scaling factor and offset), and the formula is as follows: ,in, is the mean of the batch data, is the variance in the batch data, To prevent small constants with zero denominators, and The scaling and offset parameters are learned based on the conditions, and the normalization parameters will be dynamically adjusted based on the conditions.

[0063] In addition, the residual link technology is also used in the feature fusion layer to help the feature fusion layer better transmit key information.

[0064] In the image generation layer, after obtaining the mapping information required to generate the target azimuth image, the mapping module is used Mapping target information Mapping from high-dimensional to two-dimensional image space, the generated azimuth is SAR target image , defined as .

[0065] Through the above topological structure, the generator can obtain two images with azimuth angles and SAR input image and Mapped to azimuth The ability to obtain SAR target images.

[0066] In the early stage of training, since the generator is not optimized, the generated images are often of poor quality and cannot be used for subsequent tasks. Therefore, the present invention uses adversarial training to continuously optimize the performance of the generator with the discriminator as a guide.

[0067] The discriminator is trained by receiving generated images and real images, and its goal is to classify real images as "real" and generated images as "fake". In this process, the discriminator will gradually learn the distribution characteristics of real images, so that it can effectively detect defects in generated images.

[0068] Specifically, the discriminator includes a similarity discriminator and an azimuth discriminator, which respectively provide optimization directions for the authenticity and azimuth accuracy of the generated image.

[0069] For the similarity discriminator, the similarity discriminator is mainly used to learn the distribution characteristics of the real SAR label image, judge the similarity between the SAR target image generated by the generator and the corresponding real SAR image, and optimize the generator according to the judgment result.

[0070] The above similarity discriminator adopts a fully convolutional structure, using strided convolution instead of pooling layer so that the network can learn its own spatial downsampling, and uses leaky rectified linear unit (LeakyReLU) activation function in all layers.

[0071] Meanwhile, the similarity discriminator uses batch normalization and removes all fully connected layers to support deeper network architectures.

[0072] In the similarity discriminator, the SAR image undergoes a series of convolutions, normalization, activations, and is flattened at the final convolutional layer before being input into a vector multiplication.

[0073] In addition, after updating the parameters of the similarity discriminator, weight clipping is used to ensure that the objective function meets the requirements of the Wasserstein distance.

[0074] The similarity discriminator gradually optimizes the generator by learning the feature distribution of the real image, so that the SAR image generated by the generator is closer to the appearance and structural characteristics of the real image.

[0075] In addition to being as close as possible to the appearance and distribution characteristics of the real image, the SAR target image generated by the embodiment of the present invention should also ensure the accuracy of the generated image at a specific azimuth angle. Therefore, the embodiment of the present invention introduces an azimuth angle identifier.

[0076] The azimuth discriminator is used to identify the correspondence between the input azimuth control variable and the SAR target image features generated by the generator, optimize the azimuth accuracy of the generator when generating images according to the recognition results, and guide the generator to generate images that meet the target azimuth requirements.

[0077] Taking into account the sensitivity of azimuth, the convolution layer of the azimuth discriminator adopts deformable convolution, which can capture the contour and direction information of the target by changing the shape of the convolution kernel.

[0078] The above-mentioned azimuth discriminator also adopts a fully convolutional structure, uses batch normalization and LeakyReLU activation function, and flattens the last convolution layer of the azimuth discriminator before output, avoiding information loss caused by fixed layer feature limitations and improving the adaptability and generalization ability of the network.

[0079] In addition, to further improve the identification accuracy, the predicted azimuth output by the azimuth discriminator will be optimized with the true azimuth through cross entropy loss to quantify the predicted azimuth deviation and gradually adjust the weight of the generator. This end-to-end supervision mechanism enables the generator to gradually adjust the internal feature extraction mode during training and more accurately map the azimuth control variable to the image feature space.

[0080] The training process of the SAR image generation model provided by the embodiment of the present invention is a competition between the generator and the discriminator.

[0081] Specifically, the loss function of the generator and the similarity discriminator is Defined as:

[0082] ,in, is the preset expectation operator, represents the Euclidean distance, represents the real SAR label image, represents the azimuth of the real SAR label image, Representing images The distribution of is the similarity discriminator, is the azimuth discriminator, For the generator.

[0083] The above loss function The first term in indicates that we expect the discriminator to correctly identify real images as “real”; the second term indicates that the discriminator can correctly identify the images generated by the generator as “fake”, thereby providing an optimization signal for the generator; the third term is for the generator to generate images with the desired orientation.

[0084] Loss function for the azimuth discriminator Defined as: ,in, is the azimuth discriminator, represents the real SAR label image, represents the azimuth of the real SAR label image, represents the Euclidean distance.

[0085] The above loss function It aims to minimize the azimuth distance between the real SAR image and the generated SAR image.

[0086] During training, the Adam optimizer is used for both generator and discriminator optimization, with an initial learning rate of 0.0005 and momentum hyperparameters set to 0.5 and 0.999; at the same time, the weight clipping range of the similarity discriminator during training is set to -0.01 to 0.01. In order to balance the adversarial game between the generator and the discriminator, the generator is optimized 10 times every time the discriminator is optimized.

[0087] The SAR image generation model of the present invention achieves dual optimization goals through the cooperation of a similarity discriminator and an azimuth discriminator. The similarity discriminator improves the realism of the generated target image, while the azimuth discriminator ensures the azimuth control accuracy of the generated target image.

[0088] In summary, the SAR image generation model proposed in this invention not only effectively solves the problem of poor image generation quality, but also realizes high-precision control of the target azimuth, so that the images generated by the model have better applicability and consistency. Therefore, the generation angle controlled generative adversarial network GAC-GAN can obtain a more stable SAR image generation training process than other GAN architectures, and can generate SAR target images with more accurate azimuths.

[0089] The converged SAR image generation model is used to generate SAR target images of specified azimuth angles. Figure 3 The comparison chart of SAR target image generation effect is shown. Figure 3 The first column of images is SAR input image 1, the second column of images is SAR input image 2, the third column of images is SAR label image, and the fourth column of images is SAR target image.

[0090] Will Figure 3 The SAR input image 1 and SAR input image 2 in the first row (size 1*128*128, azimuth angle 38° and 52° respectively) are input into the converged SAR image generation model, and the azimuth control variable α=0.7 is given. Ideally, the azimuth angle of the generated image should be The azimuth angle of the SAR target image generated by the model is determined to be 50.1° by the azimuth angle discriminator, with an error of only 2.1°, and the visual effect is highly consistent with the real SAR label image. Figure 3 The other groups of generated results in the same way are similar to the azimuth of the label image. Not only are the visual similarities high, but the azimuth errors of the generated results are all within 5°, which is highly accurate. This shows that the SAR image generation model based on GAC-GAN proposed in this patent can generate SAR images with accurate azimuth angles.

[0091] The SAR image generation model provided by the embodiment of the present invention uses randomly selected image pairs from the data set as generator input, selects images in the middle of the image azimuth as labels, reduces the requirements for the azimuth distribution of the training data, and improves the model usability. By introducing an azimuth control variable to assist the dual-input SAR image generator, the generator is trained to learn the relationship between the target azimuth and the azimuth of the input image, and uses real SAR images to generate images with specified azimuths, thereby generating target images with controllable azimuths and accuracy.

[0092] In addition, an embodiment of the present invention further provides a SAR image generation method, which specifically includes:

[0093] A SAR image generation model as described above is trained using a training data set to obtain a converged SAR image generation model, wherein the training data set includes a first SAR input image and a second SAR input image based on different azimuth angles, an azimuth angle control variable, and a real SAR label image.

[0094] At each stage of training, the generator tries to become better so that its images are more difficult for the discriminator to distinguish as fake; while the discriminator tries to improve its judgment ability so that it can more accurately distinguish between real images and generated fake images, and the two are trained interactively. Ideally, when the training is completed, the images generated by the generator can confuse the discriminator, and the discriminator cannot distinguish between real and fake images. At this time, the training is close to the "Nash equilibrium", the quality of the images generated by the generator is close to the real image, and the discriminator's discrimination ability has reached its limit. Figure 2 A schematic flow chart of a SAR image generation method provided by an embodiment of the present invention is shown.

[0095] Based on the converged SAR image generation model, a SAR target image of a specified azimuth angle is generated according to the input azimuth angle control parameters.

[0096] In addition, an embodiment of the present invention further provides a SAR image generation device, which includes: a processor and a memory; the memory is used to store one or more program instructions; the processor is used to run one or more program instructions to execute the steps of a SAR image generation method as described above.

[0097] In addition, an embodiment of the present invention further provides a SAR image generation system, the system comprising:

[0098] A generation model training module is used to train a SAR image generation model as described above using a training data set to obtain a converged SAR image generation model. The training data set includes a first SAR input image and a second SAR input image based on different azimuth angles, an azimuth angle control variable, and a real SAR label image.

[0099] The SAR image generation module is used to generate a SAR target image of a specified azimuth angle according to the input azimuth angle control parameters based on the converged SAR image generation model.

[0100] In addition, an embodiment of the present invention further provides a computer program product, which includes computer program instructions. When the computer program instructions are executed by a processor, the steps of the SAR image generation method described above are implemented.

[0101] The embodiment of the present invention trains a SAR image generation model through a GAC-GAN algorithm. When the model converges, a SAR target image of a specified azimuth angle can be directly generated through an azimuth control variable, without the need to generate a specific relationship between the image azimuth angle and an input image pair; and a dual SAR input image generator with controllable azimuth angle can solve the problem of unstable generation effect of a GAN model, and the use of azimuth control variables can better help the generator learn feature information required to generate a target image of a specified azimuth angle from an input image pair; in addition, when the GAC-GAN model converges, a large number of target images can be quickly generated without the need to extract SAR image features again.

[0102] In the embodiment of the present invention, the processor may be an integrated circuit chip having the ability to process signals. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, or discrete hardware components. The methods, steps, and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware. The storage medium may be a memory, for example, a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct RAM bus random access memory (DRRAM).The storage medium described in the embodiment of the present invention is intended to include but is not limited to these and any other suitable types of memory. Those skilled in the art should be aware that in one or more of the above examples, the functions described in the present invention can be implemented by a combination of hardware and software. When the software is applied, the corresponding function can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication medium includes any medium that is convenient for transmitting a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access. Although the present invention has been described in detail above with general descriptions and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to it on the basis of the present invention. Therefore, these modifications or improvements made on the basis of not departing from the spirit of the present invention all belong to the scope of protection claimed in the present invention.

[0103] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Those skilled in the art may make some simple modifications, equivalent changes or modifications using the technical contents disclosed above, which all fall within the protection scope of the present invention.

Claims

1. A SAR image generation model, characterized in that: The SAR image generation model is a generative adversarial network (GAC-GAN) with controlled generation angle, which is used to generate SAR images of specified azimuth angles. The SAR image generation model includes a generator and a discriminator; The generator includes a feature extraction layer, a feature fusion layer, and an image generation layer; The generator is used to extract SAR image features from a first SAR input image and a second SAR input image inputted at different azimuth angles, and generate a SAR target image at a specified azimuth angle according to an azimuth angle control variable; The discriminator includes a similarity discriminator and an azimuth discriminator; The discriminator is used to discriminate the SAR target image generated by the generator and the corresponding real SAR label image, calculate the azimuth information of the generated image, and provide an optimization direction for the generator according to the discrimination result.

2. A SAR image generation model according to claim 1, characterized in that: The feature extraction layer is used to extract the received The first SAR input image , based on the second azimuth The second SAR input image and the azimuth control variable Input three parallel input modules , and , extract the first SAR input image The corresponding first target feature , the second SAR input image The corresponding second target feature and the azimuth control variable Corresponding azimuth control features , the formula is defined as: , , ; The feature fusion layer is used to combine the first target feature The second target feature and the azimuth control feature Input to the information fusion module , in the first target feature The second target feature and the azimuth control feature Information mapping is performed between them, and the azimuth angle is adaptively obtained as Target mapping information corresponding to the SAR target image , the formula is defined as: ; The image generation layer is used to utilize the mapping module The target mapping information Mapping from high-dimensional to two-dimensional image space, the generated azimuth is SAR target image , defined as .

3. A SAR image generation model according to claim 1, characterized in that: The similarity discriminator is used to learn the distribution characteristics of the real SAR label image, judge the similarity between the SAR target image generated by the generator and the corresponding real SAR image, and optimize the generator according to the judgment result; The similarity discriminator adopts a fully convolutional structure and combines batch normalization and LeakyReLU activation function to enhance feature extraction capability; The azimuth discriminator is used to identify the corresponding relationship between the azimuth control variable and the SAR target image feature generated by the generator, and optimize the azimuth accuracy of the generator when generating the image according to the identification result; The azimuth discriminator adopts a fully convolutional structure and combines batch normalization and LeakyReLU activation function, wherein the convolution layer adopts deformable convolution to enhance feature extraction capability; The azimuth discriminator gradually adjusts the generator weights according to the deviation of the output predicted azimuth from the true azimuth.

4. A SAR image generation model according to claim 3, characterized in that: The loss function between the generator and the similarity discriminator Defined as: ,in, is the preset expectation operator, represents the Euclidean distance, represents the real SAR label image, represents the azimuth of the real SAR label image, Representing images The distribution of is the similarity discriminator, is the azimuth discriminator, For the generator; The loss function of the azimuth discriminator Defined as: ,in, is the azimuth discriminator, represents the real SAR label image, represents the azimuth of the real SAR label image, represents the Euclidean distance.

5. The SAR image generation model according to claim 2, characterized in that: In the feature extraction layer, before the SAR input image is input into the input module, the electromagnetic scattering features of the SAR input image are extracted using the attribute scattering center modulated convolution kernel; The attribute scattering center modulated convolution kernel is used to enhance the scattering feature extraction capability of SAR images and improve the target generation accuracy; The feature extraction layer adopts a residual connection mechanism.

6. A SAR image generation model according to claim 2, characterized in that: The feature fusion layer adopts conditional batch normalization processing; The feature fusion layer adopts a residual connection mechanism.

7. A SAR image generation method, characterized in that: The method comprises: The SAR image generation model according to any one of claims 1 to 6 is trained using a training data set to obtain a converged SAR image generation model, wherein the training data set includes a first SAR input image and a second SAR input image based on different azimuth angles, an azimuth angle control variable, and a real SAR label image; Based on the converged SAR image generation model, a SAR target image of a specified azimuth angle is generated according to the input azimuth angle control parameter.

8. A SAR image generation device, characterized in that: The device comprises: a processor and a memory; The memory is used to store one or more program instructions; The processor is used to run one or more program instructions to execute the steps of the SAR image generation method according to claim 7.

9. A SAR image generation system, characterized in that: The system comprises: A generation model training module, used to train the SAR image generation model according to any one of claims 1 to 6 using a training data set to obtain a converged SAR image generation model, wherein the training data set includes a first SAR input image and a second SAR input image based on different azimuth angles, an azimuth angle control variable, and a real SAR label image; The SAR image generation module is used to generate a SAR target image of a specified azimuth angle based on the converged SAR image generation model and according to the input azimuth angle control parameters.

10. A computer program product, characterized in that The computer program product comprises computer program instructions, which, when executed by a processor, implement the steps of a SAR image generation method according to claim 7.

Images