Multi-attitude ISAR image generation method based on conditional generative adversarial network
By generating multi-pose ISAR images based on condition-based generation of adversarial networks, the problem of multi-pose imaging of ISAR imaging radar is solved, and the ISAR sample capacity is expanded to support more comprehensive target recognition tasks.
Patent Information
- Application Number
- CN202510058146.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-14
- Publication Date
- 2025-05-06
AI Technical Summary
Existing ISAR imaging radars are difficult to image targets in multiple postures in complex environments, resulting in insufficient sample capacity and affecting target recognition tasks.
A multi-pose ISAR image generation method based on conditional generation adversarial network is adopted. By constructing a conditional generation adversarial network, a generator and discriminator of U-net structure are used to optimize the generator to generate multi-pose ISAR images.
It realizes the generation of ISAR image samples in other poses from ISAR image samples in existing poses, expands the ISAR sample capacity, provides more comprehensive data support, and provides richer data resources for target recognition tasks.
Smart Images
Figure CN119942208A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image generation technology, and in particular to a multi-pose ISAR image generation method based on a conditional generative adversarial network. Background Art
[0002] With the continuous advancement of radar system hardware level and signal processing technology, the existing broadband ISAR imaging radar has a high resolution and can obtain detailed information such as the shape, size, material, etc. of moving targets such as missiles and aircraft. When using ISAR to image a target, the scattering characteristics presented by the target will have great differences at different observation angles. Therefore, the shape of the radar image is complex and the azimuth anisotropy is significant. However, in practical applications, especially in complex environments such as battlefields, facing non-cooperative targets, ISAR can often only image the target at a limited azimuth angle, which leads to the lack of such targets in the angle domain. If the ISAR image samples in the existing posture can be used to generate ISAR image samples in other postures, it is expected to expand the ISAR sample capacity and provide data support for subsequent target recognition and other tasks.
[0003] Due to the lack of open source and sample-rich datasets like MSTAR, there are few research results on multi-pose ISAR image sample generation. It is necessary to provide a method that can achieve end-to-end mapping from ISAR image samples in existing poses to ISAR image samples in other poses, so as to obtain ISAR image samples in different poses. Summary of the invention
[0004] Based on this, it is necessary to provide a multi-pose ISAR image generation method based on a conditional generative adversarial network that can generate multi-pose ISAR images to address the above technical problems.
[0005] A multi-pose ISAR image generation method based on a conditional generative adversarial network, the method comprising: Obtain ISAR image samples under various postures of the cooperative target; the ISAR image samples include a training set; construct a conditional generative adversarial network; the conditional generative adversarial network includes a data preprocessing module and a generative adversarial network module; the generative adversarial network module includes a U-net-based generator and a discriminator; The difference in azimuth between the ISAR image sample obtained by the expected output during the training process and the input ISAR image sample is set in the data preprocessing module; The real ISAR image in the adversarial loss is obtained according to the input ISAR image samples and differences; the zero-order moment and the second-order moment of the input image are calculated according to the image moment method, the angle parameters are designed using the zero-order moment and the second-order moment, and the image angle is constructed according to the angle parameters; the azimuth loss of the generator is set using the image angle; the generator loss function is set using the adversarial loss, the azimuth loss and the pre-set reconstruction loss; The generator of the conditional generative adversarial network is optimized according to the generator loss function to obtain the optimized conditional generative adversarial network; the optimized conditional generative network is trained using the training set to obtain the trained conditional generative adversarial network; and multi-pose ISAR images are generated according to the trained conditional generative adversarial network.
[0006] In one embodiment, the adversarial loss is
[0007] in, represents the real ISAR image, represents the pseudo ISAR image output by the generator, Indicates the attitude rotation generated from the input ISAR image sample α The generator of Represents a discriminator for discriminating the output rotated ISAR image samples.
[0008] In one embodiment, the angle parameters are designed using the zero-order moment and the second-order moment, including: The angle parameters designed using the zero-order moment and the second-order moment include , , ,in, represents the zero-order moment, , , Both represent second-order moments.
[0009] In one embodiment, constructing an image angle according to an angle parameter includes: The image angle is constructed according to the angle parameter .
[0010] In one embodiment, the azimuth loss of the generator is set using the image angle, including: The azimuth loss of the generator is set using the image angle as
[0011] in, represents the real ISAR image, Pseudo ISAR image representing the output of the generator.
[0012] In one embodiment, the preset reconstruction loss is:
[0013] in, represents the real ISAR image, Pseudo ISAR image representing the output of the generator.
[0014] In one embodiment, the generator loss function is set using adversarial loss, azimuth loss, and a preset reconstruction loss, including: The generator loss function is set using adversarial loss, azimuth loss and pre-set reconstruction loss as
[0015] in, Represents resistance to loss, represents the reconstruction loss, Indicates azimuth loss.
[0016] The above-mentioned multi-pose ISAR image generation method based on conditional generative adversarial network first constructs a conditional generative adversarial network; the conditional generative adversarial network includes a data preprocessing module and a generative adversarial network module; the generative adversarial network module includes a generator and a discriminator based on U-net, and an adversarial loss is set to ensure the overall similarity between the generated samples and the real samples. The azimuth loss based on image moments is introduced, and the zero-order moment is designed to reflect the total intensity information of the image, indicating the total amount of scattered energy. The second-order moment describes the geometric distribution characteristics of the image and captures the position distribution changes of the scattering points in space. These moment characteristics have significant differences in ISAR images of different azimuths. By calculating the moment information of the input image, the target azimuth parameters are designed and the target image angle is constructed, so that the generative network generates images that meet the target angle. The angular features of the generated image are calculated based on the image moments to ensure the rationality of the generated samples in the angle domain. The azimuth loss function is directly constructed using the zero-order moment and the second-order moment, so that the network learns the scattering characteristics that are closer to the real physical changes. The azimuth loss based on the image moment is introduced to ensure that the generator can generate ISAR image samples rotated at a specific angle. Finally, the reconstruction loss is set to represent the gap between the generated rotated image and the label, ensuring the basic feature consistency between the generated sample and the input sample. This application combines the azimuth loss based on the image moment through the conditional generative adversarial network, uses the zero-order moment and the second-order moment to describe the scattering energy and geometric distribution characteristics of the ISAR image, constructs the target azimuth parameters and guides the generator to learn the scattering change law of the target angle. The generator extracts the multi-scale features of the target through the U-net structure, and combines the adversarial loss, azimuth loss and reconstruction loss optimization, so that the network can not only generate multi-pose images of the training target, but also generate ISAR images of non-cooperative targets rotated at different angles based on the general scattering characteristics and angle change laws. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 A schematic diagram of a flow chart of a multi-pose ISAR image generation method based on a conditional generative adversarial network in one embodiment; Figure 2 A schematic diagram of the structure of a conditional generative adversarial network in one embodiment; Figure 3 A schematic diagram of a convolutional neural network framework in a data preprocessing module in one embodiment; Figure 4 A schematic diagram of an angle preprocessing network framework in a data preprocessing module in another embodiment; Figure 5 A generator network framework diagram in one embodiment; Figure 6 FIG. 4 is a diagram of a discriminator network framework in one embodiment. DETAILED DESCRIPTION
[0018] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0019] In one embodiment, Figure 1 As shown, a multi-pose ISAR image generation method based on a conditional generative adversarial network is provided, comprising the following steps: Step 102, obtaining ISAR image samples under various postures of the cooperative target; the ISAR image samples include a training set; constructing a conditional generative adversarial network; the conditional generative adversarial network includes a data preprocessing module and a generative adversarial network module; the generative adversarial network module includes a U-net-based generator and a discriminator.
[0020] The structural diagram of the conditional generation adversarial network is as follows Figure 2 As shown, it includes a data preprocessing module and a generative adversarial network module, wherein the convolutional neural network framework in the data preprocessing module is as follows Figure 3 As shown in the figure, the data preprocessing module uses a convolutional neural network to preprocess the input ISAR image samples and the input rotation angle α After the one-hot processing, the data is processed through a fully connected network. The angle preprocessing network framework in the data preprocessing module is as follows: Figure 4 As shown in the figure. The U-net-based generator retains the detailed information of high-resolution ISAR images and supports multi-scale feature extraction to capture the complex scattering changes of angle rotation. The discriminator uses the discriminator structure used in the generative adversarial network to distinguish between generated samples and real samples, and improves the authenticity and angle consistency of generated samples through adversarial training. The generator network framework is shown in the figure. Figure 5 As shown, the discriminator network framework is as follows Figure 6 shown.
[0021] Step 104: setting the azimuth difference between the ISAR image sample obtained as the expected output during the training process and the input ISAR image sample in the data preprocessing module.
[0022] Set the azimuth difference between the input ISAR image and the target output ISAR image. The difference strategy provides clear information about the rotation target azimuth, guiding the generation network to generate samples at different angles. The difference is embedded in the generation network as a conditional input to guide the generator to learn the angle-dependent changes in scattering characteristics. During the training process, the azimuth difference between the expected output ISAR image sample and the input ISAR image sample is set. α , and then the proposed conditional generative adversarial network is trained to obtain a network that can rotate the input ISAR image sample attitude angleα Conditional Generative Adversarial Networks.
[0023] Step 106, obtaining a true ISAR image in the adversarial loss according to the input ISAR image sample and the difference; calculating the zero-order moment and the second-order moment of the input image according to the image moment method, designing the angle parameters using the zero-order moment and the second-order moment, and constructing the image angle according to the angle parameters; setting the azimuth loss of the generator using the image angle; setting the generator loss function using the adversarial loss, the azimuth loss and the pre-set reconstruction loss.
[0024] This application ensures the overall similarity between the generated samples and the real samples by setting the adversarial loss. More importantly, in this application, the azimuth loss based on the image moment is introduced, and the zero-order moment is designed to reflect the total intensity information of the image, indicating the total amount of scattered energy. The second-order moment describes the geometric distribution characteristics of the image and captures the position distribution changes of the scattering points in space. These moment characteristics have significant differences in ISAR images at different azimuths. By calculating the moment information of the input image, the target azimuth parameters are designed and the target image angle is constructed, so that the generation network generates images that meet the target angle. The angular features of the generated image are calculated based on the image moment to ensure the rationality of the generated samples in the angle domain. The azimuth loss function is directly constructed using the zero-order moment and the second-order moment, so that the network learns the scattering characteristics that are closer to the real physical changes. By introducing the azimuth loss based on the image moment, it is ensured that the generator can achieve the generation of ISAR image samples rotated at a specific angle. Finally, the reconstruction loss is set to represent the gap between the generated rotated image and the label, ensuring the basic feature consistency between the generated sample and the input sample.
[0025] Step 108, optimizing the generator of the conditional generative adversarial network according to the generator loss function to obtain an optimized conditional generative adversarial network; training the optimized conditional generative network using a training set to obtain a trained conditional generative adversarial network; and generating a multi-pose ISAR image according to the trained conditional generative adversarial network.
[0026] The ISAR image samples of the same cooperative target in different postures are used as the training set and validation set of the network. The azimuth loss based on image moments is introduced to ensure that the generator can generate ISAR image samples rotated at a specific angle. The conditional generative adversarial network obtained by training has a good effect on the cooperative target used in training, and can also generate ISAR images rotated at different angles for non-cooperative targets that are not involved in training.
[0027] The above-mentioned multi-pose ISAR image generation method based on conditional generative adversarial network first constructs a conditional generative adversarial network; the conditional generative adversarial network includes a data preprocessing module and a generative adversarial network module; the generative adversarial network module includes a generator and a discriminator based on U-net, and an adversarial loss is set to ensure the overall similarity between the generated samples and the real samples. The azimuth loss based on image moments is introduced, and the zero-order moment is designed to reflect the total intensity information of the image, indicating the total amount of scattered energy. The second-order moment describes the geometric distribution characteristics of the image and captures the position distribution changes of the scattering points in space. These moment characteristics have significant differences in ISAR images of different azimuths. By calculating the moment information of the input image, the target azimuth parameters are designed and the target image angle is constructed, so that the generative network generates images that meet the target angle. The angular features of the generated image are calculated based on the image moments to ensure the rationality of the generated samples in the angle domain. The azimuth loss function is directly constructed using the zero-order moment and the second-order moment, so that the network learns the scattering characteristics that are closer to the real physical changes. The azimuth loss based on the image moment is introduced to ensure that the generator can generate ISAR image samples rotated at a specific angle. Finally, the reconstruction loss is set to represent the gap between the generated rotated image and the label, ensuring the basic feature consistency between the generated sample and the input sample. This application combines the azimuth loss based on the image moment through the conditional generative adversarial network, uses the zero-order moment and the second-order moment to describe the scattering energy and geometric distribution characteristics of the ISAR image, constructs the target azimuth parameters and guides the generator to learn the scattering change law of the target angle. The generator extracts the multi-scale features of the target through the U-net structure, and combines the adversarial loss, azimuth loss and reconstruction loss optimization, so that the network can not only generate multi-pose images of the training target, but also generate ISAR images of non-cooperative targets rotated at different angles based on the general scattering characteristics and angle change laws.
[0028] In one embodiment, the adversarial loss is
[0029] in, represents the real ISAR image, represents the pseudo ISAR image output by the generator, Indicates the attitude rotation generated from the input ISAR image sample α The generator of Represents a discriminator for discriminating the output rotated ISAR image samples.
[0030] In a specific embodiment, the generator hopes that the generated false ISAR images can be identified as real ISAR images by the discriminator; conversely, the discriminator hopes to correctly identify the input real and fake ISAR images. Therefore, during the training process, the discriminator is optimized by calculating the cross entropy between the prediction results of the real image and the all-1 matrix, and the cross entropy between the prediction results of the generated image and the all-0 matrix. It enables the generator to generate real ISAR samples. It enables the discriminator to better distinguish the generated ISAR samples from the real ISAR samples. The generation ability of the former and the discrimination ability of the latter are gradually improved during training. The reason for this is that the loss needs to be amplified, which can better optimize the two networks and avoid the problem of gradient disappearance. In addition, during the training process, the generator and the discriminator are not optimized at the same time, but are trained and optimized alternately.
[0031] In one embodiment, the angle parameters are designed using the zero-order moment and the second-order moment, including: The angle parameters designed using the zero-order moment and the second-order moment include , , ,in, represents the zero-order moment, , , Both represent second-order moments.
[0032] In one embodiment, constructing an image angle according to an angle parameter includes: The image angle is constructed according to the angle parameter .
[0033] In one embodiment, the azimuth loss of the generator is set using the image angle, including: The azimuth loss of the generator is set using the image angle as
[0034] in, represents the real ISAR image, Pseudo ISAR image representing the output of the generator.
[0035] In one embodiment, the preset reconstruction loss is:
[0036] in, represents the real ISAR image, Pseudo ISAR image representing the output of the generator.
[0037] In one embodiment, the generator loss function is set using adversarial loss, azimuth loss and a preset reconstruction loss, including: The generator loss function is set using adversarial loss, azimuth loss and pre-set reconstruction loss as
[0038] in, Represents resistance to loss, represents the reconstruction loss, Indicates azimuth loss.
[0039] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0040] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM). In addition, for trained models, GPUs are often still needed for calculations during actual use. GPUs (graphics processing units) have powerful parallel computing capabilities, which can greatly improve the speed and efficiency of model operations, meet the requirements for response time, processing power, etc. in actual use, and thus ensure that the entire application process can run smoothly and efficiently.
[0041] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0042] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.
Claims
1. A multi-pose ISAR image generation method based on conditional generative adversarial network, characterized in that: The method comprises: Acquire ISAR image samples under various postures of the cooperative target; the ISAR image samples include a training set to construct a conditional generative adversarial network; the conditional generative adversarial network includes a data preprocessing module and a generative adversarial network module; the generative adversarial network module includes a U-net-based generator and a discriminator; The data preprocessing module sets the difference in azimuth between the ISAR image sample obtained as the expected output during the training process and the input ISAR image sample; According to the input ISAR image samples and differences, a real ISAR image in the adversarial loss is obtained; according to the image moment method, the zero-order moment and the second-order moment of the input image are calculated, the angle parameters are designed using the zero-order moment and the second-order moment, and the image angle is constructed according to the angle parameters; the azimuth loss of the generator is set using the image angle; the generator loss function is set using the adversarial loss, the azimuth loss and the pre-set reconstruction loss; The generator of the conditional generative adversarial network is optimized according to the generator loss function to obtain an optimized conditional generative adversarial network; the optimized conditional generative network is trained using the training set to obtain a trained conditional generative adversarial network; and a multi-pose ISAR image is generated according to the trained conditional generative adversarial network.
2. The method according to claim 1, characterized in that The adversarial loss is in, represents the real ISAR image, represents the pseudo ISAR image output by the generator, Indicates the attitude rotation generated from the input ISAR image sample α The generator of Represents a discriminator for discriminating the output rotated ISAR image samples.
3. The method according to claim 1, characterized in that The angle parameters are designed by using the zero-order moment and the second-order moment, including: The angle parameters designed using the zero-order moment and the second-order moment include , , ,in, represents the zero-order moment, , , Both represent second-order moments.
4. The method according to claim 3, characterized in that Constructing an image angle according to the angle parameter, including: The image angle is constructed according to the angle parameter: 。 5. The method according to claim 4, characterized in that The image angle is used to set the azimuth loss of the generator, including: Using the image angle to set the generator's azimuth loss is in, represents the real ISAR image, Pseudo ISAR image representing the output of the generator.
6. The method according to claim 1, characterized in that The pre-set reconstruction loss is: in, represents the real ISAR image, Pseudo ISAR image representing the output of the generator.
7. The method according to claim 1, characterized in that The generator loss function is set using the adversarial loss, the azimuth loss and the preset reconstruction loss, including: The generator loss function is set using the adversarial loss, azimuth loss and pre-set reconstruction loss as follows: in, Represents resistance to loss, represents the reconstruction loss, Indicates azimuth loss.