A high-resolution radar forward-looking imaging method based on a generative adversarial network

Abstract

The application discloses a high-resolution radar forward-looking imaging method based on a generative adversarial network, and comprises the following steps: pre-processing a high-resolution radar image, obtaining a low-resolution image according to a forward-looking imaging model, and constructing a data set; constructing a generative adversarial network model, including a generator and a discriminator; inputting a to-be-processed image into the generator to obtain pseudo data, inputting the pseudo data and real data into the discriminator for binary classification discrimination; inputting the data set into the generative adversarial network for training, taking adversarial loss, perception loss and the similarity between a generated image and an original image as optimization objectives, alternately updating the discriminator and the generator of the generative adversarial network, and optimizing the generator; and inputting the to-be-processed image into the trained generator to obtain a corresponding high-resolution image. The application can extract the features of a forward-looking image to a greater extent and restore the texture details of a high-resolution image.

Description

Technical Field

[0001] This invention belongs to the field of radar imaging technology, and relates to airborne radar forward-looking imaging signal processing technology, specifically to a high-resolution radar forward-looking imaging method based on generative adversarial networks. Background Technology

[0002] To acquire high-resolution two-dimensional images, moving platform radars often employ Synthetic Aperture Radar (SAR) or Doppler Beam Sharpening (DBS) techniques for imaging. SAR works by transmitting pulse signals at a specific frequency using a small-sized single antenna during the radar platform's movement, and then coherently accumulating the echo signals to form an equivalent long linear array. DBS utilizes the different Doppler frequencies of targets in different azimuths to distinguish them. However, existing SAR and DBS imaging cannot cover the area directly in front of the flight path. This is because the Doppler gradient of the echoes received by SAR and DBS is almost zero directly in front of and below the radar platform's path, leading to a sharp decrease in azimuth resolution. Furthermore, the Doppler center frequencies of echoes from symmetrical targets directly in front and below are the same, causing left-right blurring and creating what is known as an imaging "blind zone." The forward-looking area is precisely a crucial and sensitive region for moving platforms, holding significant importance for blind landings, terminal guidance, and battlefield reconnaissance.

[0003] Real-beam imaging technology can be used for forward-looking imaging. However, its azimuth resolution is limited by the antenna beamwidth and operating range, making high-resolution azimuth imaging impossible. To improve the resolution of real aperture radar (RAR), researchers have proposed many algorithms to enhance azimuth resolution. Currently, the main forward-looking imaging techniques include: real-beam imaging, deconvolution imaging, bistatic SAR forward-looking imaging, monopulse imaging, and array radar super-resolution imaging.

[0004] Deconvolution imaging technology utilizes the convolution of the azimuth direction of the radar echo with the horizontal plane of the antenna pattern and the azimuth scattering points, and the convolution of the range direction of the echo with the vertical plane of the antenna pattern and the range scattering points. Therefore, ideally, the accurate position of the target can be reconstructed by deconvolution in both the range and azimuth directions. However, because the antenna pattern used in the deconvolution process differs from the actual pattern, it is difficult to obtain a precise and stable solution, and the ideal azimuth resolution cannot be achieved.

[0005] In bistatic SAR, the transmitter and receiver are placed on two different platforms during forward-looking imaging, and the equidistant lines and equal Doppler lines are approximately orthogonal, effectively improving the resolution of the forward-looking region. However, this technology is still immature and faces many new theoretical and technical challenges, making it unsuitable for practical applications.

[0006] Monopulse technology utilizes two antennas to simultaneously receive echoes, obtaining the angle of the scattering point's deviation from the beam center using a single pulse. Monopulse technology offers high angle measurement accuracy and fast data acquisition rates. However, when the imaging area has complex terrain and the platform is in motion, the apparent center of the target may deviate from its actual center, resulting in angular flicker. Furthermore, it cannot distinguish between multiple targets within a single beam, so its performance deteriorates drastically when multiple targets are present in the beam.

[0007] To address the above issues, deep learning methods based on neural networks offer a new approach to forward-looking imaging. A generative adversarial network (GAN) is constructed to characterize the features of high-resolution images through supervised learning, and then directly generates high-resolution images in an end-to-end manner. Essentially, the problem is transformed into minimizing the loss function between the predicted and ground truth images. Summary of the Invention

[0008] Purpose of the invention: The technical problem to be solved by the present invention is to address the deficiencies involved in the background technology by providing a high-resolution radar forward-looking imaging method based on generative adversarial networks. The method uses generative adversarial algorithms to explore the complex nonlinear mapping relationship between real data and reconstructed images through network training, and establishes an implicit imaging model to fit the accurate mapping relationship between input and output to the maximum extent.

[0009] Technical Solution: This invention provides a high-resolution radar forward-looking imaging method based on generative adversarial networks, specifically including the following steps:

[0010] (1) Preprocess the high-resolution radar images, obtain low-resolution images based on the forward-looking imaging model, and construct a dataset;

[0011] (2) Construct a generative adversarial network model, including a generator and a discriminator; by inputting the image to be processed into the generator, pseudo data is obtained, and the pseudo data and real data are input into the discriminator for binary classification;

[0012] (3) Input the dataset obtained in step (1) into the generative adversarial network for training. With adversarial loss, perceptual loss and similarity between the generated image and the original image as optimization objectives, alternately update the discriminator and generator of the generative adversarial network and optimize the generator.

[0013] (4) Input the image to be processed into the trained generator to obtain the corresponding high-resolution image.

[0014] Furthermore, the implementation process of step (1) is as follows:

[0015] High-resolution radar images are cropped into high-resolution datasets of uniform size; high-resolution images are convolved with antenna patterns and noise is added to obtain low-resolution datasets.

[0016] Further, the generator described in step (2) is implemented based on the FPN structure, consisting of an input layer, an FPN layer, a head layer, an interpolation and reconstruction module, a residual block, and an output layer. The input layer receives a low-resolution image as input. The FPN layer concatenates feature maps of different scales to obtain rich, multi-scale feature information. Multiple head layers are added above the FPN layer, each responsible for generating a portion of the high-resolution image. The interpolation and reconstruction module concatenates the high-resolution images generated by the head layers and fuses them together through upsampling. The residual block adds the reconstructed image to the input low-resolution image and limits the output to the range [-1, 1] through the tanh activation function. The output layer is a convolutional layer that converts the output of the residual block into the final high-resolution image.

[0017] Furthermore, the discriminator described in step (2) consists of a convolutional layer, a BN layer, and a ReLU activation layer, and finally obtains the discriminant loss through a fully connected layer and a sigmoid activation function.

[0018] Furthermore, the residual block consists of two sets of convolutional layers, BN layers, and activation layers, wherein the activation function is the ReLU function, and the activation layer of the second set is actually an element-wise operation layer; the edge and texture features of the low-resolution image are used as prior information, and the characteristics of the recursive residual network are used to extract edges and textures on the residual branches. Finally, different features are combined and reconstructed through sub-pixel convolutional layers.

[0019] Furthermore, the interpolation reconstruction module includes upsampling, feature map stitching, convolution, normalization, activation function, re-upsampling, and residual blocks; the upsampling operation reconstructs a low-resolution image into a high-resolution image using bilinear interpolation; the feature map stitching operation concatenates feature maps of different scales to form part of the high-resolution image.

[0020] Furthermore, the implementation process of step (3) is as follows:

[0021] Using the PyTorch deep learning platform, the optimization function is Adam, the base learning rate is set to 10e-4, the learning policy step is 100000, and gamma is 0.1. GPU is used for accelerated training. Pairs of high- and low-resolution images in the training set are used as sample inputs. The low-resolution image is input into the generator to obtain the reconstructed high-resolution image. The reconstructed high-resolution image and the real high-resolution image are fed into the perceptual network to obtain the perceptual loss, and finally the generation loss is obtained. The two high-resolution images are fed into the discriminator to obtain the discrimination information, and the discrimination information is backpropagated to the generator for parameter adjustment.

[0022] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention is based on a conditional generative adversarial network with a dual-scale discriminator, introduces a feature pyramid as the core module of the generator, constructs a complex nonlinear mapping relationship between real data and reconstructed images, and introduces perceptual loss to improve the generation effect; the processing results of simulation data and real data obtained show that the imaging effect is significantly enhanced compared with the real beam imaging method. Attached Figure Description

[0023] Figure 1 This is a flowchart of the present invention;

[0024] Figure 2 This is a schematic diagram of the generator network structure;

[0025] Figure 3 This is a schematic diagram of the discriminator network structure;

[0026] Figure 4 For use in simulated imaging of ground scenes;

[0027] Figure 5 This is a diagram of the received forward-looking imaging echo signal;

[0028] Figure 6 This is an image showing the result of imaging a simulated ground scene using the method proposed in this invention.

[0029] Figure 7 This is an image of the measured data;

[0030] Figure 8 This is the result of imaging the measured data using the method proposed in this invention. Detailed Implementation

[0031] The present invention will now be described in further detail with reference to the accompanying drawings.

[0032] This invention provides a high-resolution radar forward-looking imaging technology based on generative adversarial networks, the processing flow of which is as follows: Figure 1 As shown, it includes the following steps:

[0033] Step 1: Preprocess the high-resolution radar images, obtain low-resolution images based on the forward-looking imaging model, and construct a dataset.

[0034] High-resolution radar images are cropped into high-resolution datasets of uniform size; high-resolution images are convolved with antenna patterns and noise is added to obtain low-resolution datasets.

[0035] Step 2: Construct a generative adversarial network model, consisting of two neural networks: a generator and a discriminator, as shown below. Figure 2 , Figure 3 As shown, by inputting the image to be processed into the generator to obtain pseudo-data, the pseudo-data and real data are input into the discriminator for binary classification. With the parameters of the feature extractor fixed, the discriminator and generator of the generative adversarial network are alternately updated with adversarial loss, perceptual loss, and the similarity between the generated image and the original image as optimization objectives, thereby achieving the goal of optimizing the generative model.

[0036] like Figure 2 As shown, the generator is implemented based on the FPN structure, consisting of an input layer, an FPN layer, a head layer, an interpolation and reconstruction module, a residual block, and an output layer. The input layer receives a low-resolution image as input; the FPN layer concatenates feature maps of different scales to obtain rich, multi-scale feature information; multiple head layers are added above the FPN layer, each responsible for generating a portion of the high-resolution image; the interpolation and reconstruction module concatenates the high-resolution images generated by the head layers and fuses them together through upsampling; the residual block adds the reconstructed image to the input low-resolution image and limits the output to the range [-1, 1] using the tanh activation function; the output layer is a convolutional layer that converts the output of the residual block into the final high-resolution image.

[0037] The FPN layer extracts and fuses feature information at different scales through feature pyramids and lateral connections, thereby improving the detail and realism of the generated image. Specifically, the feature pyramid operation extracts multi-scale features from the input image through convolution and downsampling. The implementation uses a predefined model called FPN, where the `num_filters_fpn` parameter specifies the number of output channels for the feature pyramid layer, and the `norm_layer` parameter specifies the normalization layer used. The lateral connection operation fuses feature maps from different levels through cross-scale connections. In the implementation, FPN outputs five feature maps at different scales, each of which is concatenated to subsequent layers. The scale upsampling operation upsamples low-resolution feature maps to high resolution for subsequent generation of more detailed and realistic images.

[0038] The role of the head layer is to generate portions of a high-resolution image at different scales based on feature maps of different scales. By combining multiple head layers, a complete high-resolution image is ultimately generated. Each head layer includes three operations: convolution, normalization, and activation function, used to extract, process, and transform feature information. The convolutional layer extracts features through convolution, where the `num_filters_fpn` parameter specifies 3 input channels, the `num_filters` parameter specifies 3 output channels, the `kernel_size` parameter specifies the kernel size, and the `padding` parameter specifies the padding size to ensure that the output feature map is the same size as the input feature map. The normalization layer normalizes the output of the convolutional layer to accelerate convergence and improve model stability, where the `norm_layer` parameter specifies the normalization layer used. The activation function performs a non-linear transformation on the output of the normalization layer, increasing the model's non-linear fitting ability; in this implementation, the ReLU activation function is used.

[0039] The interpolation and reconstruction module merges feature maps of different scales to generate a high-resolution complete image. This module includes operations such as upsampling, feature map concatenation, convolution, normalization, activation functions, further upsampling, and residual blocks. Upsampling reconstructs a high-resolution image from a low-resolution image using bilinear interpolation. Feature map concatenation concatenates feature maps of different scales to form a portion of the high-resolution image. In practice, this is implemented using the `torch.cat()` function. Convolutional layers perform convolution on the concatenated feature maps to extract richer and more complex feature information. Normalization layers normalize the output of the convolutional layers to accelerate convergence and improve model stability; the `norm_layer` parameter specifies the normalization layer used. Activation functions perform nonlinear transformations on the output of the normalized layers, increasing the model's nonlinear fitting ability; in practice, the ReLU activation function is used. Upsampling further reconstructs the low-resolution image into a high-resolution image.

[0040] Residual blocks, through skip connections, add the input low-resolution image to the high-resolution image after feature extraction, increasing the model's depth, non-linear expressiveness, and learning efficiency, thereby improving the quality of the generated image. Stacking multiple residual blocks further enhances the model's expressive power and improves image reconstruction results. Specifically, convolutional layers perform convolution operations on the output of the previous layer to extract feature information; the `kernel_size` parameter specifies the kernel size, and the `padding` parameter specifies the edge padding size. Normalization layers normalize the output of the convolutional layers to accelerate convergence and improve model stability. Activation functions perform non-linear transformations on the output of the normalization layers, increasing the model's non-linear fitting ability; in this implementation, the ReLU activation function is used. Convolutional layers again perform convolution operations on the output of the previous layer to extract feature information; the `kernel_size` parameter specifies the kernel size, and the `padding` parameter specifies the edge padding size. The normalization layer normalizes the output of the convolutional layers to accelerate convergence and improve model stability. Residual connection: This operation performs a residual connection between the output of the convolutional layer in step 4 and the input, allowing the network to learn better feature representations from the residuals; the activation function performs a non-linear transformation on the output after the residual connection, increasing the model's non-linear fitting ability. In the specific implementation, the ReLU activation function is used.

[0041] Step 3: Input the dataset obtained in Step 1 into the Generative Adversarial Network for training. With adversarial loss, perceptual loss, and the similarity between the generated image and the original image as optimization objectives, alternately update the discriminator and generator of the Generative Adversarial Network and optimize the generator.

[0042] Using the PyTorch deep learning platform, the optimization function is Adam, the base learning rate is set to 10e-4, the learning policy step is 100000, and gamma is 0.1. GPU is used for accelerated training. Pairs of high- and low-resolution images in the training set are used as sample inputs. The low-resolution image is input into the generator to obtain the reconstructed high-resolution image. The reconstructed high-resolution image and the real high-resolution image are fed into the perceptual network to obtain the perceptual loss, and finally the generation loss is obtained. The two high-resolution images are fed into the discriminator to obtain the discrimination information, and the discrimination information is backpropagated to the generator for parameter adjustment.

[0043] The generator's loss function consists of two parts: PerceptualLoss and MeanSquaredError Loss. PerceptualLoss compares the similarity of images at the feature layer, while MeanSquaredError Loss compares the similarity of two images at the pixel level. Finally, the total loss of the two loss functions is calculated using the weight parameters, and this value is returned. The PerceptualLoss class uses a pre-trained feature extraction network to calculate the similarity between images. It includes a contentFunc method to extract image features, which constructs a VGG19 model and truncates its 14th layer (conv3_3), returning the network portion before that layer. Additionally, the initialize method initializes the loss function and sets the type of loss function used. In the get_loss method, the input fakeIm and realIm are transformed to the range [-1, 1] and standardized, then fed into contentFunc to extract features and calculate the loss between fakeIm and realIm.

[0044] The discriminator's loss function first calculates the mean D_fake of the fake samples generated by the forward propagation generator, representing the probability that a fake sample is classified as a real sample. Next, it calculates the mean D_real of the real samples generated by the discriminator, representing the probability that a real sample is classified as a real sample. Combining these two means, the discriminator's loss function, loss_D, is calculated. Additionally, the calc_gradient_penalty method is called to calculate the gradient penalty term. This method interpolates the real samples and the generated fake samples, then calculates the gradient penalty term gradient_penalty by calculating the gradient of the discriminator's output corresponding to the interpolated sample. Finally, the gradient penalty term is added to the discriminator's loss function loss_D to obtain the final discriminator loss.

[0045] Step 4: Input the image to be processed into the trained generator to obtain the corresponding high-resolution image.

[0046] The image to be recovered is input into the generator to obtain a high-resolution image. A high-resolution SAR image is selected as the ground simulation scene, and the target scene is as follows: Figure 4 As shown, this is used to generate radar echoes. Figure 5 For comparison of imaging results of the target scene, Figure 5 The received echo data shows that the angular resolution is low, making it impossible to distinguish point targets, and the outline features of area targets are blurred. Figure 6 The imaging results are as shown by the method proposed in this invention. The measured data are then reconstructed, as shown below.Figure 7 As shown, Figure 8 The image shows the results of imaging the measured data using the method proposed in this invention. As can be seen, the azimuth resolution has been greatly improved, the texture details of the image have been well restored, and the target outline is more distinct, proving the effectiveness of the algorithm.

[0047] This invention establishes a generative adversarial network (GAN), consisting of two neural networks: a generator and a discriminator. By inputting the image to be processed into the generator, pseudo-data is obtained. This pseudo-data and real data are then input into the discriminator for binary classification. With fixed feature extractor parameters, the discriminator and generator of the GAN are alternately updated with adversarial loss, perceptual loss, and the similarity between the generated image and the original image as optimization objectives, thereby optimizing the generator. Based on the mathematical modeling of ground scattering point radar echo signals, this invention uses existing high-resolution SAR images as the simulation scene to simulate the echo signals of a moving platform's forward-looking array radar, establishing a dataset for training and testing a deep learning network for forward-looking imaging, solving the problem of difficulty in obtaining forward-looking imaging datasets and labels. This invention utilizes the characteristic that the data volume of radar forward-looking imaging images is much smaller than that of complex echo data at the same dimension, greatly improving processing speed during image transmission and reconstruction. This invention innovatively applies GANs to radar forward-looking imaging technology, combining simulation data for training, improving the accuracy and stability of image reconstruction, while simultaneously increasing processing speed by reducing data volume, demonstrating high practicality and application value.

Claims

1. A high-resolution radar forward-looking imaging method based on generative adversarial networks, characterized in that, Includes the following steps: (1) Preprocess the high-resolution radar images, obtain low-resolution images based on the forward-looking imaging model, and construct a dataset; (2) Construct a generative adversarial network model, including a generator and a discriminator; By inputting the image to be processed into the generator, pseudo data is obtained. The pseudo data and real data are then input into the discriminator for binary classification. (3) Input the dataset obtained in step (1) into the generative adversarial network for training. With adversarial loss, perceptual loss and similarity between the generated image and the original image as optimization objectives, alternately update the discriminator and generator of the generative adversarial network and optimize the generator. (4) Input the image to be processed into the trained generator to obtain the corresponding high-resolution image; The generator in step (2) is based on the FPN structure and consists of an input layer, an FPN layer, a head layer, an interpolation and reconstruction module, a residual block, and an output layer. The input layer receives a low-resolution image as input. The FPN layer concatenates feature maps of different scales to obtain rich, multi-scale feature information. Multiple head layers are added above the FPN layer, each responsible for generating a portion of the high-resolution image. The interpolation and reconstruction module concatenates the high-resolution images generated by the head layers and fuses them together through upsampling. The residual block adds the reconstructed image to the input low-resolution image and limits the output to the range [-1, 1] using the tanh activation function. The output layer is a convolutional layer that converts the output of the residual block into the final high-resolution image. The FPN layer extracts and fuses feature information of different scales through feature pyramids and lateral connections. The feature pyramid operation extracts multi-scale features of the input image through convolution and downsampling. The lateral connection operation fuses feature maps from different levels through cross-scale connections. Each head layer includes three operations: convolution, normalization, and activation function, used to extract, process, and transform feature information. The residual block consists of two sets of convolutional layers, BN layers, and activation layers, with the activation function being the ReLU function. The activation layer in the second set is actually an element-wise operation layer. The edge and texture features of the low-resolution image are used as prior information. The characteristics of the recursive residual network are used to extract edges and textures on the residual branches. Finally, the different features are combined and reconstructed through sub-pixel convolutional layers. The interpolation reconstruction module includes upsampling, feature map stitching, convolution, normalization, activation function, re-upsampling, and residual blocks. The upsampling operation reconstructs a low-resolution image into a high-resolution image using bilinear interpolation. The feature map stitching operation concatenates feature maps of different scales to form part of the high-resolution image.

2. The high-resolution radar forward-looking imaging method based on generative adversarial networks according to claim 1, characterized in that, The implementation process of step (1) is as follows: High-resolution radar images are cropped into high-resolution datasets of uniform size; high-resolution images are convolved with antenna patterns and noise is added to obtain low-resolution datasets.

3. The high-resolution radar forward-looking imaging method based on generative adversarial networks according to claim 1, characterized in that, The discriminator in step (2) consists of a convolutional layer, a BN layer, and a ReLU activation layer. Finally, the discriminant loss is obtained through a fully connected layer and a sigmoid activation function.

4. The high-resolution radar forward-looking imaging method based on generative adversarial networks according to claim 1, characterized in that, The implementation process of step (3) is as follows: Using the PyTorch deep learning platform, the optimization function is Adam, the base learning rate is set to 10e-4, the learning policy step is 100000, and gamma is 0.

1. GPU is used for accelerated training. Pairs of high- and low-resolution images in the training set are used as sample inputs. The low-resolution image is input into the generator to obtain the reconstructed high-resolution image. The reconstructed high-resolution image and the real high-resolution image are fed into the perceptual network to obtain the perceptual loss, and finally the generation loss is obtained. Two high-resolution images are fed into the discriminator to obtain discrimination information, which is then backpropagated to the generator for parameter adjustment.

Images