A multi-channel forward-looking SAR azimuth super-resolution imaging method based on deep learning

By using deep learning-based generative adversarial networks, the problem of low resolution in multi-channel SAR forward-looking imaging is solved, achieving efficient super-resolution imaging, especially in the area near the track, overcoming the difficulties in parameter adjustment and the large amount of computation in traditional methods.

CN116699611BActive Publication Date: 2025-12-12UNIV OF ELECTRONICS SCI & TECH OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310633009.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2025-12-12
Estimated Expiration
2043-05-31

AI Technical Summary

Technical Problem

Existing multi-channel SAR forward-looking imaging technology suffers from low resolution, difficulty in parameter adjustment, and high computational load, making it difficult to achieve high-resolution forward-looking imaging.

Method used

A deep learning-based multi-channel forward-looking SAR azimuth super-resolution imaging method is adopted. By constructing a generative adversarial network, end-to-end mapping is performed using a generator and a discriminator. The method is trained by combining cycle consistency and adversarial loss functions to achieve image super-resolution reconstruction.

Benefits of technology

It improves the resolution of multi-channel SAR forward-looking imaging, especially in the area near the track, solves the problems of difficult parameter adjustment and large computational load, and realizes efficient super-resolution imaging.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116699611B_ABST
    Figure CN116699611B_ABST
Patent Text Reader

Abstract

The application discloses a kind of based on deep learning's multi-channel forward-looking SAR azimuth super-resolution imaging method, first, the original clear image is forward-looking imaging using multi-channel forward-looking SAR imaging algorithm, the image pair data set of original clear image and forward-looking imaging after image is constructed, then network model is built, model training is carried out by adjusting network parameter, model test is carried out using the weight parameter of training completion, and super-resolution imaging result is obtained.The method of the application overcomes the problem of low imaging resolution in the adjacent area of flight path during multi-channel SAR forward-looking imaging, compared with existing super-resolution algorithm, there is no complex parameter adjustment difficulty, the problem of large amount of calculation.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of radar imaging, and particularly relates to a multi-channel forward-looking SAR azimuth super-resolution imaging method based on deep learning. BACKGROUND

[0002] Radar forward-looking imaging has important applications in autonomous landing, autonomous navigation, forward-looking reconnaissance guidance and other fields. However, the existing single base synthetic aperture radar (SAR) or Doppler beam sharpening (DBS) technology has a forward-looking imaging blind area due to the Doppler symmetry blur and the small change in the viewing angle of the forward-looking area. The bistatic SAR can realize the imaging of the forward-looking area of the receiving platform through the separation of the transmitter and receiver, but the imaging autonomy is limited due to the need for the assistance of an external radiation source, and the separation of the transmitter and receiver introduces complex synchronization and motion compensation problems. The multi-channel forward-looking SAR can solve the left-right blur problem in the conventional single base SAR forward-looking imaging through the reception of the echo by multiple azimuth channels on a single platform, but the azimuth resolution is still poor due to the small change in the angle of the target in the front-looking area.

[0003] The document "Lu Jingyue, Zhang Lei, Wang Guanyong. Pre-view multi-channel synthetic aperture radar deblurring imaging method. Journal of electronics and information, 2018, 40(12): 2820-2825" proposes an imaging method of space zero point constraint adaptive beam forming under ideal straight flight path, which uses the spatial degree of freedom of limited array to solve left and right Doppler ambiguity, and realizes pre-view imaging, but the azimuth resolution in the flight direction is still poor. The document "Jiang Yunhui. A forward-looking wide sector high-resolution radar scanning imaging method. Telecommunication technology, 2019, 59(12): 1411-1416" uses single pulse method for imaging in the forward-looking area, but due to the mechanism limitation, only beam sharpening function can be realized, and there is obvious angle flicker problem. The document "Zou Jianwu, Zhu Mingbo, Li Wei, etc. L1 norm regularization and its constraint method for radar azimuth super-resolution. Optoelectronics and control, 2015, 22(08): 33-36+53" establishes L1 norm regularization model under the prior information of sparse target, and the super-resolution imaging of forward-looking area can be realized by solving the model, but there is a problem of difficult selection of regularization parameter. The document "Zhang J, Ghanem B. ISTA-Net: Interpretable optimization-inspired deep network for image compressive sensing. IEEE conference on computer vision and pattern recognition, 2018." proposes an image reconstruction method combining soft threshold iteration algorithm ISTA and deep network structure, which solves the problems of difficult parameter selection and large calculation to a certain extent, but the performance is seriously dependent on the accuracy of the model, and the application is greatly limited. SUMMARY

[0004] To solve the above technical problems, the present application provides a multi-channel forward-looking SAR azimuth super-resolution imaging method based on deep learning, which aims to overcome the problem of low resolution of multi-channel SAR forward-looking imaging, and the problems of difficult parameter adjustment and large calculation of traditional super-resolution algorithm, and to realize multi-channel radar forward-looking super-resolution imaging.

[0005] The technical scheme of the present application is: a multi-channel forward-looking SAR azimuth super-resolution imaging method based on deep learning, the specific steps are as follows:

[0006] A, multi-channel forward-looking SAR imaging;

[0007] Firstly, the echo data of the region to be imaged is obtained.

[0008] The multi-channel radar forward-looking imaging adopts a single-transmit multi-receive channel configuration, and multiple receiving channels receive simultaneously; the transmitting signal is set as a linear frequency modulation pulse, and the echo signals received by multiple channels S echo (y i ,t r ,t a ) are expressed as follows:

[0009]

[0010] Wherein, β0 represents a constant set in advance, K r represents the distance frequency, c represents the speed of light, λ represents the wavelength of the transmitting signal, t r represents the fast time, t a represents the slow time, y i represents the azimuth coordinate of the i-th receiving antenna, w r represents the distance envelope, R(t a ,y i ) represents the two-way distance history.

[0011] The distance history R Tx (t a ) from any point P(x0, y0) in the observed scene to the transmitting antenna and the distance history R Rx (t a ,y i ) to different receiving antennas are expressed as follows:

[0012]

[0013]

[0014] Wherein, v r represents the flight speed of the platform, and h represents the flight height of the platform.

[0015] Then, for a point target, the two-way distance history R(t a ,y i ) is expressed as:

[0016] R(t a ,y i ) = R Rx (t a ,y i ) + R Tx (t a ) (4)

[0017] Secondly, the obtained data is subjected to distance direction pulse compression. The matching function of pulse compression is set as S ref (t r ) = exp(-jπK r tr 2 ), then the pulse compressed signal S c (y i ,t r ,t a ) is expressed as:

[0018]

[0019] where IFFT represents an inverse Fourier transform operator and FFT represents a Fourier transform operator

[0020] Next, synthetic aperture azimuth focusing processing is performed. The range pulse compressed data S c (y i ,t r ,t a ) of each channel is coherently accumulated using a polar coordinate back projection (BP) algorithm, and the i-th channel reconstruction result f bp_i (i,ρ,γ) is expressed as:

[0021]

[0022] where ρ represents a slant range and γ represents an azimuth angle.

[0023] Finally, the reconstruction results of each channel are coherently accumulated to obtain an imaging result f bp (ρ,γ) without Doppler ambiguity, and the expression is as follows:

[0024]

[0025] where M represents the number of channels.

[0026] B, construct a data set;

[0027] First, the scene image is randomly cropped to the same size and used as an X-domain image.

[0028] Then, the method in step A is used to perform multi-channel SAR forward-looking imaging, and Gaussian white noise is added according to actual requirements, and the output is output under the condition of keeping the same size as the X-domain image, and the output result is used as a Y-domain image.

[0029] Repeat the above steps to batch process a certain number of image pairs, and finally randomly divide them into a training set and a test set according to the actual requirements.

[0030] C, build a network model;

[0031] The basic structure of the generative adversarial network is adopted, the end-to-end mapping characteristics are used to realize super resolution, the overall network model is built, and the network is mainly composed of a generator network and a discriminator network.

[0032] The generator network structure comprises an encoder, a converter and a decoder.

[0033] The encoder extracts features from the input image by using three layers of convolutional layers, compresses the whole image into a feature vector, and then converts the feature vector to another image domain through nine residual blocks in the converter.

[0034] The overall generator adopts ReLU as the activation function, increases the random inactivation layer, adopts the instance normalization layer, and introduces the global residual connection.

[0035] The discriminator network structure adopts the PatchGAN mode, traverses the whole image through a 70*70 sliding window, and focuses on the local information in the image.

[0036] The overall network structure is composed of the generator module and the discriminator module in series, and is composed of two reverse symmetrical loops.

[0037] Firstly, the input X-domain image Input_X is converted into a Y-domain image Generated_Y by the generator, and then the reconstructed X-domain image Cyclic_X is obtained by the reverse generator; the difference between the two is represented by the cycle consistency loss; the discriminator D X and D Y is responsible for judging whether the input image conforms to the original sample distribution of the corresponding image domain; the other half of the network performs similar operations symmetrically; the two loops share the generator and the discriminator.

[0038] D, the training model;

[0039] The training model minimizes the loss function, and the loss function of the model includes three aspects of adversarial loss, cycle consistency loss and same mapping loss.

[0040] The adversarial loss is composed of two parts, and for the mapping G A (x):X→Y, the adversarial loss can be represented as:

[0041]

[0042] Wherein, X and Y represent two image domains, G A represents the generator of the mapping G A (x):X→Y, and G BG A G B (x):X→Y, x and y represent images belonging to X domain and Y domain respectively; D X and D Y represent discriminators corresponding to X domain and Y domain respectively; p data (x) and p data (y) represent sample distributions of X domain and Y domain respectively. denotes mathematical expectation; similarly, for mapping G B (y):Y→X, there is a similar adversarial loss L GAN (G B ,D X ,Y,X).

[0043] Cycle consistency loss is introduced, for each image in X domain, it will go through a cycle: x→G A (x)→G B (G A (x)).

[0044] where the difference between x and G B (G A (x)) is the forward cycle consistency loss, similarly, if the input is Y domain image, it is called backward consistency loss; then the cycle consistency loss is represented as:

[0045]

[0046] The same mapping loss is a supplement to the cycle consistency loss, represented as:

[0047]

[0048] In summary, the overall loss function is represented as:

[0049]

[0050] where λ1 and λ2 represent weight parameters, used to control the relative importance of the three loss functions.

[0051] The training code is based on the PyTorch deep learning framework, using the image pool strategy, that is, when updating the discriminator network parameters, not only the new samples generated in this iteration are used, but also a series of historical samples generated by the generator are input into the discriminator network, both accounting for 50%.

[0052] Then, the training set is input into the network for training. The weight parameters are saved once every 5 rounds, and the weight parameters of the current last round are saved in real time. When training, the PyTorch visualization tool visdom is used to monitor the training status in real time.

[0053] E, test model;

[0054] Load the last round of weight parameters and test set data into the network model, and test the test data set, and the result of the network output is the super-resolution imaging result.

[0055] The method of the present application first uses a multi-channel forward-looking SAR imaging algorithm to perform forward-looking imaging on the original clear image, constructs an image pair data set of the original clear image and the forward-looking imaged image, then builds a network model, adjusts the network parameters for model training, uses the trained weight parameters for model testing, and obtains the super-resolution imaging result. The method of the present application overcomes the problem of low imaging resolution in the track adjacent area in multi-channel SAR forward-looking imaging, and compared with the existing super-resolution algorithm, there is no problem of complex parameter adjustment difficulty and large amount of calculation. BRIEF DESCRIPTION OF DRAWINGS

[0056] Figure 1 A flow chart of a multi-channel forward-looking SAR azimuth super-resolution imaging method based on deep learning according to the present application. The geometric model diagram of multi-channel SAR forward-looking imaging in the embodiment of the present application.

[0057] Figure 2 The geometric model diagram of multi-channel SAR forward-looking imaging in the embodiment of the present application.

[0058] Figure 3 The schematic diagram of the original observation scene in the embodiment of the present application.

[0059] Figure 4 The result diagram after forward-looking imaging of the original observation scene in the embodiment of the present application.

[0060] Figure 5 The data set structure diagram in the embodiment of the present application.

[0061] Figure 6 The generator network structure diagram in the embodiment of the present application.

[0062] Figure 7 The discriminator network structure diagram in the embodiment of the present application.

[0063] Figure 8 The overall network structure diagram in the embodiment of the present application.

[0064] Figure 9 The original observation scene, the forward-looking imaging result and the super-resolution imaging result diagram after adding 30dB Gaussian white noise to the forward-looking imaging result in the embodiment of the present application. DETAILED DESCRIPTION

[0065] The present application will be further described below in conjunction with the drawings and embodiments.

[0066] As Figure 1 shown, a flow chart of a multi-channel forward-looking SAR azimuth super-resolution imaging method based on deep learning of the present application, the specific steps are as follows:

[0067] A, multi-channel forward-looking SAR imaging;

[0068] In this embodiment, the geometric configuration of the multi-channel forward-looking SAR is as shown in Figure 2 The multi-channel forward-looking SAR parameters are shown in Table 1:

[0069] Table 1

[0070] Parameter Symbol Value Unit Platform flying height h 5000 m Platform flying speed v r ]]> 400 m / s Pulse width [CAT r ]]> 1 μs Signal bandwidth B 60 MHz Array antenna length [[ L e ]]> 3.00 m Distance direction sampling points <![CDATA[N r ]]> 256 Channel number <![CDATA[N a ]]> 16 Synthetic aperture dimension sampling points <![CDATA[N sa ]]> 128 Pulse repetition frequency PRF 400 Hz Transmit signal wavelength λ 0.0315 m

[0071] As Figure 2 shown, in an xyz space coordinate system, O represents the origin of the coordinate system, and the forward-looking multi-channel radar flies at a speed v r = 400 m / s and a height h = 5000 m along the X-axis direction at a constant speed, forming a synthetic aperture with a length L s = 52.5 m. The center position of the target scene is X c = 5000 m, Y c = 0 m, and Z c = 0, i.e. the angle between the aircraft and the scene center is 45°. On the platform, each channel is uniformly arranged along the Y-axis direction, and the transmission channel T x transmits signals at a pulse repetition frequency (PRF = 400 Hz), and each channel receives signals simultaneously, where R xi represents the i-th receiving channel. In the simulation implementation process of this embodiment, it is assumed that the radar transmits a linear frequency modulation pulse signal with a wavelength λ = 0.0315 m, a pulse width T r = 1 μs, and a bandwidth B = 60 MHz. The number of sampling points in the range direction N r is 256, the number of channels N a is 16, and the number of sampling points in the synthetic aperture dimension N sa is 128.

[0072] First, for the multi-channel forward-looking SAR working mode, the echo data of the region to be imaged is obtained. Then the obtained data is subjected to range direction pulse compression.

[0073] Finally, for the single-channel synthetic aperture echo data, according to equations (2) and (3), (4), the coherent accumulation is performed according to equation (6) using the back projection (BP) algorithm to obtain the azimuth direction focused result of the channel, at this time there is left-right blur in the imaging result. Then, the imaging results of each channel are subjected to coherent accumulation according to equation (7) to obtain the imaging result without blur.

[0074] B, constructing a data set;

[0075] First, the high-resolution SAR image is randomly cropped according to the size of 128x128 to obtain the original observed scene image, that is, the X-domain image, as shown in the result. Figure 3

[0076] Then, it is taken as an observed scene to obtain echoes according to the multi-channel forward-looking SAR working mode, and forward-looking imaging is realized according to step A, and Gaussian white noise with a signal-to-noise ratio of 30 dB is added in the imaging result, and the output is output under the condition of keeping the same size as the X-domain image, and the output result is taken as the Y-domain image, as shown in the result. Figure 4

[0077] The above operation is repeated to obtain 1200 pairs of image pairs in batches, and they are randomly divided into a training set and a test set according to a ratio of 8:2, and the number of the training set and the test set is 960 pairs and 240 pairs respectively. The data set structure diagram is shown in Figure 5

[0078] C, building a network model;

[0079] The basic structure of the generative adversarial network is adopted, and the end-to-end mapping characteristics thereof are used to realize super-resolution, and the overall network model is built, which mainly consists of a generator network and a discriminator network.

[0080] The generator network structure diagram is shown in Figure 6 The generator network structure includes an encoder, a converter and a decoder.

[0081] The encoder uses three convolutional layers (Conv layers) to extract features from the input image, and compresses the whole image into a feature vector; then, the feature vector is converted to another image domain through 9 residual blocks in the converter; finally, the decoder recovers low-level features from the feature vector through the deconvolutional layer (DeConv layer), thereby obtaining the generated image.

[0082] The overall generator uses ReLU as the activation function, increases the random inactivation layer (Dropout layer) to increase the randomness of the network and prevent overfitting, and uses the instance normalization layer (Instance Norm layer) instead of the batch normalization layer commonly used in image classification tasks to maintain the independence between each image instance and accelerate model convergence; at the same time, global residual connection is also introduced to make the network converge faster, the model generalization ability is enhanced, and overfitting is prevented.

[0083] The discriminator network structure is shown in Figure 7 ​​​As shown, the PatchGAN is used to traverse the entire image through a 70x70 sliding window to focus on the local information in the image. Unlike the original PatchGAN structure, the Wassertein distance is used instead of the original loss function, the original Sigmoid function activation layer is removed, and all batch normalization layers (Batch Norm layers) are replaced with instance normalization layers (Instance Norm layers). The entire network has 5 convolution layers (Conv layers), and LeakyReLU is used as the activation function.

[0084] The overall network structure is as shown in Figure 8 The overall model structure connects the generator module and the discriminator module to form two groups of reverse symmetric loops, and shares the generator and the discriminator.

[0085] First, the input X-domain image Input_X is converted into a Y-domain image Generated_Y by the generator, and then the reconstructed X-domain image Cyclic_X is obtained by the reverse generator. The difference between the two is represented by the cycle consistency loss. The discriminator D X and D Y is responsible for judging whether the input image conforms to the original sample distribution of the corresponding image domain. The other half of the network performs similar operations symmetrically.

[0086] D, the training model;

[0087] The goal of training the model is to minimize the loss function, and the loss function of this model includes three aspects: adversarial loss, cycle consistency loss, and same mapping loss.

[0088] Among them, the adversarial loss aims to make the generated image as close as possible to the real data distribution; in order to make the converted image retain the original image information and prevent pattern collapse, the cycle consistency loss is introduced; the same mapping loss is a supplement to the cycle consistency loss, and its purpose is to further constrain the generated image to be consistent with the original input in content.

[0089] The training code is based on the Pytorch deep learning framework. An image pool strategy is adopted, that is, when updating the discriminator network parameters, not only the new samples generated in this iteration are used, but also a series of historical samples generated by the generator are input into the discriminator network, both accounting for 50%, so as to reduce model oscillation, improve the stability of the training process, and the size of the buffer pool is set to 50. The random inactivation layer inactivation ratio of the generator is set to 0.5 to increase the randomness of the network and prevent overfitting. The optimization algorithm adopts the Adam optimization algorithm, and the initial learning rate is maintained at 0.0002 for the first 100 epochs, and is linearly decayed to 0 from the 100th epoch. Batch Size can be modified according to the actual hardware condition, which is set to 1 here, and the number of iterations is set to 200 epochs. The cycle consistency loss weight λ1 is set to 10, and the same mapping loss weight λ2 is set to 0.5. The specific training parameters in the embodiment are shown in Table 2:

[0090] Table 2

[0091]

[0092] The training set is input into the model for training, and a total of 200 rounds of training are performed. The weight parameters are saved once every 5 iterations, and the last round of weight parameters is updated in real time. During training, the PyTorch visualization tool visdom is used to monitor the training status in real time, so as to timely adjust various parameters and avoid time consumption caused by incorrect operation. The adversarial loss, cycle consistency loss and same mapping loss of each round during training can be obtained by formula (8), (9) and (10). After 200 iterations, the loss value is significantly reduced, and the model gradually tends to be stable. The adversarial loss is reduced to 0.3604, the cycle consistency loss is reduced to 0.1385, and the same mapping loss is reduced to 0.05.

[0093] E, test the model;

[0094] The last round of weight parameters and test set data are loaded into the network model, and the test data is input into the network model. The output result of the network is the super-resolution imaging result. The effect comparison is shown in FIG. 8, wherein the original scene image, the forward-looking imaging result and the super-resolution imaging result are shown from left to right. It can be seen that the overall structure of the super-resolution imaging result is basically consistent with the original scene, the edge is clear, the point target is visible, and especially the azimuth resolution of the adjacent area of the flight path is greatly improved. Figure 9

[0095] In summary, the method of the present application overcomes the problem of low imaging resolution in the adjacent area of the flight path in multi-channel SAR forward-looking imaging. Compared with the existing super-resolution algorithm, there is no problem of complex parameter adjustment difficulty and large calculation amount.

[0096] ​Those skilled in the art will appreciate that the embodiments described herein are presented for purposes of illustration and understanding of the principles of the application and should not be construed as limiting the scope of the application to such specifically enumerated embodiments. Various modifications and changes can be made thereto by those skilled in the art without departing from the spirit and principles of the application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the application should be included in the scope of the claims of the application.

Claims

1. A deep learning-based multi-channel forward-looking SAR azimuth super-resolution imaging method, the specific steps being as follows: A. Multi-channel forward-looking SAR imaging; First, obtain echo data of a region to be imaged; The multi-channel radar forward-looking imaging adopts a single-transmitting and multi-receiving channel configuration, and multiple receiving channels simultaneously receive; the transmitting signal is a linear frequency modulation pulse, and the echo signals received by the multiple channels The expression is as follows: ; wherein, denotes a pre-set constant, denotes a range rate, denotes the speed of light, denotes the wavelength of the transmitted signal, denotes a fast time, denotes a slow time, denotes a first denotes an azimuth coordinate of the receiving antenna, denotes a range bin, denotes a two-way range history; Observing any point in the scene Distance history to the transmitting antenna And distance history to different receiving antennas The expression is as follows: ; ; wherein, represents a flight speed of the platform, represents a flight height of the platform; Then for a point target, the two-way range history is represented as: ; Secondly, the obtained data is pulse compressed in distance direction; the matching function of pulse compression is set as The pulse compressed signal is represented as: ; Where IFFT represents an inverse Fourier transform operator, and FFT represents a Fourier transform operator Then, a synthetic aperture azimuth focusing processing is performed; the range pulse compression data of each channel is focused Using a polar coordinate back-projection (BP) algorithm for coherent accumulation, the first channel reconstruction result is represented as: ; wherein denotes the slant range, denotes the azimuth angle; Finally, the reconstruction results of each channel are coherently accumulated to obtain the imaging results without Doppler ambiguity The expression is as follows: ; Where M represents the number of channels; B. Constructing a data set; First, randomly crop the scene image to the same size as the X-domain image; Then, perform multi-channel forward-looking SAR imaging on the X-domain image using the method in step A, and add Gaussian white noise according to actual requirements, and output the result while keeping the same size as the X-domain image, and use the output result as the Y-domain image; Repeat the above steps to obtain a certain number of image pairs through batch processing, and finally randomly divide the training set and the test set according to the actual requirements; C. Building a network model; The basic structure of a generative adversarial network is adopted to realize super-resolution by using its end-to-end mapping characteristics, and the overall network model is built, mainly composed of a generator network and a discriminator network; The generator network structure includes an encoder, a converter, and a decoder; The encoder uses three convolutional layers to extract features from the input image and compresses the entire image into a feature vector; then, the feature vector is converted to another image domain through nine residual blocks in the converter; finally, the decoder recovers low-level features from the feature vector through deconvolution layers to obtain a generated image; The overall generator uses ReLU as the activation function, adds a random inactivation layer, uses an instance normalization layer, and introduces global residual connection; The discriminator network structure adopts the PatchGAN mode, and focuses on the local information in the image by sliding windows traversing the entire image; uses the Wassertein distance instead of the original loss function, removes the original Sigmoid function activation layer, replaces all batch normalization layers with instance normalization layers, and the entire network has five convolutional layers and uses LeakyReLU as the activation function. The overall network structure is a series connection of the generator module and the discriminator module, and is composed of two reverse symmetric loops; First, the input X-domain image Input_X is converted into a Y-domain image Generated_Y by the generator, and then the reconstructed X-domain image Cyclic_X is obtained by the reverse generator; the difference between the two is represented by the cycle consistency loss; the discriminator and is responsible for judging whether the image input therein conforms to the original sample distribution of the corresponding image domain; the other half of the network symmetrically performs similar operations; the two loops share the generator and the discriminator; D. Training the model; The model is trained to minimize the loss function, and the loss function of this model includes three aspects: adversarial loss, cycle consistency loss, and same mapping loss; where the adversarial loss consists of two parts, for the mapping The adversarial loss can be expressed as: ; wherein, and denote two image domains, denotes a generator of the mapping denotes a generator of the inverse mapping and denote images belonging to the domain and domain, respectively; and denote discriminators for the corresponding domain and domain, respectively; and denote sample distributions for the domain and domain, respectively; denotes the mathematical expectation; similarly, for the mapping there is a similar adversarial loss ;​​ Introduce cycle consistency loss, to Each image of the domain will go through a cycle: ; wherein, and The difference between them is the forward cycle consistency loss. Similarly, if the input is a domain image, it is called backward consistency loss. Then the cycle consistency loss is represented as: ; The same mapping loss is a supplement to the cycle consistency loss and is represented as: ; In summary, the overall loss function is represented as: ; wherein, and denote weight parameters for controlling the relative importance of the three loss functions; The training code is based on the PyTorch deep learning framework and uses an image pool strategy, that is, when updating the discriminator network parameters, not only the new samples generated in this iteration are used, but also a series of historical samples generated by the generator are input into the discriminator network, and both occupy 50%; Then, the training set is input into the network for training; the weight parameters are saved once every 5 rounds, and the weight parameters of the last round are saved in real time; the PyTorch visualization tool visdom is used to monitor the training status in real time during training; E. Test the model; Load the last round of weight parameters and the test set data into the network model, and test the test data set, and the output result of the network is the super-resolution imaging result.