SAR imaging motion compensation method and device based on generative adversarial network
Through the SAR imaging motion compensation method based on the generative adversarial network, the phase error in the SAR image map is learned and converted, and the problem of insufficient phase error compensation in the prior art is solved, and the quality and reliability of SAR imaging are improved.
Patent Information
- Application Number
- CN202510078646.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-17
- Publication Date
- 2025-05-27
AI Technical Summary
In the case of large error in the measurement of platform motion posture, existing SAR imaging technology is difficult to effectively compensate for phase errors, resulting in a decrease in image quality and blurring and geometric distortion.
Using the SAR imaging motion compensation method based on the generative adversarial network, by constructing a motion-compensated generative adversarial network, the joint distribution between the SAR image map without phase error and phase motion error is learned, and the SAR image map with phase error is directly converted into an image map without phase error.
SAR self-focus correction of motion errors is realized, and the two-dimensional phase errors caused by motion errors can be accurately estimated, imaging quality is improved, and the reliability and applicability of motion compensation are improved.
Smart Images
Figure CN120044522A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of radar imaging technology, and in particular relates to a SAR imaging motion compensation method and device based on a generative adversarial network, which is suitable for motion compensation of synthetic aperture radar (SAR) imaging, and can be used to improve the SAR imaging quality when the motion posture measurement error of platforms such as unmanned aerial vehicles and vehicles is large. Background Art
[0002] Synthetic aperture radar (SAR) is a system that forms images by synthesizing radar signals reflected from the ground or targets. It has the ability to observe the ground and sea surface all day and all night under different climate and lighting conditions. It plays an important role in many applications such as earth resource exploration, ocean monitoring and urban planning. In order to obtain high-quality images, the SAR system must maintain a constant motion speed on its platform and acquire echo data with uniform spatial sampling. However, these conditions are not always met during observation, resulting in phase errors in SAR images, reducing the quality of SAR images and causing blur and geometric distortion. Phase errors are generally caused by two reasons in the measurement of SAR radar systems: attitude error and trajectory error. Attitude error is caused by vibration of the platform on which the radar is carried or inconsistent attitude of the platform. Due to attitude error, the radar measurement direction continues to change, resulting in phase error. The trajectory error occurs because the platform carrying the radar is not a linear path but a non-ideal path, resulting in errors in the distance, angle and relative speed between the radar and the target.
[0003] In order to compensate for the phase error, SAR image motion compensation methods have been proposed in recent years. Typical methods include: the phase gradient-based autofocus algorithm (PGA), which estimates the motion error of the strong scattering points in the SAR image directly, and estimates the Doppler error of any order; in addition, the Map Drift (MD) autofocus method based on frequency division processing can compensate for large bandwidth or ultra-wideband echo data.
[0004] The above-mentioned existing phase compensation methods all have inevitable disadvantages. The PGA algorithm relies on strong scattering points in the image. For images with few strong scattering points or uniform scattering, it is difficult to achieve phase compensation. The MD autofocus algorithm can only estimate the second-order error and cannot correct the higher-order error. Summary of the invention
[0005] The object of the present invention is to propose a SAR imaging motion compensation method and device based on a generative adversarial network in view of the deficiencies of the above-mentioned existing technologies. As a motion compensation method for correcting the phase error in SAR images, based on the generative adversarial network for generating motion compensation, the joint distribution between the SAR images without phase error and the SAR images with phase motion error is learned to solve the phase error correction problem and improve the focusing quality of SAR imaging.
[0006] To achieve the above object, the present invention adopts the following technical solutions:
[0007] A SAR imaging motion compensation method based on a generative adversarial network, comprising the following steps:
[0008] Step 1: Collect actual SAR images without motion error;
[0009] Step 2: Superimpose phase errors on the actual SAR images without motion error to obtain actual SAR images with motion error;
[0010] Step 3: Slice the actual SAR images with motion error and the actual SAR images without motion error in both the azimuth and range directions to obtain phase error slice samples and error-free slice samples;
[0011] Step 4: Construct a generative adversarial network for motion compensation and a total loss function;
[0012] Step 5: Train the generative adversarial network for motion compensation based on the phase error slice samples and error-free slice samples;
[0013] Step 6: Slice the SAR image to be recognized with motion error in both the azimuth and range directions to obtain a sliced SAR image and input it into the trained generative adversarial network for motion compensation to generate a predicted SAR image without motion error.
[0014] The generative adversarial network for motion compensation as described above includes a generator, a first discriminator, and a second discriminator.
[0015] The generator generates a predicted SAR image with motion error and a predicted SAR image without motion error based on the phase error slice samples and error-free slice samples.
[0016] The predicted SAR image with motion error is input into the first discriminator for discrimination.
[0017] The predicted SAR image without motion error is input into the second discriminator for discrimination.
[0018] The generator as described above includes a first encoder, a second encoder, a shared unit, a first decoder, and a second decoder.
[0019] The phase error slice samples are input into the first encoder,
[0020] The error-free slice samples are input into the second encoder,
[0021] The output features of the first encoder and the second encoder are fused in the shared unit to obtain the fused features,
[0022] The fused features are respectively input into the first decoder and the second decoder,
[0023] The first decoder and the second decoder respectively output the predicted SAR image maps with motion errors and the predicted SAR image maps without motion errors.
[0024] As described above, both the first encoder and the second encoder sequentially include an image input layer, a convolutional layer, a ReLU activation function, two downsampling modules, and three residual modules,
[0025] Both the first decoder and the second decoder include three residual modules, two upsampling modules, a convolutional layer, and a Tanh activation function connected in sequence.
[0026] As described above, the shared unit includes a connection layer and four residual modules,
[0027] The output features of the first encoder and the second encoder are both input into the connection layer of the shared unit. In the connection layer of the shared unit, the output features of the first encoder and the second encoder are added in the channel dimension, and then the features obtained by adding in the channel dimension are dimension-reduced. The connection layer outputs the dimension-reduced features, and the dimension-reduced features output by the connection layer are further processed by four residual modules to obtain the fused features.
[0028] As described above, the total loss function is the weighted sum of the reconstruction loss, the cyclic reconstruction loss, and the adversarial loss.
[0029] As described above, the reconstruction loss is based on the following formula:
[0030]
[0031] where L recon1 is the first reconstruction loss, L recon2 is the second reconstruction loss, x1 and x2 are respectively the actual SAR image maps with motion errors and the actual SAR image maps without motion errors input, G1(x1, x2) is the predicted SAR image map with motion errors generated by the generator, is the actual SAR image map with motion errors, G2(x1, x2) is the predicted SAR image map without motion errors generated by the generator, is the actual SAR image without motion error, and E is the expectation operator.
[0032] The cyclic reconstruction loss is based on the following formula:
[0033] L cycle-recon = E[||x1 - G1(G1(x1, x2), G2(x1, x2))|| 2 + ||x2 - G2(G1(x1, x2), G2(x1, x2))|| 2
[0034] where L cycle-recon is the cyclic reconstruction loss, G1(G1(x1, x2), G2(x1, x2)) is the predicted SAR image with motion error obtained by processing G1(x1, x2) and G2(x1, x2) through the generator, and G2(G1(x1, x2), G2(x1, x2)) is the predicted SAR image without motion error obtained by processing G1(x1, x2) and G2(x1, x2) through the generator.
[0035] The adversarial loss is based on the following formula:
[0036]
[0037]
[0038] where L GAN1 and L GAN2 are the first adversarial loss and the second adversarial loss respectively. is the discrimination result of the first discriminator for the actual SAR image with motion error, and D1(G1(x1, x2)) is the discrimination result of the first discriminator for the predicted SAR image with motion error generated by the generator. is the discrimination result of the second discriminator for the actual SAR image without motion error, and D2(G2(x1, x2)) is the discrimination result of the second discriminator for the predicted SAR image without motion error generated by the generator.
[0039] A computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above motion compensation method are implemented.
[0040] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above motion compensation method are implemented.
[0041] A computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the above motion compensation method are implemented.
[0042] The present invention has the following beneficial effects compared with the prior art:
[0043] (1) By using the method of deep learning, a generative adversarial network for motion compensation is constructed. Based on the actual SAR image without motion error and the actual SAR image with motion error, the generative adversarial network for motion compensation directly converts the SAR image with phase error into a SAR image without phase error, realizing the SAR autofocus correction for motion error, being able to accurately estimate the two-dimensional phase error caused by motion error, and improving the imaging quality.
[0044] (2) Compared with the traditional motion compensation method, the present invention does not require the distribution of strong scatterers in the image observation scene. According to the image with phase error, error estimation and correction can be realized, improving the reliability and applicability of motion compensation.
[0045] The additional methods and advantages of the present invention will be given in the following description part. Some methods and advantages will become obvious from the following description, or be learned through the practice of the present invention. Description of the Drawings
[0046] Figure 1 is the implementation flowchart of the present invention;
[0047] Figure 2 is a comparison example diagram of the actual SAR image without motion error and the actual SAR image with motion error constructed by the present invention. Among them, (a) is the actual SAR image without motion error, and (b) is the actual SAR image with motion error;
[0048] Figure 3 is the overall structure diagram of the generative adversarial network for motion compensation constructed by the present invention;
[0049] Figure 4 is the encoder structure diagram in the generator constructed by the present invention;
[0050] Figure 5 is the structure diagram of the shared unit in the generator constructed by the present invention;
[0051] Figure 6 is the decoder structure diagram in the generator constructed by the present invention;
[0052] Figure 7 is the structure diagram of the discriminator constructed by the present invention;
[0053] Figure 8 is the structure diagram of the downsampling module constructed by the present invention;
[0054] Figure 9 is the structure diagram of the upsampling module constructed by the present invention;
[0055] Figure 10 It is the structural diagram of the residual module constructed by the present invention. Detailed implementation manners
[0056] For the convenience of those skilled in the art to understand and apply the present invention, the present invention will be described in detail below in combination with specific embodiments. It should be emphasized that the embodiments described herein are only used to illustrate and explain the present invention and do not limit the scope of the present invention.
[0057] Referring to Figure 1 , a SAR imaging motion compensation method based on a generative adversarial network, comprising the following steps:
[0058] Step 1: Collect SAR radar echo data of various typical surface scenes (such as cities, mountains, sea targets, and forest vegetation, etc.). The positions and postures of the radar motion platform orbits during the observation process of these SAR radar echo data are accurate. Imaging is performed through conventional SAR imaging algorithms (such as RD imaging method, CS imaging method, and ω-k imaging method, etc.) to obtain a precisely focused single-look complex image as the actual SAR image without motion error, as shown in Figure 2 (a) in
[0059] Step 2: Artificially add position and attitude motion errors to the radar motion platform orbit. Calculate the phase error caused by the position and attitude motion errors through the SAR imaging model, superimpose the phase error on the SAR radar echo data used in Step 1, and then perform imaging through the SAR imaging algorithm to obtain a single-look complex image that cannot be completely correctly focused as the actual SAR image with motion error, as shown in Figure 2 (b) in
[0060] Step 3: Generate an actual SAR image set with motion error and an actual SAR image set without motion error based on the actual SAR image with motion error and the actual SAR image without motion error. The dimensions of the actual SAR image with motion error and the actual SAR image without motion error are width × height × number of channels as H × W × C. In this embodiment, H × W × C is 1024 × 1024 × 1024. Before inputting into the generative adversarial network for motion compensation, slice the actual SAR image with motion error and the actual SAR image without motion error in both the azimuth and range directions to obtain phase error slice samples and error-free slice samples, considering the memory size limit for training and the scene information that the slices should contain. The dimensions of the phase error slice samples and the corresponding error-free slice samples are (H / p) × (W / p) × (C), where p is the size of the patch, and usually p is 2 or 4.
[0061] Step 4: Construct a generative adversarial network for motion compensation
[0062] The network architecture of the generative adversarial network is based on the variational autoencoder (VAE) and the generative adversarial network (GAN).
[0063] Referring to Figure 3 , the generative adversarial network includes a generator, a first discriminator, and a second discriminator. The encoder and decoder parts of the generator are the same as those of the variational autoencoder VAE, while the decoder and discriminator parts are similar to those of GAN.
[0064] The generator includes a first encoder, a second encoder, a shared unit, a first decoder, and a second decoder.
[0065] The phase error slice samples and the corresponding error-free slice samples are respectively input into the first encoder and the second encoder. The output features of the first encoder and the second encoder are data in different domains. The shared unit of the generation network learns the joint distribution of the data in different domains, and then the predicted SAR image maps with motion errors and the predicted SAR image maps without motion errors are respectively generated by the first decoder and the second decoder. The SAR image maps with motion errors and the SAR image maps without motion errors are respectively judged practically by the first discriminator and the second discriminator.
[0066] Referring to Figure 4 , both the first encoder and the second encoder sequentially include an image input layer, a convolutional layer, a ReLU activation function, two downsampling modules, and three residual modules. Each of the two downsampling modules includes a convolutional layer, an instance normalization layer, and a ReLU activation function. The convolutional layer downsamples the input features with a stride of 2. The instance normalization layer is used to improve the training convergence speed and reduce the sensitivity to hyperparameters, and the residual module is used to extract the features of the downsampled data.
[0067] Referring to Figure 5 , the shared unit includes a connection layer and four residual modules. After the phase error slice samples and the corresponding error-free slice samples are respectively input into the first encoder and the second encoder, the output features with the same two dimensions (H / 4×W / 4×1024 in this embodiment) are formed. The output features of the first encoder and the second encoder are both input into the connection layer of the shared unit. In the connection layer of the shared unit, the output features of the first encoder and the second encoder are added in the channel dimension. (The purpose of adding in the channel dimension is that when making predictions after training, only the SAR image map with motion errors to be recognized is input into the first encoder, and the second encoder does not need to be input again, and features of the same dimension size can be obtained). Further, the features added in the channel dimension are dimension-reduced, and the connection layer outputs the dimension-reduced features.
[0068] In the shared unit, the features after dimensionality reduction of the connection layer output are further processed through four residual modules to learn the joint distribution of the two types of data in the same space and obtain fused features.
[0069] Referring to Figure 6 , both the first decoder and the second decoder include three residual modules, two upsampling modules, a convolutional layer, and a Tanh activation function connected in sequence.
[0070] As Figure 9 shown, the upsampling module includes an upsampling unit, an instance normalization layer, and a ReLU activation function.
[0071] As Figure 10 shown, the residual module includes a first convolutional layer, a first instance normalization layer, a ReLU activation function, a second convolutional layer, a second instance normalization layer, and an addition module.
[0072] The first decoder and the second decoder respectively upsample the fused features of the output of the shared part, so that the first decoder and the second decoder output the corresponding predicted SAR image maps with motion errors and the predicted SAR image maps without motion errors.
[0073] Referring to Figure 7 , the first discriminator and the second discriminator adopt the PatchGAN structure in Pix2Pix.
[0074] Both the first discriminator and the second discriminator include an image input layer, a convolutional layer, a ReLU activation function, five cyclic layers, and a convolutional module.
[0075] The five cyclic layers include a convolutional layer, a batch normalization layer, and a ReLU activation function connected in sequence and looping 5 times as a whole.
[0076] The convolutional module implements four times of 1 / 2 downsampling.
[0077] The discriminator determines the reality through four times of 1 / 2 downsampling, and inserts a batch normalization layer between the convolutional layer and the ReLU activation function to accelerate training and reduce the sensitivity to network initialization. The structural configurations of the first discriminator and the second discriminator are the same.
[0078] The total loss function of the generative adversarial network is the weighted sum of the reconstruction loss, the cycle reconstruction loss, and the adversarial loss. In this embodiment, the weights of the self-reconstruction loss, the cycle reconstruction loss, and the adversarial loss are set to 10, 10, and 1 respectively.
[0079] Reconstruction Loss: The reconstruction loss is used to ensure that the predicted SAR image map generated by the decoder is similar to the actual SAR image map, and usually uses a pixel-level error metric. The formula is:
[0080]
[0081] Among them, L recon1 is the first reconstruction loss, and L recon2 is the second reconstruction loss. x1 and x2 are the actual SAR image maps with motion errors and without motion errors in the input, respectively. G1(x1, x2) is the predicted SAR image map with motion errors generated by the generator. is the actual SAR image map with motion errors, and G2(x1, x2) is the predicted SAR image map without motion errors generated by the generator. is the actual SAR image map without motion errors, and E is the expectation operator.
[0082] The formula for the Cycle Reconstruction Loss is:
[0083] L cycle-recon = E[||x1 - G1(G1(x1, x2), G2(x1, x2))|| 2 + ||x2 - G2(G1(x1, x2), G2(x1, x2))|| 2
[0084] Among them, L cycle-recon is the cycle reconstruction loss. G1(G1(x1, x2), G2(x1, x2)) is the predicted SAR image map with motion errors obtained by processing G1(x1, x2) and G2(x1, x2) through the generator. G2(G1(x1, x2), G2(x1, x2)) is the predicted SAR image map without motion errors obtained by processing G1(x1, x2) and G2(x1, x2) through the generator.
[0085] GAN Loss: The GAN loss is used in the generative adversarial network. The goal of the generator is to generate outputs that are difficult to distinguish from the actual images, and the goal of the discriminator is to distinguish between actual and generated images. The formula is:
[0086]
[0087] Among them, L GAN1 and L GAN2 are the first adversarial loss and the second adversarial loss, respectively. is the discrimination result of the first discriminator for the actual SAR image map with motion errors, and D1(G1(x1, x2)) is the discrimination result of the first discriminator for the predicted SAR image map with motion errors generated by the generator. is the discrimination result of the second discriminator for the actual SAR image without motion error, and D2(G2(x1, x2)) is the discrimination result of the second discriminator for the predicted SAR image without motion error generated by the generator.
[0088] Step Five: Sample Training and Verification.
[0089] Use 90% of the actual SAR images with motion errors and the actual SAR images without motion errors for training, and use the remaining 10% of the actual SAR images with motion errors and the actual SAR images without motion errors for performance testing.
[0090] During the training process, the Adam optimization method with a learning rate of 0.0001 is used, and the optimization parameters are selected as the gradient decay factor of 0.5, the squared gradient decay factor of 0.999, and the weight decay regularization of 0.0001.
[0091] Through learning in multiple epochs, extract the features of the two types of image data and learn the joint distribution of the two types of data until the total loss function is minimized.
[0092] Step Six: Slice the SAR image to be recognized with motion errors in the azimuth and range directions to obtain sliced SAR images and input them into the trained generator to generate predicted SAR images without motion errors.
[0093] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.
[0094] Embodiment 2:
[0095] In this embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.
[0096] Embodiment 3:
[0097] In this embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0098] Embodiment 4:
[0099] In this embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.
[0100] The above description is only a specific embodiment of the present invention and does not limit the wide application of the present invention. Obviously, for professionals in the field, after understanding the content and principle of the present invention, various modifications and adjustments can be made to the form and details without departing from the basic idea and structure of the present invention. These improvements and changes based on the concept of the present invention are still within the scope of protection of the claims of the present invention.
Claims
1. A SAR imaging motion compensation method based on a generative adversarial network, characterized in that: The following steps are involved: Step 1: Collect actual SAR images without motion errors; Step 2: superimpose the phase error on the actual SAR image without motion error to obtain the actual SAR image with motion error; Step 3: Slice the actual SAR image with motion error and the actual SAR image without motion error in both azimuth and distance directions to obtain phase error slice samples and error-free slice samples; Step 4: Construct the motion-compensated generative adversarial network and total loss function; Step 5: Train the motion-compensated generative adversarial network based on the phase error slice samples and the error-free slice samples; Step 6: Slice the SAR image with motion error in both azimuth and distance directions to obtain the sliced SAR image, and input it into the trained motion-compensated generative adversarial network to generate a predicted SAR image without motion error.
2. The SAR imaging motion compensation method based on a generative adversarial network according to claim 1, characterized in that: The motion compensated generative adversarial network comprises a generator, a first discriminator and a second discriminator, The generator generates a predicted SAR image with motion error and a predicted SAR image without motion error based on the phase error slice samples and the error-free slice samples. The predicted SAR image with motion error is input to the first discriminator for discrimination. The predicted SAR image without motion error is input to the second discriminator for discrimination.
3. The SAR imaging motion compensation method based on a generative adversarial network according to claim 1, characterized in that: The generator includes a first encoder, a second encoder, a sharing unit, a first decoder, and a second decoder, The phase error slice samples are input into the first encoder, The error-free slice samples are input into the second encoder, The output features of the first encoder and the second encoder are fused in the shared unit to obtain the fused features. The fused features are input into the first decoder and the second decoder respectively. The first decoder and the second decoder output a predicted SAR image with motion errors and a predicted SAR image without motion errors, respectively.
4. The SAR imaging motion compensation method based on a generative adversarial network according to claim 3, characterized in that: The first encoder and the second encoder each include an image input layer, a convolution layer, a ReLU activation function, two downsampling modules, and three residual modules in sequence. The first decoder and the second decoder each include three residual modules, two upsampling modules, a convolutional layer, and a Tanh activation function connected in sequence.
5. The SAR imaging motion compensation method based on generative adversarial network according to claim 3, characterized in that: The shared unit includes a connection layer and four residual modules, The output features of the first encoder and the second encoder are both input into the connection layer of the shared unit. In the connection layer of the shared unit, the output features of the first encoder and the second encoder are added in the channel dimension, and then the added features in the channel dimension are reduced in dimension. The connection layer outputs the reduced-dimensional features. The reduced-dimensional features output by the connection layer are further processed through four residual modules to obtain fused features.
6. The SAR imaging motion compensation method based on a generative adversarial network according to claim 3, characterized in that: The total loss function is the weighted sum of reconstruction loss, cyclic reconstruction loss, and adversarial loss.
7. The SAR imaging motion compensation method based on a generative adversarial network according to claim 6, characterized in that: The reconstruction loss is based on the following formula: Among them, L recon1 is the first reconstruction loss, L recon2 is the second reconstruction loss, x1 and x2 are the input actual SAR image with motion error and the actual SAR image without motion error, respectively, G1(x1, x2) is the predicted SAR image with motion error generated by the generator, is the actual SAR image with motion error, G2(x1, x2) is the predicted SAR image without motion error generated by the generator, is the actual SAR image without motion error, E is the expected operator, The cycle reconstruction loss is based on the following formula: L cycle-recon =E[||x1-G1(G1(x1,x2),G2(x1,x2))|| 2 +||x2-G2(G1(x1,x2),G2(x1,x2))|| 2 ] Among them, L cycle-recon is the cyclic reconstruction loss, G1(G1(x1, x2), G2(x1, x2)) is the predicted SAR image with motion error obtained by processing G1(x1, x2) and G2(x1, x2) by the generator, G2(G1(x1, x2), G2(x1, x2)) is the predicted SAR image without motion error obtained by processing G1(x1, x2) and G2(x1, x2) by the generator, The adversarial loss is based on the following formula: Among them, L GAN1 and L GAN2 They are the first adversarial loss and the second adversarial loss, is the discrimination result of the first discriminator for the actual SAR image with motion error, and D1(G1(x1, x2)) is the discrimination result of the first discriminator for the predicted SAR image with motion error generated by the generator. is the discrimination result of the second discriminator for the actual SAR image without motion error, and D2(G2(x1, x2)) is the discrimination result of the second discriminator for the predicted SAR image without motion error generated by the generator.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the motion compensation method according to any one of claims 1 to 7 are implemented.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the motion compensation method according to any one of claims 1 to 7 are implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the motion compensation method according to any one of claims 1 to 7 are implemented.