A cross-domain generative adversarial SAR image compensation method and a computer readable medium

Abstract

The application obtains multiple SAR geometric correction images and optical images in the same area, obtains a mask image corresponding to each SAR geometric correction image based on a SAR image overlay area positioning method of a spatial geometric ratio relationship, and obtains a SAR image overlay compensation data set containing the SAR image, the optical image and the mask image through block processing. A cross-domain generated SAR overlay compensation adversarial network is constructed, an optimized cross-domain generated SAR overlay compensation adversarial network is obtained through training of a compensation adversarial network loss function model, an optical image is generated into a SAR image, and overlay compensation of the SAR image is realized. The application does not need to calculate a SAR image ascending / descending track geometric relationship, has high data processing efficiency, adopts a supervised cross-domain generated adversarial network model, guarantees maximization of similarity between the cross-domain generated SAR image and the original SAR image, and effectively compensates for an information loss problem caused by SAR image overlay.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of microwave remote sensing, and particularly relates to a SAR image compensation method based on cross-domain generative adversarial and a computer readable medium. BACKGROUND

[0002] Synthetic Aperture Radar (SAR) is a kind of active imaging sensor for microwave detection of ground targets with all-weather and all-day imaging. Due to the special side-looking geometric imaging mode of SAR, SAR images in mountainous areas with large terrain undulations will have serious geometric distortion - overlap. The overlap phenomenon leads to information loss of SAR images, affects image quality, and seriously restricts subsequent applications such as ground feature classification and target identification based on SAR images. The existing overlap area compensation method based on SAR image ascending / descending track data is affected by different factors such as image incidence angle, overlap degree, and multi-temporal, which brings great difficulty to overlap compensation.

[0003] With the extension of deep learning methods to various fields, remote sensing data enhancement based on deep learning methods has also become a new hot research. In 2020, Jun-Yan Zhu published the Cycle GAN, which realized the robust domain migration of natural images. Many researchers have carried out various application researches such as image color, cloud and fog removal, and image enhancement based on this generative adversarial network method.

[0004] In view of the SAR image overlap compensation requirement, combined with the ability of the cycle generative adversarial network, through the cycle consistent generative adversarial network architecture, the optical image is converted into the SAR image, which can be used as an effective source for SAR image overlap information compensation. In addition, this method can also be applied to information loss compensation tasks of multi-type remote sensing images such as optical images, hyperspectral images, and night light images, and is the key direction of future image data information compensation. SUMMARY

[0005] The application proposes a SAR image compensation method based on cross-domain generative adversarial and a computer readable medium to solve the problem of information loss in subsequent applications caused by overlap phenomenon in mountainous areas of spaceborne SAR orthoimage.

[0006] The technical scheme of the method of the application is a SAR image compensation method based on cross-domain generative adversarial, and the specific steps are as follows:

[0007] Step 1: In the same geographical area, a plurality of SAR images and a plurality of optical images are obtained, and the plurality of SAR images are combined with DEM data for geometric correction to obtain a plurality of geometric corrected SAR images;

[0008] Step 2: obtaining a mask image corresponding to each SAR image according to a pixel ratio between each geometrically corrected SAR image and a corresponding region pixel of each SAR image through a SAR image overlay region positioning method based on a spatial geometric ratio relationship;

[0009] Step 3: performing block processing on each geometrically corrected SAR image, each optical image and the mask image corresponding to each SAR image respectively to sequentially obtain a plurality of sub-images of each SAR image, a plurality of sub-images of each optical image and a plurality of sub-images of each mask image;

[0010] Step 4: constructing a cross-domain generated SAR overlay compensation adversarial network, inputting each sub-image of each geometrically corrected SAR image, each sub-image of each optical image and each sub-image of each mask image into the cross-domain generated SAR overlay compensation adversarial network to output each generated sub-image of each geometrically corrected SAR image and each generated sub-image of each optical image, constructing a compensation adversarial network loss function model and obtaining an optimized cross-domain generated SAR overlay compensation adversarial network through ADAM algorithm optimization training;

[0011] Step 5: inputting the optical image data to be processed, the SAR image data to be processed and the corresponding mask image to be processed into the optimized SAR overlay compensation adversarial network model to output the SAR image data after overlay compensation;

[0012] As preferred, the plurality of sub-images of each geometrically corrected SAR image in step 3 is specifically defined as follows:

[0013] {SAR k (x, y), k ∈ [1, K], x ∈ [1, X], y ∈ [1, Y]}

[0014] wherein, SAR k (x, y) represents the sub-image corresponding to the xth row and yth column of the kth SAR image in the geometrically corrected SAR image, K represents the total number of SAR images after geometric correction processing, X is the number of sub-image rows of the Kth geometrically corrected SAR image, and Y is the number of sub-image columns of the Kth geometrically corrected SAR image;

[0015] The plurality of sub-images of each optical image in step 3 is specifically defined as follows:

[0016] {OPT k (x, y), k ∈ [1, K], x ∈ [1, X], y ∈ [1, Y]}

[0017] wherein, OPT k(x, y) represents the corresponding sub-image of the xth row and yth column of the kth optical image in the optical image, K represents the total number of SAR images after geometric correction, X is the number of sub-images of the Kth scene after geometric correction, and Y is the number of columns of the sub-images of the Kth scene after geometric correction;

[0018] The plurality of sub-images of each mask image in step 3 is specifically defined as follows:

[0019] {Mask k (x, y), k ∈ [1, K], x ∈ [1, X], y ∈ [1, Y]}

[0020] Wherein, Mask k (x, y) represents the corresponding sub-image of the xth row and yth column of the kth mask image in the corresponding mask image of the SAR image after geometric correction, K represents the total number of SAR images after geometric correction, X is the number of sub-images of the Kth scene after geometric correction, and Y is the number of columns of the sub-images of the Kth scene after geometric correction;

[0021] As preferred, the cross-domain SAR overlay compensation adversarial network in step 4 specifically comprises:

[0022] The SAR overlay compensation adversarial network comprises an optical image generator network, a SAR image generator network, an optical image discriminator network, and a SAR image discriminator network;

[0023] The optical image generator network and the optical image discriminator network are cascaded, and the SAR image generator network and the SAR image discriminator network are cascaded to form a cross-domain SAR overlay compensation adversarial network;

[0024] The optical image generator network is composed of a first encoder, a first residual module, a first decoder, a second encoder, a second residual module, and a second decoder in sequence;

[0025] The SAR image generator network is composed of a third encoder, a third residual module, a third decoder, a fourth encoder, a fourth residual module, and a fourth decoder in sequence;

[0026] The first encoder of the optical image generator network adopts a convolution module of ReflectionPad2d and CIL in cascade, which is used for inputting the sub-images of each SAR image after geometric correction and the sub-images of each mask image, performing feature coding processing through a convolution layer, and outputting an encoded feature map;

[0027] The first residual module of the optical image generator network adopts a plurality of cascaded Residual block modules for inputting the encoded feature map of the encoder output of the optical image generator network, performing feature enhancement through a residual convolution block, and outputting an enhanced feature map;

[0028] The first decoder of the optical image generator network adopts a CTIR and ReflectionPad2d cascaded deconvolution module for inputting the enhanced feature map of the residual module of the optical image generator network, performing feature decoding processing through a deconvolution layer, and outputting a simulated optical image sub-image corresponding to each sub-image of each geometrically corrected SAR image to the fourth encoder of the SAR image generator network;

[0029] The optical image generator network inputs the sub-image of each geometrically corrected SAR image and the sub-image of each mask image to the first encoder of the optical image generator network, and outputs the simulated optical image sub-image to the fourth encoder of the SAR image generator network by the first decoder, and is specifically defined as follows:

[0030] Sim_OPT k (x, y) = G opt (SAR k (x, y) - Mask k (x, y))

[0031] k ∈ [1, K], x ∈ [1, X], y ∈ [1, Y]

[0032] Wherein, Sim_OPT k (x, y) represents the xth row and yth column corresponding sub-image of the kth generated simulated optical image, SAR k (x, y) represents the xth row and yth column corresponding sub-image of the kth SAR image in the geometrically corrected SAR image, Mask k (x, y) represents the xth row and yth column corresponding sub-image of the kth mask image in the corresponding mask image of the geometrically corrected SAR image, K represents the total number of SAR images after geometric correction processing, X is the number of rows of sub-images of the Kth geometrically corrected SAR image, Y is the number of columns of sub-images of the Kth geometrically corrected SAR image, G opt represents the optical image generator network;

[0033] The second encoder of the optical image generator network adopts a ReflectionPad2d and CIL cascaded convolution module for inputting the sub-image of each simulated SAR image, performing feature encoding processing through a convolution layer, and outputting an encoded feature map;

[0034] The second residual module of the optical image generator network employs multiple cascaded residual block modules, which are used to input the encoded feature map output by the encoder of the optical image generator network, perform feature enhancement through residual convolutional blocks, and output the enhanced feature map.

[0035] The second decoder of the optical image generator network uses a cascaded deconvolution module of CTIR and ReflectionPad2d to input the feature map enhanced by the residual module of the optical image generator network. The feature is then processed by the deconvolution layer to output the cyclic simulated SAR image sub-image corresponding to each sub-image of each geometrically corrected SAR image to the SAR image discriminator network.

[0036] The optical image generator network receives a sub-image of each simulated SAR image as input. A second encoder and a second decoder output the cyclically generated sub-images of the simulated optical image to the SAR image discriminator network. The specific definitions are as follows:

[0037] Cycle_OPT k (x, y) = G opt (Sim_SAR k (x, y))

[0038] Among them, Cycle_OPT k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated optical image, Sim_SAR k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th simulated SAR image. K represents the total number of SAR images after geometric correction, X is the number of sub-image rows in the K-th geometrically corrected SAR image, Y is the number of sub-image columns in the K-th geometrically corrected SAR image, and G... opt Represents an optical image generator network;

[0039] The third encoder of the SAR image generator network uses a convolutional module of ReflectionPad2d and CIL cascaded to take a sub-image of each optical image as input, perform feature encoding processing through the convolutional layer, and output an encoded feature map.

[0040] The third residual module of the SAR image generator network uses multiple cascaded residual block modules to input the encoded feature map output by the encoder of the SAR image generator network, perform feature enhancement through residual convolutional blocks, and output a feature map.

[0041] The third decoder of the SAR image generator network uses a cascaded deconvolution module of CTIR and ReflectionPad2d to input the feature map enhanced by the residual module of the SAR image generator network. The feature is then processed by the deconvolution layer to output the simulated SAR image sub-image corresponding to each sub-image of each optical image to the second encoder of the optical image generator network.

[0042] The SAR image generator network takes a sub-image of each optical image as input and outputs it to the second encoder of the optical image generator network via a third encoder and a third decoder. The specific definitions are as follows:

[0043] Sim_SAR k (x, y) = G sar (OPT k (x, y))

[0044] k∈[1,K],x∈[1,X],y∈[1,Y]

[0045] Among them, Sim_SAR k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th simulated SAR image, OPT k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th optical image, K represents the total number of SAR images after geometric correction, X is the number of sub-image rows in the k-th geometrically corrected SAR image, Y is the number of sub-image columns in the k-th geometrically corrected SAR image, and G... sar This represents the SAR image generator network;

[0046] The fourth encoder of the SAR image generator network uses a convolutional module of ReflectionPad2d and CIL cascaded to take a sub-image of each simulated optical image as input, perform feature encoding processing through the convolutional layer, and output an encoded feature map.

[0047] The fourth residual module of the SAR image generator network uses multiple cascaded residual block modules to input the encoded feature map output by the encoder of the SAR image generator network, perform feature enhancement through residual convolutional blocks, and output a feature map.

[0048] The fourth decoder of the SAR image generator network uses a cascaded deconvolution module of CTIR and ReflectionPad2d to input the feature map enhanced by the residual module of the SAR image generator network. The feature decoding is performed through the deconvolution layer, and the cyclic simulated SAR image sub-image corresponding to each sub-image of each optical image is output to the SAR image discriminator network.

[0049] The SAR image generator network takes a sub-image of each simulated optical image as input, passes it to the SAR image generator network via a fourth encoder, and outputs a sub-image of the cyclically simulated SAR image via a fourth decoder to the SAR image discriminator network. The specific definitions are as follows:

[0050] Cycle_SAR k (x, y) = G sar (Sim_OPT k (x, y) + Mask k (x, y))

[0051] Among them, Cycle_SAR k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated SAR image, Sim_OPT k (x, y) represents the sub-image Mask corresponding to the x-th row and y-th column of the k-th generated simulated optical image. k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th mask image in the geometrically corrected SAR image, K represents the total number of geometrically corrected SAR images, X is the number of rows of sub-images in the k-th geometrically corrected SAR image, Y is the number of columns of sub-images in the k-th geometrically corrected SAR image, and G... sar This represents the SAR image generator network;

[0052] The optical image discriminator network is composed of patchGAN fully convolutional modules;

[0053] The optical image discriminator network is used to generate the loss of the simulated optical image by taking a sub-image of each simulated optical image and a sub-image of each optical image as input. The specific definition is as follows:

[0054] V opt-simopt (x, y) = D opt (Sim_OPT k (x, y), OPT k (x, y))V opt-simopt ∈[0,1]

[0055] Among them, V opt-simopt(x, y) represents the contrast loss result of a sub-image of each simulated optical image and a sub-image of each optical image, the range of which is [0, 1]. Sim_OPT k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th generated simulated optical image, OPT k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th optical image, K represents the total number of SAR images after geometric correction, X is the number of sub-image rows in the k-th geometrically corrected SAR image, Y is the number of sub-image columns in the k-th geometrically corrected SAR image, and D opt This represents an optical image discriminator network;

[0056] The optical image discriminator network is used to take as input a sub-image of each cyclic simulated optical image and a sub-image of each optical image, and to generate the loss of the cyclic simulated optical image, which is specifically defined as follows:

[0057] V cycleopt-opt (x, y) = D opt (Cycle_OPT k (x, y), OPT k (x, y))V cycleopt-opt ∈[0,1]

[0058] Among them, V cycleopt-opt (x, y) represents the contrast loss result of a sub-image of each cycled simulated optical image and a sub-image of each optical image, the range of which is [0, 1]. Cycle_OPT k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated optical image, OPT k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th optical image, K represents the total number of SAR images after geometric correction, X is the number of sub-image rows in the k-th geometrically corrected SAR image, Y is the number of sub-image columns in the k-th geometrically corrected SAR image, and D opt This represents an optical image discriminator network;

[0059] The SAR image discriminator network is composed of patchGAN fully convolutional modules;

[0060] The SAR image discriminator network is used as input for sub-images of each simulated SAR image and sub-images of each geometrically corrected SAR image. k (x, y), the loss for generating simulated SAR images, is defined as follows;

[0061] V sar-simsar (x, y) = D sar (Sim_SARk (x, y), SAR k (x, y))

[0062] V sar-simsar ∈[0,1]

[0063] Among them, V sar-simsar (x, y) represents the contrast loss result of a sub-image of each simulated SAR image and a sub-image of each geometrically corrected SAR image, with the result ranging from [0, 1]. Sim_SAR k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th generated simulated SAR image, SAR k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th geometrically corrected SAR image, K represents the total number of geometrically corrected SAR images, X is the number of sub-image rows in the k-th geometrically corrected SAR image, Y is the number of sub-image columns in the k-th geometrically corrected SAR image, and D... sar This represents the SAR image discriminator network;

[0064] The SAR image discriminator network is used to take as input sub-images of each cyclically simulated SAR image and sub-images of each geometrically corrected SAR image, and generates the loss for the cyclically simulated SAR image, specifically defined as follows:

[0065]

[0066] V cyclesar-sar ∈[0,1]

[0067] Among them, V cyclesar-sar (x, y) represents the contrast loss result of a sub-image of each cyclically simulated SAR image and a sub-image of each geometrically corrected SAR image, with the result ranging from [0, 1]. This represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated SAR image. k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th geometrically corrected SAR image, K represents the total number of geometrically corrected SAR images, X is the number of sub-image rows in the k-th geometrically corrected SAR image, Y is the number of sub-image columns in the k-th geometrically corrected SAR image, and D... sar This represents the SAR image discriminator network;

[0068] The compensation adversarial network loss function model described in step 4 is specifically defined as follows:

[0069] Loss all =(Loss opt2sar +Loss sar2opt )+Losscycle +λLoss Rec

[0070] Where λ is a hyperparameter, and Loss sar2opt Generate adversarial loss for optical images, Loss opt2sar Generate adversarial loss for SAR images, Loss cycle Loss is the loss for cycle consistency. Rec For reconstruction loss;

[0071] The optical image generation adversarial loss is defined as follows:

[0072]

[0073] Among them, Loss opt2sar SAR stands for Image Generative Adversarial Loss. k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th geometrically corrected SAR image, Sim_SAR k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th generated simulated SAR image, D sar Let X represent the SAR image discriminator network, K represent the total number of SAR images after geometric correction, X represent the number of sub-image rows of the Kth geometrically corrected SAR image, and Y represent the number of sub-image columns of the Kth geometrically corrected SAR image.

[0074] The SAR image generation adversarial loss is defined as follows:

[0075]

[0076] Among them, Loss sar2opt OPT generates adversarial loss for optical images. k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th optical image, Sim_OPT k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th generated simulated optical image, D opt Let X represent the optical image discriminator network, K represent the total number of SAR images after geometric correction, X represent the number of sub-image rows of the Kth geometrically corrected SAR image, and Y represent the number of sub-image columns of the Kth geometrically corrected SAR image.

[0077] The cycle consistency loss is defined as follows:

[0078]

[0079] Among them, Loss cycle For the cycle consistency loss, OPT k(x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th optical image. Mask k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th mask image in the geometrically corrected SAR image. k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th geometrically corrected SAR image, K represents the total number of geometrically corrected SAR images, X is the number of sub-image rows in the k-th geometrically corrected SAR image, Y is the number of sub-image columns in the k-th geometrically corrected SAR image, and G... op□ G represents an optical image generator network. sar This represents the SAR image generator network;

[0080] The reconstruction loss is defined as follows:

[0081]

[0082] Among them, Loss Rec For reconstruction loss, Cycle_OPT k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated optical image. OPT represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated SAR image. k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th optical image, SAR k (x, y) represents the sub-image corresponding to the x-th row and y-th column of the k-th geometrically corrected SAR image, K represents the total number of geometrically corrected SAR images, X is the number of sub-image rows in the k-th geometrically corrected SAR image, Y is the number of sub-image columns in the k-th geometrically corrected SAR image, and G... opt G represents an optical image generator network. sar This represents the SAR image generator network;

[0083] The present invention also provides a computer-readable medium storing a computer program executed by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the method for assessing the state of the transformer combination.

[0084] The advantages of this invention compared to existing technologies are:

[0085] A cross-domain generative adversarial network (GAN) approach was adopted to generate SAR images from optical images, effectively solving the problem of SAR image overlay compensation. Compared with traditional SAR image overlay compensation methods, this method does not require consideration of the accurate geometric relationship of the SAR image's ascent and descent trajectories, does not require complex calculations, and has high data processing efficiency.

[0086] A supervised cross-domain generative adversarial network (GAN) model is adopted. Building upon existing GANs, masking information is introduced as an auxiliary method to maximize the similarity between the generated SAR image and the original SAR image. Through a bidirectional cyclic cross-domain GAN model, SAR images are generated from optical images, effectively compensating for information loss caused by SAR image overlay. Attached Figure Description

[0087] Figure 1 : Flowchart of the method according to an embodiment of the present invention;

[0088] Figure 2 : Network architecture diagram of an embodiment of the present invention. Detailed Implementation

[0089] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0090] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0091] The following is combined with Figures 1-2 The technical solution of the method in the embodiments of the present invention is a cross-domain generative adversarial SAR image compensation method, and the specific steps are as follows:

[0092] Step 1: Within the same geographical area, acquire multiple SAR images and multiple optical images, and combine the multiple SAR images with DEM data to perform geometric correction processing to obtain multiple geometrically corrected SAR images;

[0093] Step 2: Based on the pixel ratio between the corresponding region pixels of each geometrically corrected SAR image and each SAR image, obtain the mask image corresponding to each SAR image using the SAR image overlay region localization method based on spatial geometric ratio relationship;

[0094] Step 3: Divide each geometrically corrected SAR image, each optical image, and the corresponding mask image of each SAR image into blocks to obtain multiple sub-images of each SAR image, multiple sub-images of each optical image, and multiple sub-images of each mask image.

[0095] The multiple sub-images of each geometrically corrected SAR image mentioned in step 3 are specifically defined as follows:

[0096] SAR k (x, y), k∈[1, K], x∈[1, X], y∈[1, Y]}

[0097] Among them, SAR k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th SAR image in the geometrically corrected SAR image, K=31 represents the total number of SAR images after geometric correction, X=5 is the number of rows of sub-images in the 6th geometrically corrected SAR image, and Y=4 is the number of columns of sub-images in the 6th geometrically corrected SAR image.

[0098] The multiple sub-images of each optical image mentioned in step 3 are specifically defined as follows:

[0099] {OPT k (x, y), k∈[1, K], x∈[1, X], y∈[1, Y]}

[0100] Where OPTk(x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th optical image, K=31 represents the total number of SAR images after geometric correction, X=5 is the number of sub-image rows of the 6th geometrically corrected SAR image, and Y=4 is the number of sub-image columns of the 6th geometrically corrected SAR image.

[0101] The multiple sub-images of each mask image mentioned in step 3 are specifically defined as follows:

[0102] {Mask k (x, y), k∈[1, K], x∈[1, X], y∈[1, Y]}

[0103] Among them, Mask k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th mask image in the mask image corresponding to the geometrically corrected SAR image, K=31 represents the total number of SAR images after geometric correction, X=5 is the number of rows of sub-images in the 6th geometrically corrected SAR image, and Y=4 is the number of columns of sub-images in the 6th geometrically corrected SAR image.

[0104] Step 4: Construct a cross-domain generative SAR overlay compensation adversarial network. Input each sub-image of each geometrically corrected SAR image, each sub-image of each optical image, and each sub-image of each mask image into the cross-domain generative SAR overlay compensation adversarial network to obtain each generated sub-image of each geometrically corrected SAR image and each generated sub-image of each optical image. Construct a loss function model for the compensation adversarial network and optimize it through ADAM algorithm training to obtain the optimized cross-domain generative SAR overlay compensation adversarial network.

[0105] like Figure 2 As shown, the cross-domain generation of the SAR overlay compensation adversarial network described in step 4 specifically includes:

[0106] The SAR overlay compensation adversarial network includes an optical image generator network, a SAR image generator network, an optical image discriminator network, and a SAR image discriminator network.

[0107] The optical image generator network and the optical image discriminator network are cascaded together, and the SAR image generator network and the SAR image discriminator network are cascaded together to form a single network.

[0108] The optical image generator network is composed of a first encoder, a first residual module, a first decoder, a second encoder, a second residual module, and a second decoder cascaded in sequence.

[0109] The SAR image generator network is composed of a third encoder, a third residual module, a third decoder, a fourth encoder, a fourth residual module, and a fourth decoder cascaded in sequence.

[0110] The first encoder of the optical image generator network uses a convolutional module of ReflectionPad2d and CIL cascaded to take as input sub-images of each geometrically corrected SAR image and sub-images of each mask image, and performs feature encoding processing through the convolutional layer to output an encoded feature map.

[0111] The first residual module of the optical image generator network uses multiple cascaded residual block modules to input the encoded feature map output by the encoder of the optical image generator network, perform feature enhancement through residual convolutional blocks, and output the enhanced feature map.

[0112] The first decoder of the optical image generator network uses a cascaded deconvolution module of CTIR and ReflectionPad2d to input the feature map enhanced by the residual module of the optical image generator network. The feature is then processed by the deconvolution layer to output the simulated optical image sub-image corresponding to each sub-image of each geometrically corrected SAR image to the fourth encoder of the SAR image generator network.

[0113] The optical image generator network receives sub-images of each geometrically corrected SAR image and each mask image as input. A first encoder and a first decoder output a sub-image of the simulated optical image to the fourth encoder of the SAR image generator network, specifically defined as follows:

[0114] Sim_OPT k (x, y) = G opt (SAR k (x, y) - Mask k (x, y))

[0115] k∈[1,K],x∈[1,X],y∈[1,Y]

[0116] Among them, Sim_OPT k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th generated simulated optical image, SAR k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th SAR image in the geometrically corrected SAR image. Mask k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th mask image in the geometrically corrected SAR image, K=31 represents the total number of SAR images after geometric correction, X=5 is the number of rows of the sub-images in the 6th geometrically corrected SAR image, Y=4 is the number of columns of the sub-images in the 6th geometrically corrected SAR image, G opt Represents an optical image generator network;

[0117] The second encoder of the optical image generator network uses a convolutional module of ReflectionPad2d and CIL cascaded to take a sub-image of each simulated SAR image as input, perform feature encoding processing through the convolutional layer, and output an encoded feature map.

[0118] The second residual module of the optical image generator network employs multiple cascaded residual block modules, which are used to input the encoded feature map output by the encoder of the optical image generator network, perform feature enhancement through residual convolutional blocks, and output the enhanced feature map.

[0119] The second decoder of the optical image generator network uses a cascaded deconvolution module of CTIR and ReflectionPad2d to input the feature map enhanced by the residual module of the optical image generator network. The feature is then processed by the deconvolution layer to output the cyclic simulated SAR image sub-image corresponding to each sub-image of each geometrically corrected SAR image to the SAR image discriminator network.

[0120] The optical image generator network receives a sub-image of each simulated SAR image as input. A second encoder and a second decoder output the cyclically generated sub-images of the simulated optical image to the SAR image discriminator network. The specific definitions are as follows:

[0121] Cycle_OPT k (x, y) = G opt (Sim_SAR k (x, y))

[0122] Among them, Cycle_OPT k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th generated cyclic simulated optical image, Sim_SAR k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th simulated SAR image, KK=31 represents the total number of SAR images after geometric correction, X=5 is the number of rows of sub-images in the 6th geometrically corrected SAR image, Y=4 is the number of columns of sub-images in the 6th geometrically corrected SAR image, G opt Represents an optical image generator network;

[0123] The third encoder of the SAR image generator network uses a convolutional module of ReflectionPad2d and CIL cascaded to take a sub-image of each optical image as input, perform feature encoding processing through the convolutional layer, and output an encoded feature map.

[0124] The third residual module of the SAR image generator network uses multiple cascaded residual block modules to input the encoded feature map output by the encoder of the SAR image generator network, perform feature enhancement through residual convolutional blocks, and output a feature map.

[0125] The third decoder of the SAR image generator network uses a cascaded deconvolution module of CTIR and ReflectionPad2d to input the feature map enhanced by the residual module of the SAR image generator network. The feature is then processed by the deconvolution layer to output the simulated SAR image sub-image corresponding to each sub-image of each optical image to the second encoder of the optical image generator network.

[0126] The SAR image generator network takes a sub-image of each optical image as input and outputs it to the second encoder of the optical image generator network via a third encoder and a third decoder. The specific definitions are as follows:

[0127] Sim_SAR k (x, y) = G sar (OPTk (x, y))

[0128] k∈[1,K],x∈[1,X],y∈[1,Y]

[0129] Among them, Sim_SAR k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th simulated SAR image, OPT k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th optical image, K=31 represents the total number of SAR images after geometric correction, X=5 is the number of rows of sub-images in the 6th geometrically corrected SAR image, Y=4 is the number of columns of sub-images in the 6th geometrically corrected SAR image, G sar This represents the SAR image generator network;

[0130] The fourth encoder of the SAR image generator network uses a convolutional module of ReflectionPad2d and CIL cascaded to take a sub-image of each simulated optical image as input, perform feature encoding processing through the convolutional layer, and output an encoded feature map.

[0131] The fourth residual module of the SAR image generator network uses multiple cascaded residual block modules to input the encoded feature map output by the encoder of the SAR image generator network, perform feature enhancement through residual convolutional blocks, and output a feature map.

[0132] The fourth decoder of the SAR image generator network uses a cascaded deconvolution module of CTIR and ReflectionPad2d to input the feature map enhanced by the residual module of the SAR image generator network. The feature decoding is performed through the deconvolution layer, and the cyclic simulated SAR image sub-image corresponding to each sub-image of each optical image is output to the SAR image discriminator network.

[0133] The SAR image generator network takes a sub-image of each simulated optical image as input, passes it to the SAR image generator network via a fourth encoder, and outputs a sub-image of the cyclically simulated SAR image via a fourth decoder to the SAR image discriminator network. The specific definitions are as follows:

[0134] Cycle_SAR k (x, y) = G sar (Sim_OPT k (x, y) + Mask k (x, y))

[0135] Among them, Cycle_SAR k(x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th generated cyclic simulated SAR image, Sim_OPT k (x, y) represents the sub-image Mask corresponding to the 5th row and 4th column of the 6th generated simulated optical image. k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th mask image in the geometrically corrected SAR image, K=31 represents the total number of SAR images after geometric correction, X=5 is the number of rows of the sub-images in the 6th geometrically corrected SAR image, Y=4 is the number of columns of the sub-images in the 6th geometrically corrected SAR image, G sar This represents the SAR image generator network;

[0136] The optical image discriminator network is composed of patchGAN fully convolutional modules;

[0137] The optical image discriminator network is used to generate the loss of the simulated optical image by taking a sub-image of each simulated optical image and a sub-image of each optical image as input. The specific definition is as follows:

[0138] V opt-simopt (x, y) = D opt (Sim_OPT k (x, y), OPT k (x, y))V opt-simopt ∈[0,1]

[0139] Among them, V opt-simopt (x, y) represents the contrast loss result of a sub-image of each simulated optical image and a sub-image of each optical image, the range of which is [0, 1]. Sim_OPT k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th generated simulated optical image, OPT k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th optical image, K=31 represents the total number of SAR images after geometric correction, X=5 is the number of rows of sub-images in the 6th geometrically corrected SAR image, Y=4 is the number of columns of sub-images in the 6th geometrically corrected SAR image, D opt This represents an optical image discriminator network;

[0140] The optical image discriminator network is used to take as input a sub-image of each cyclic simulated optical image and a sub-image of each optical image, and to generate the loss of the cyclic simulated optical image, which is specifically defined as follows:

[0141] V cycleopt-opt (x, y) = D opt (Cycle_OPT k (x, y), OPTk (x, y))V cycleopt-opt ∈[0,1]

[0142] Among them, V cycleopt-opt (x, y) represents the contrast loss result of a sub-image of each cycled simulated optical image and a sub-image of each optical image, the range of which is [0, 1]. Cycle_OPT k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th generated cyclic simulated optical image, OPT. k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th optical image, K=31 represents the total number of SAR images after geometric correction, X=5 is the number of rows of sub-images in the 6th geometrically corrected SAR image, Y=4 is the number of columns of sub-images in the 6th geometrically corrected SAR image, D opt This represents an optical image discriminator network;

[0143] The SAR image discriminator network is composed of patchGAN fully convolutional modules;

[0144] The SAR image discriminator network is used as input for sub-images of each simulated SAR image and sub-images of each geometrically corrected SAR image. k (x, y), the loss for generating simulated SAR images, is defined as follows;

[0145] V sar-simsar (x, y) = D sar (Sim_SAR k (x, y), SAR k (x, y))

[0146] V sar-simsar ∈[0,1]

[0147] Among them, V sar-simsar (x, y) represents the contrast loss result of a sub-image of each simulated SAR image and a sub-image of each geometrically corrected SAR image, with the result ranging from [0, 1]. Sim_SAR k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th generated simulated SAR image. k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th geometrically corrected SAR image, K = 31 represents the total number of geometrically corrected SAR images, X = 5 is the number of rows of sub-images in the 6th geometrically corrected SAR image, Y = 4 is the number of columns of sub-images in the 6th geometrically corrected SAR image, and D sar This represents the SAR image discriminator network;

[0148] The SAR image discriminator network is used to take as input sub-images of each cyclically simulated SAR image and sub-images of each geometrically corrected SAR image, and generates the loss for the cyclically simulated SAR image, specifically defined as follows:

[0149]

[0150] V cyclesar-sar ∈[0,1]

[0151] Among them, V cyclesar-sar (x, y) represents the contrast loss result of a sub-image of each cyclically simulated SAR image and a sub-image of each geometrically corrected SAR image, with the result ranging from [0, 1]. This represents the sub-image corresponding to the 5th row and 4th column of the 6th generated cyclic simulated SAR image. k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th geometrically corrected SAR image, K = 31 represents the total number of geometrically corrected SAR images, X = 5 is the number of rows of sub-images in the 6th geometrically corrected SAR image, Y = 4 is the number of columns of sub-images in the 6th geometrically corrected SAR image, and D sar This represents the SAR image discriminator network;

[0152] The compensation adversarial network loss function model described in step 4 is specifically defined as follows:

[0153] Loss all =(Loss opt2sar +Loss sar2opt )+Loss cycle +λLoss Rec

[0154] Where λ is a hyperparameter, and Loss sar2opt Generate adversarial loss for optical images, Loss opt2sar Generate adversarial loss for SAR images, Loss cycle Loss is the loss for cycle consistency. Rec For reconstruction loss;

[0155] The optical image generation adversarial loss is defined as follows:

[0156]

[0157] Among them, Loss opt2sar SAR stands for Image Generative Adversarial Loss. k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th geometrically corrected SAR image, Sim_SAR k(x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th generated simulated SAR image, D sar This represents the SAR image discriminator network, K=31 represents the total number of SAR images after geometric correction, X=5 represents the number of sub-image rows of the 6th geometrically corrected SAR image, and Y=4 represents the number of sub-image columns of the 6th geometrically corrected SAR image.

[0158] The SAR image generation adversarial loss is defined as follows:

[0159]

[0160] Among them, Loss sar2opt OPT generates adversarial loss for optical images. k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th optical image in Sim_OPT. k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th generated simulated optical image, D opt Let K = 31 represent the optical image discriminator network, X = 5 represent the number of sub-image rows in the geometrically corrected SAR image of the 6th scene, and Y = 4 represent the number of sub-image columns in the geometrically corrected SAR image of the 6th scene.

[0161] The cycle consistency loss is defined as follows:

[0162]

[0163] Among them, Loss cycle For the cycle consistency loss, OPT k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th optical image in the optical image, Mask k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th mask image in the geometrically corrected SAR image. k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th geometrically corrected SAR image, K = 31 represents the total number of geometrically corrected SAR images, X = 5 is the number of rows of sub-images in the 6th geometrically corrected SAR image, Y = 4 is the number of columns of sub-images in the 6th geometrically corrected SAR image, G opt G represents an optical image generator network. sar This represents the SAR image generator network;

[0164] The reconstruction loss is defined as follows:

[0165]

[0166] Among them, Loss Rec For reconstruction loss, Cycle_OPT k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th generated cyclic simulated optical image. OPT represents the sub-image corresponding to the 5th row and 4th column of the 6th generated cyclic simulated SAR image. k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th optical image in the optical image, SAR k (x, y) represents the sub-image corresponding to the 5th row and 4th column of the 6th geometrically corrected SAR image, K = 31 represents the total number of geometrically corrected SAR images, X = 5 is the number of rows of sub-images in the 6th geometrically corrected SAR image, Y = 4 is the number of columns of sub-images in the 6th geometrically corrected SAR image, G opt G represents an optical image generator network. sar This represents the SAR image generator network;

[0167] Step 5: Input the optical image data to be processed, the SAR image data to be processed, and the corresponding mask image to be processed into the optimized SAR overlay compensation adversarial network model, and output the overlay compensated SAR image data.

[0168] A specific embodiment of the present invention also provides a computer-readable medium.

[0169] The computer-readable medium is a server workstation;

[0170] The server workstation stores the computer program executed by the electronic device. When the computer program runs on the electronic device, it causes the electronic device to execute the steps of the cross-domain generative adversarial SAR image compensation method according to the embodiments of the present invention.

[0171] It should be understood that any parts not described in detail in this specification belong to the prior art.

[0172] It should be understood that the above description of the preferred embodiments is quite detailed, but it should not be considered as a limitation on the scope of protection of this invention. Those skilled in the art, under the guidance of this invention, can make substitutions or modifications without departing from the scope of protection of the claims of this invention, and all such substitutions or modifications fall within the scope of protection of this invention. The scope of protection of this invention should be determined by the appended claims.

Claims

1. A cross-domain generative adversarial SAR image compensation method, characterized in that, Includes the following steps: Step 1: Within the same geographical area, acquire multiple SAR images and multiple optical images, and combine the multiple SAR images with DEM data to perform geometric correction processing to obtain multiple geometrically corrected SAR images; Step 2: Based on the pixel ratio between the corresponding region pixels of each geometrically corrected SAR image and each SAR image, obtain the mask image corresponding to each SAR image using the SAR image overlay region localization method based on spatial geometric ratio relationship; Step 3: Divide each geometrically corrected SAR image, each optical image, and the corresponding mask image of each SAR image into blocks to obtain multiple sub-images of each SAR image, multiple sub-images of each optical image, and multiple sub-images of each mask image. Step 4: Construct a cross-domain generative SAR overlay compensation adversarial network. Input each sub-image of each geometrically corrected SAR image, each sub-image of each optical image, and each sub-image of each mask image into the cross-domain generative SAR overlay compensation adversarial network to obtain each generated sub-image of each geometrically corrected SAR image and each generated sub-image of each optical image. Construct a loss function model for the compensation adversarial network and optimize it through ADAM algorithm training to obtain the optimized cross-domain generative SAR overlay compensation adversarial network. Step 5: Input the optical image data to be processed, the SAR image data to be processed, and the corresponding mask image to be processed into the optimized cross-domain generation SAR overlay compensation adversarial network, and output the overlay compensated SAR image data. The compensation adversarial network loss function model described in step 4 is specifically defined as follows: in, For hyperparameters, Generate adversarial loss for optical images. Generate adversarial loss for SAR images. For cycle consistency loss, For reconstruction loss; The optical image generation adversarial loss is defined as follows: in, SAR stands for Image Generative Adversarial Loss. This represents the sub-image corresponding to the x-th row and y-th column of the k-th geometrically corrected SAR image. This represents the sub-image corresponding to the x-th row and y-th column of the k-th generated simulated SAR image. Let X represent the SAR image discriminator network, K represent the total number of SAR images after geometric correction, X represent the number of sub-image rows of the Kth geometrically corrected SAR image, and Y represent the number of sub-image columns of the Kth geometrically corrected SAR image. The SAR image generation adversarial loss is defined as follows: in, Generate adversarial loss for optical images. This represents the sub-image corresponding to the x-th row and y-th column of the k-th optical image. This represents the sub-image corresponding to the k-th frame of the generated simulated optical image, in the x-th row and y-th column. Let X represent the optical image discriminator network, K represent the total number of SAR images after geometric correction, X represent the number of sub-image rows of the Kth geometrically corrected SAR image, and Y represent the number of sub-image columns of the Kth geometrically corrected SAR image. The cycle consistency loss is defined as follows: in, For cycle consistency loss, This represents the sub-image corresponding to the x-th row and y-th column of the k-th optical image. This represents the sub-image corresponding to the x-th row and y-th column of the k-th mask image in the mask image corresponding to the geometrically corrected SAR image. Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th geometrically corrected SAR image, K represent the total number of geometrically corrected SAR images, X represent the number of rows of sub-images in the k-th geometrically corrected SAR image, and Y represent the number of columns of sub-images in the k-th geometrically corrected SAR image. Represents an optical image generator network. This represents the SAR image generator network; The reconstruction loss is defined as follows: in, To reconstruct the loss, This represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated optical image. This represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated SAR image. This represents the sub-image corresponding to the x-th row and y-th column of the k-th optical image. Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th geometrically corrected SAR image, K represent the total number of geometrically corrected SAR images, X represent the number of rows of sub-images in the k-th geometrically corrected SAR image, and Y represent the number of columns of sub-images in the k-th geometrically corrected SAR image. Represents an optical image generator network. This represents the SAR image generator network.

2. The SAR image compensation method for cross-domain generative adversarial methods according to claim 1, characterized in that, The multiple sub-images of each geometrically corrected SAR image mentioned in step 3 are specifically defined as follows: in, Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th SAR image in the geometrically corrected SAR image, K represent the total number of SAR images after geometric correction, X is the number of sub-image rows of the k-th geometrically corrected SAR image, and Y is the number of sub-image columns of the k-th geometrically corrected SAR image. The multiple sub-images of each optical image mentioned in step 3 are specifically defined as follows: in, Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th optical image, K represent the total number of SAR images after geometric correction, X represent the number of sub-image rows of the K-th geometrically corrected SAR image, and Y represent the number of sub-image columns of the K-th geometrically corrected SAR image. The multiple sub-images of each mask image mentioned in step 3 are specifically defined as follows: in, Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th mask image in the geometrically corrected SAR image, where K represents the total number of SAR images after geometric correction, X is the number of rows of sub-images in the k-th geometrically corrected SAR image, and Y is the number of columns of sub-images in the k-th geometrically corrected SAR image.

3. The SAR image compensation method for cross-domain generative adversarial learning according to claim 1, characterized in that, Step 4, which describes the cross-domain generation of the SAR overlay compensation adversarial network, specifically includes: The cross-domain generative SAR overlay compensation adversarial network includes an optical image generator network, a SAR image generator network, an optical image discriminator network, and a SAR image discriminator network. The optical image generator network and the optical image discriminator network are cascaded together, and the SAR image generator network and the SAR image discriminator network are cascaded together to form a single network. The optical image generator network is composed of a first encoder, a first residual module, a first decoder, a second encoder, a second residual module, and a second decoder cascaded in sequence. The SAR image generator network consists of a third encoder, a third residual module, a third decoder, a fourth encoder, a fourth residual module, and a fourth decoder cascaded together.

4. The SAR image compensation method for cross-domain generative adversarial learning according to claim 3, characterized in that, The first encoder of the optical image generator network uses a convolutional module of ReflectionPad2d and CIL cascaded to take as input sub-images of each geometrically corrected SAR image and sub-images of each mask image, and performs feature encoding processing through the convolutional layer to output an encoded feature map. The first residual module of the optical image generator network uses multiple cascaded residual block modules to input the encoded feature map output by the encoder of the optical image generator network, perform feature enhancement through residual convolutional blocks, and output the enhanced feature map. The first decoder of the optical image generator network uses a cascaded deconvolution module of CTIR and ReflectionPad2d to input the feature map enhanced by the residual module of the optical image generator network. The feature is then processed by the deconvolution layer to output the simulated optical image sub-image corresponding to each sub-image of each geometrically corrected SAR image to the fourth encoder of the SAR image generator network. The optical image generator network receives sub-images of each geometrically corrected SAR image and each mask image as input. A first encoder and a first decoder output a sub-image of the simulated optical image to the fourth encoder of the SAR image generator network, specifically defined as follows: in, This represents the sub-image corresponding to the k-th frame of the generated simulated optical image, in the x-th row and y-th column. This represents the sub-image corresponding to the x-th row and y-th column of the k-th SAR image in the geometrically corrected SAR image. Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th mask image in the geometrically corrected SAR image, where K represents the total number of geometrically corrected SAR images, X is the number of rows of the sub-images in the k-th geometrically corrected SAR image, and Y is the number of columns of the sub-images in the k-th geometrically corrected SAR image. Represents an optical image generator network; The second encoder of the optical image generator network uses a convolutional module of ReflectionPad2d and CIL cascaded to take a sub-image of each simulated SAR image as input, perform feature encoding processing through the convolutional layer, and output an encoded feature map. The second residual module of the optical image generator network employs multiple cascaded residual block modules, which are used to input the encoded feature map output by the encoder of the optical image generator network, perform feature enhancement through residual convolutional blocks, and output the enhanced feature map. The second decoder of the optical image generator network uses a cascaded deconvolution module of CTIR and ReflectionPad2d to input the feature map enhanced by the residual module of the optical image generator network. The feature is then processed by the deconvolution layer to output the cyclic simulated SAR image sub-image corresponding to each sub-image of each geometrically corrected SAR image to the SAR image discriminator network. The optical image generator network receives a sub-image of each simulated SAR image as input. A second encoder and a second decoder output the cyclically generated sub-images of the simulated optical image to the SAR image discriminator network. The specific definitions are as follows: in, This represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated optical image. Let K represent the sub-image corresponding to the x-th row and y-th column of the k-th simulated SAR image, and let K represent the total number of SAR images after geometric correction. Let X be the number of rows of sub-images in the k-th geometrically corrected SAR image, and Y be the number of columns of sub-images in the k-th geometrically corrected SAR image. This represents an optical image generator network.

5. The SAR image compensation method for cross-domain generative adversarial learning according to claim 4, characterized in that, The third encoder of the SAR image generator network uses a convolutional module of ReflectionPad2d and CIL cascaded to take a sub-image of each optical image as input, perform feature encoding processing through the convolutional layer, and output an encoded feature map. The third residual module of the SAR image generator network uses multiple cascaded residual block modules to input the encoded feature map output by the encoder of the SAR image generator network, perform feature enhancement through residual convolutional blocks, and output a feature map. The third decoder of the SAR image generator network uses a cascaded deconvolution module of CTIR and ReflectionPad2d to input the feature map enhanced by the residual module of the SAR image generator network. The feature is then processed by the deconvolution layer to output the simulated SAR image sub-image corresponding to each sub-image of each optical image to the second encoder of the optical image generator network. The SAR image generator network takes a sub-image of each optical image as input and outputs it to the second encoder of the optical image generator network via a third encoder and a third decoder. The specific definitions are as follows: in, This represents the sub-image corresponding to the x-th row and y-th column of the k-th simulated SAR image. Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th optical image, K represent the total number of SAR images after geometric correction, X represent the number of rows of sub-images in the k-th geometrically corrected SAR image, and Y represent the number of columns of sub-images in the k-th geometrically corrected SAR image. This represents the SAR image generator network; The fourth encoder of the SAR image generator network uses a convolutional module of ReflectionPad2d and CIL cascaded to take a sub-image of each simulated optical image as input, perform feature encoding processing through the convolutional layer, and output an encoded feature map. The fourth residual module of the SAR image generator network uses multiple cascaded residual block modules to input the encoded feature map output by the encoder of the SAR image generator network, perform feature enhancement through residual convolutional blocks, and output a feature map. The fourth decoder of the SAR image generator network uses a cascaded deconvolution module of CTIR and ReflectionPad2d to input the feature map enhanced by the residual module of the SAR image generator network. The feature decoding is performed through the deconvolution layer, and the cyclic simulated SAR image sub-image corresponding to each sub-image of each optical image is output to the SAR image discriminator network. The SAR image generator network takes a sub-image of each simulated optical image as input, passes it to the SAR image generator network via a fourth encoder, and outputs a sub-image of the cyclically simulated SAR image via a fourth decoder to the SAR image discriminator network. The specific definitions are as follows: in, This represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated SAR image. This represents the sub-image corresponding to the x-th row and y-th column of the k-th generated simulated optical image. Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th mask image in the geometrically corrected SAR image, where K represents the total number of geometrically corrected SAR images, X is the number of rows of the sub-images in the k-th geometrically corrected SAR image, and Y is the number of columns of the sub-images in the k-th geometrically corrected SAR image. This represents the SAR image generator network.

6. The SAR image compensation method for cross-domain generative adversarial learning according to claim 4, characterized in that, The optical image discriminator network is composed of patchGAN fully convolutional modules; The optical image discriminator network is used to generate the loss of the simulated optical image by taking a sub-image of each simulated optical image and a sub-image of each optical image as input. The specific definition is as follows: in, This represents the contrast loss result for each sub-image of the simulated optical image and each sub-image of the optical image, with the result ranging from... This represents the sub-image corresponding to the k-th frame of the generated simulated optical image, in the x-th row and y-th column. Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th optical image, K represent the total number of SAR images after geometric correction, X represent the number of rows of sub-images in the k-th geometrically corrected SAR image, and Y represent the number of columns of sub-images in the k-th geometrically corrected SAR image. This represents an optical image discriminator network; The optical image discriminator network is used to take as input a sub-image of each cyclic simulated optical image and a sub-image of each optical image, and to generate the loss of the cyclic simulated optical image, which is specifically defined as follows: in, This represents the contrast loss result for a sub-image of each cyclically simulated optical image and a sub-image of each optical image, with the result ranging from... This represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated optical image. Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th optical image, K represent the total number of SAR images after geometric correction, X represent the number of rows of sub-images in the k-th geometrically corrected SAR image, and Y represent the number of columns of sub-images in the k-th geometrically corrected SAR image. This represents an optical image discriminator network.

7. The SAR image compensation method for cross-domain generative adversarial learning according to claim 5, characterized in that, The SAR image discriminator network is composed of patchGAN fully convolutional modules; The SAR image discriminator network is used as input for sub-images of each simulated SAR image and sub-images of each geometrically corrected SAR image. The loss for generating simulated SAR images is defined as follows; in, This represents the contrast loss result for a sub-image of each simulated SAR image and a sub-image of each geometrically corrected SAR image, with the result ranging from [value missing]. This represents the sub-image corresponding to the x-th row and y-th column of the k-th generated simulated SAR image. Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th geometrically corrected SAR image, K represent the total number of geometrically corrected SAR images, X represent the number of rows of sub-images in the k-th geometrically corrected SAR image, and Y represent the number of columns of sub-images in the k-th geometrically corrected SAR image. This represents the SAR image discriminator network; The SAR image discriminator network is used to take as input sub-images of each cyclically simulated SAR image and sub-images of each geometrically corrected SAR image, and generates the loss for the cyclically simulated SAR image, specifically defined as follows: in, This represents the contrast loss result for a sub-image of each cyclically simulated SAR image and a sub-image of each geometrically corrected SAR image. The range of this result is... This represents the sub-image corresponding to the x-th row and y-th column of the k-th generated cyclic simulated SAR image. Let X represent the sub-image corresponding to the x-th row and y-th column of the k-th geometrically corrected SAR image, K represent the total number of geometrically corrected SAR images, X represent the number of rows of sub-images in the k-th geometrically corrected SAR image, and Y represent the number of columns of sub-images in the k-th geometrically corrected SAR image. This represents the SAR image discriminator network.

8. A computer-readable medium, characterized in that, It stores a computer program executed by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the method as described in any one of claims 1-7.

Images