High-resolution remote sensing image fusion method based on ICGAN

Abstract

The present invention discloses a high-resolution remote sensing image fusion method based on ICGAN, and belongs to the field of image fusion technology. The present invention comprises acquiring remote sensing image data to construct an ICGAN model, training the model based on the remote sensing image data using a generator and a discriminator; using the generator to generate a high-resolution multispectral image, using the discriminator to distinguish the true from the false of the output result of the generator; finally, testing the model based on the trained generator, and comparing and analyzing the test results. The present invention has a strong ability to retain the spectral information of low-resolution multispectral images and the spatial information of panchromatic images, and the spatial details and texture features of the ICGAN fused image are richer, and the effect is better than other fusion methods.

Description

Technical Field

[0001] The present invention belongs to the technical field of image fusion, and in particular relates to a high-resolution remote sensing image fusion method based on an Improved Cycle-consistent Generative Adversarial Network (ICGAN). Background Art

[0002] In recent years, with the widespread development of machine learning, fusion methods based on deep learning have also developed rapidly in the field of remote sensing image fusion. The prior art discloses a deep learning-based convolutional neural network (PCNN) for remote sensing image fusion and transforms the convolutional neural network model by introducing different modules.

[0003] The initial PCNN model consists of a simple 3-layer convolutional neural network, which stacks the panchromatic image and the multispectral image together for convolution operation, realizing the fusion of spectral features and spatial features. However, the shallow model structure limits the nonlinear fitting ability of PCNN, resulting in a certain degree of spectral distortion in the fused image. In response to the problem of spectral distortion, the prior art discloses the use of a residual network to enhance the spatial details of the fused image to create PanNet, which greatly improves the spatial quality while alleviating the spectral distortion. By constructing a non-local attention residual network NLRNet, similar contextual correlations between all pixels are captured, achieving a good fusion effect. At the same time, taking advantage of the respective advantages of multispectral images and panchromatic images, a two-stream fusion convolutional neural network TFNet is proposed to extract the features of both, further enriching the convolutional neural network model for remote sensing image fusion.

[0004] The prior art also discloses that PSGAN first introduced a generative adversarial network for remote sensing image fusion, transforming the image feature fusion problem into a high-resolution multispectral image generation problem. By testing different generative network architectures, it is pointed out that using a dual-stream extractor in a generative adversarial network to extract spectral and spatial features is better than stacking low-resolution images and full-color images, and ultimately achieves a good fusion effect. At the same time, a generative adversarial network MDSSC-GAN SAM with a dual discriminator structure in the prior art constrains the generator from the texture information level and the color information level respectively.

[0005] In the absence of real high-resolution multispectral images, the exploration of remote sensing image fusion methods continues to attract widespread attention. Inspired by CycleGAN's cycle consistency in image style transfer, UCGAN inputs low-resolution images into the generator twice to extract the spectral information of the multispectral image, and globally pools the panchromatic image to extract spatial information. The UCGAN method simulates the spectral and spatial degradation model in a relatively coarse upsampling and downsampling manner, achieving the cycle consistency of the generator.

[0006] The cycle consistency of UCGAN limits the improvement of fusion quality to a certain extent, and the training effect is not ideal when applied to training data sets with low spatial resolution. Summary of the invention

[0007] In view of the problem that the cycle consistency in the prior art mentioned in the background technology limits the improvement of fusion quality to a certain extent, and the training effect is not ideal when applied to training data sets with low spatial resolution, the present invention proposes a high-resolution remote sensing image fusion method based on ICGAN. The present invention focuses on making the local clearer and the color more faithful while maintaining the global overall consistency, and reduces the model complexity while ensuring the fusion effect.

[0008] Technical solution: In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0009] A high-resolution remote sensing image fusion method based on ICGAN includes acquiring remote sensing image data to build an ICGAN model, training the model based on the remote sensing image data using a generator and a discriminator; using the generator to generate a high-resolution multispectral image, and using the discriminator to distinguish true from false output results of the generator; finally, testing the model based on the trained generator, and comparing and analyzing the test results.

[0010] As a preferred embodiment, the specific implementation steps are as follows:

[0011] S1: Obtain a remote sensing image training dataset and preprocess the low-resolution multispectral images and panchromatic images in the dataset;

[0012] S2: Use the generator to receive the image information in S1 and perform feature fusion and image reorganization on it, and output the fused image;

[0013] S3: Input the two output results in S2 into the discriminator to determine whether they are true or false, then return the results and calculate the loss;

[0014] S4: Obtain the remote sensing image test dataset and input it into the generator for testing, and output the test results.

[0015] Preferably, in S1, the specific contents of obtaining a remote sensing image training data set and preprocessing the low-resolution multispectral image and panchromatic image in the data set are:

[0016] S11: downsampling the low-resolution multispectral image in the remote sensing image training data set by 4 times to obtain a downsampled multispectral image, and upsampling the downsampled multispectral image by 4 times to obtain an upsampled multispectral image;

[0017] S12: Downsample the panchromatic image in the remote sensing image training data set by 4 times to obtain a downsampled panchromatic image, and replicate the downsampled panchromatic image 4 times at the channel level to obtain a replicated panchromatic image.

[0018] Preferably, in S2, a generator is used to receive the image information in S1 and perform feature fusion and image reorganization on it, and the specific content of the output fused image is:

[0019] S21: The generator receives the low-resolution multispectral image, the upsampled multispectral image and the panchromatic image after channel replication in S1, uses the spectral information extraction network to extract the spectral information of the low-resolution multispectral image and the upsampled multispectral image respectively, and uses the spatial information extraction network to extract the spatial information of the replicated panchromatic image;

[0020] S22: splicing and combining the spectral information and spatial information in S21 at the channel level and inputting them into the feature fusion network for feature fusion to obtain a feature map;

[0021] S23: Input the convolved feature map in S22 into the image reconstruction network and output the fused image.

[0022] Preferably, in S3, the two output results in S2 are input into the discriminator to judge the truth or falsehood, and then the results are returned and the loss is calculated as follows:

[0023] S31: Divide the input image into several small local areas, perform true and false discrimination on each local area, and combine the discrimination results of all local areas as the final discrimination result;

[0024] S32: Calculate adversarial loss, image reconstruction loss, image quality loss and cycle consistency loss.

[0025] Preferably, the specific content of calculating the adversarial loss in S32 is:

[0026] In ICGAN, mean square error is introduced as adversarial loss. The adversarial loss between the generated image and the true label is calculated in the generator loss. In the discriminator loss, the adversarial loss of the real input and the real label and the adversarial loss of the false input and the false label are averaged as the loss.

[0027] The adversarial loss calculation formula in the generator is:

[0028] ,

[0029] Where: D is the discriminator, Y 1 is the full-color image input to the discriminator, X is the upsampled low-resolution image input to the generator, and Y 2 is a full-color image that is input to the generator and replicated 4 times through the channel; True is the real label with the same size as the discriminator output, G represents the generator, The representative uses L2 loss;

[0030] In the discriminator, the discriminator loss formula is:

[0031] ,

[0032] Among them: True is the real label, Fake is the false label, Y 1 is the full color image input to the discriminator, X 1 is the original multispectral image, Y 2 is the full-color image input to the generator after channel replication 4 times, X 2 is the upsampled low-resolution image input to the generator; G represents the generator, The representative uses L2 loss.

[0033] Preferably, the specific content of calculating the image reconstruction loss in S32 is:

[0034] The image reconstruction loss calculation formula is as follows:

[0035] ,

[0036] Among them, X is the upsampled low-resolution image of the input generator, Y is the full-color image of the input generator after channel replication 4 times; G represents the generator, represents L1 loss.

[0037] Preferably, the specific content of calculating the image quality loss in S32 is:

[0038] The image quality loss is constructed using the quality lossless index. The image quality loss calculation formula is as follows:

[0039] ,

[0040] Among them, D λ is the spectral distortion, D s is the spatial distortion.

[0041] Preferably, the specific content of calculating the cycle consistency loss in S32 is:

[0042] The calculation formula of cycle consistency loss is as follows:

[0043] ,

[0044] Among them, A 1 is the upsampled low-resolution image, A 2 is the original multispectral image, and B is the full-color image with the channel replicated 4 times; The representative uses L2 loss.

[0045] Preferably, a remote sensing image test data set is obtained in S4 and input into the generator for testing, and the specific content of the test results is output:

[0046] The remote sensing image test dataset is preprocessed, and the preprocessed images are input into the trained generator to output the test results.

[0047] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0048] The ICGAN method proposed in this paper is applied to the remote sensing image fusion of the Gaofen-6 satellite. Compared with the results of different fusion methods, ICGAN has excellent comprehensive performance on the Gaofen-6 test data set, and has a strong ability to retain the spectral information of low-resolution multispectral images and the spatial information of panchromatic images. In addition, the spatial details and texture features of the ICGAN fused image are richer, and the effect is better than other fusion methods. There is still color distortion in some aspects, but overall the color distortion of ICGAN is not obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is the overall technical roadmap of the present invention;

[0050] Figure 2 It is a schematic diagram of the model architecture of the present invention;

[0051] Figure 3 is a schematic diagram of the generator architecture of the present invention;

[0052] Figure 4 is a schematic diagram of the discriminator architecture of the present invention;

[0053] Figure 5 It is a comparison diagram of the fusion results of different fusion methods of the present invention, where a, b, and c are all study areas. DETAILED DESCRIPTION

[0054] The present invention is further illustrated below in conjunction with specific examples. The examples are implemented based on the technical solutions of the present invention. It should be understood that these examples are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0055] like Figure 1 As shown, the high-resolution remote sensing image fusion method based on ICGAN provided in this embodiment includes acquiring remote sensing image data to build an ICGAN model, training the model based on the remote sensing image data using a generator and a discriminator; using the generator to generate a high-resolution multispectral image, and using the discriminator to distinguish the true from the false of the output result of the generator; finally, performing a model test based on the trained generator, and comparing and analyzing the test results. The specific implementation steps are as follows:

[0056] S1: Obtain a remote sensing image training dataset and preprocess the low-resolution multispectral images and panchromatic images in the dataset;

[0057] In this embodiment, the remote sensing image training set and the remote sensing image test set are both derived from the Gaofen-6 satellite.

[0058] S11: Downsample the low-resolution multispectral image (mul) in the remote sensing image training dataset by 4 times to obtain a downsampled multispectral image (lr), and upsample the downsampled multispectral image by 4 times to obtain an upsampled multispectral image (lr_u).

[0059] S12: Downsample the panchromatic image (pan) in the remote sensing image training dataset by 4 times to obtain a downsampled panchromatic image, and replicate the downsampled panchromatic image 4 times at the channel level to obtain a replicated panchromatic image (pan_extend).

[0060] S2: Use the generator G to receive the image information in S1 and perform feature fusion and image reorganization on it, and output the fused image;

[0061] S21: The generator G receives the low-resolution multispectral image, the upsampled multispectral image and the panchromatic image after channel replication in S1, and uses the spectral information extraction network to extract the spectral information of the low-resolution multispectral image and the upsampled multispectral image respectively, and uses the spatial information extraction network to extract the spatial information of the replicated panchromatic image;

[0062] First, each batch of images has three inputs, namely, low-resolution multispectral images, upsampled multispectral images, and replicated panchromatic images.

[0063] The spectral information extraction network receives inputs of low-resolution multispectral images and upsampled multispectral images, which contain 4 channels.

[0064] The spatial information extraction network receives the input of the copied full-color image, which contains 1 channel. After each network is completed, a ReLU layer is followed to reset the negative values ​​in the feature map to zero, enhance the nonlinear fitting ability, prevent the gradient from disappearing, and improve the performance of the generator.

[0065] S22: splicing and combining the spectral information and spatial information in S21 at the channel level and inputting them into the feature fusion network for feature fusion to obtain a feature map;

[0066] S23: Input the convolved feature map in S22 into the image reconstruction network and output the fused image;

[0067] S23 outputs the original low-resolution multispectral image (output_mul) and the upsampled multispectral image (output_lr); that is, a low-resolution result image and a high-resolution result image are output.

[0068] The grayscale values ​​of remote sensing images are all positive numbers, but under the continuous convolution of the internal network of the generator, the final output result may still be negative. Therefore, the experiment followed an activation function layer after the image reconstruction network of the generator to limit the final output result of the generator. After a large number of experimental tests, it was proved that if the final activation function is missing, the trained generator will generate a multispectral image with all negative values ​​in the first band. Although the spatial details and texture features of the full-color image are well preserved, the spectral features are seriously deviated from the original image.

[0069] S3: Input the two output results in S2 and the full-color image fusion result (pan) into the discriminator to judge the truth and return the result and calculate the loss;

[0070] S31: Divide the input image into local areas of fixed size based on the grid, use the trained discriminator to perform true and false judgment on each local area, and combine the judgment results of all local areas as the final judgment result.

[0071] The discriminator is a type of discriminator commonly used in generative adversarial networks. The main goal of the discriminator is to distinguish whether the input image is true or false. It does not directly judge the authenticity of the entire image, but divides the image into several small local areas (patches) and performs authenticity judgment on each local area. The final judgment result is a combination of these local judgment results.

[0072] The main parameter of the discriminator is the layer number (n_layers), which mainly affects the receptive field of the discriminator. For the data size of the input model, setting the discriminator n_layers equal to 3 and the receptive field to 70 better balances the local and global features of the neuron's perception of the image. The activation function used in the discriminator is LeakyReLU, which has the advantages of the ReLU function while also improving the stability of the discriminator training, so that the discriminator can better learn to distinguish between real images and generated images.

[0073] The advantage of this discriminator is that it only focuses on the discrimination of local areas, so it has a certain robustness to the scale change of the image. No matter how the size of the image changes, as long as the local feature structure is similar, it can be effectively discriminated.

[0074] S32: Calculate adversarial loss, image reconstruction loss, image quality loss and cycle consistency loss;

[0075] In the generative adversarial network model, the loss function plays an indispensable role. For the discriminator, it enables the discriminator to learn how to judge whether the input image is real or fake, and for the generator, it will point out why the image generated by the generator cannot be recognized by the discriminator. A single loss function will make the generator focus only on one aspect of the features and ignore the similarities at other levels. Therefore, this embodiment uses a combination of four loss functions in ICGAN to allow the generator to better start from the global perspective and generate images that are more realistic to human visual perception. The comprehensive loss function is calculated as follows:

[0076] ,

[0077] Among them, α, β, γ, and δ are weight factors of the four losses respectively. QNR loss measures the spectral and spatial losses of the fused image and the image before fusion, and has the largest weight for the strong constraint. The adversarial loss and image reconstruction loss jointly compare the spatial and spectral information of the fused image, and need to set a medium weight. In order to improve the generalization ability of the model and prevent the model from overfitting, the cyclic loss should not be set with a high weight. A large number of experiments have shown that when the weight factors are set to 0.01, 0.001, 1, and 0.0005, respectively, the fusion effect is the best. If one of the weights is changed, the model will not converge during training or the model test effect will be extremely poor.

[0078] L adv Denote adversarial loss:

[0079] In order to make the images generated by the generator more consistent with the real high-resolution images in details, this experiment introduced the mean square error as adversarial loss in ICGAN. The adversarial loss between the generated image and the real label is calculated in the generator loss, and the adversarial loss of the real input and the real label and the adversarial loss of the fake input and the fake label are used as the average loss in the discriminator loss.

[0080] The adversarial loss calculation formula in the generator is:

[0081] ,

[0082] Where: D is the discriminator, Y 1 is the full-color image input to the discriminator, X is the upsampled low-resolution image input to the generator, and Y 2 is a full-color image that is input to the generator and replicated 4 times through the channel. True is the true label with the same size as the discriminator output, which is a random number matrix close to 1 in the experiment. G represents the generator.

[0083] The formula means that the image generated by the generator is input into the discriminator, and the loss between the discriminator's judgment result and the true label is calculated. The representative uses L2 loss.

[0084] In the discriminator, the discriminator loss formula is:

[0085] ,

[0086] Among them: True is the real label, Fake is the false label, Y 1 is the full color image input to the discriminator, X 1 is the original multispectral image (2m), Y 2 is the full-color image input to the generator after channel replication 4 times, X 2 is the upsampled low-resolution image input to the generator. The reason for summing the real loss and the fake loss and dividing it by 2 is to balance the discriminator's ability to identify real images and fake images. Leaning towards either side will reduce the discriminator training performance.

[0087] L rec Represents the image reconstruction loss:

[0088] The low-frequency parts of general images are spectral features. Therefore, in order to keep the generated image consistent with the multispectral image in the low-frequency part, this experiment uses L1 loss to construct the image reconstruction loss, as shown in the formula:

[0089] ,

[0090] Where: X is the upsampled low-resolution image of the input generator, and Y is the full-color image of the input generator after channel replication 4 times. represents L1 loss.

[0091] L QNR Indicates image quality loss:

[0092] In order to keep the spectral and spatial details relatively consistent with the original image, this experiment uses the quality lossless index to construct the image quality loss. The loss calculation formula is as follows:

[0093] ,

[0094] Among them, D λ is the spectral distortion, D s is the spatial distortion.

[0095] L cyc Denotes cycle consistency loss:

[0096] The concept of cycle consistency generative adversarial network was first proposed by CycleGAN, and then developed in many different directions through several variants. For example, in UCGAN, they used full-color images to cycle twice when generating images, and the input of the second generator was the image of the first generation result after downsampling and upsampling. In ICGAN, cycle consistency is considered to be that the image generated by the generator should be consistent with the assumed high-resolution image, and in the same generation cycle, the image generated by the assumed high-resolution image should also be consistent with the image before the input generator. The specific calculation formula is:

[0097] ,

[0098] Among them: A 1 is the upsampled low-resolution image, A 2 is the original multispectral image, and B is the full-color image with the channel replicated 4 times. Through two comparison losses, the generator tries not to lose the feature information of the input image while keeping the generated image similar to the target image, thus achieving cycle consistency.

[0099] The two results are input into the discriminator to determine whether they are true or false, and then the results are returned and the loss is calculated. In this process, the comprehensive loss is used to continuously optimize the parameters of the generator and the discriminator.

[0100] S4: Obtain the remote sensing image test data set and input it into the generator for testing, and output the test results;

[0101] The remote sensing image test dataset is preprocessed, and the preprocessed images are input into the trained generator to output the test results.

[0102] Finally, the GF-6 test dataset is input into the trained generator, and the obtained results are compared visually and with the results obtained by other fusion methods in terms of evaluation indicators.

[0103] like Figure 3 As shown, the ICGAN method proposed in this embodiment is compared with fusion methods such as Schmidt orthogonalization fusion (GS), unsupervised cycle consistency generative adversarial network (UCGAN), and ratio transformation fusion (Brovey) on the GF-6 satellite test dataset.

[0104] Gram-Schmidt (GS): GS fusion is an orthogonal fusion method that linearly combines images of different bands to produce a set of orthogonal images. GS fusion can significantly improve the performance of image classification and target detection and reduce information redundancy.

[0105] Unsupervised Cycle-Consistent Generative Adversarial Networks (UCGAN): UCGAN is a hybrid loss optimization algorithm based on cycle consistency in an unsupervised learning framework. It does not require real images during training. It can be trained on full-scale images to utilize the rich spatial and spectral information of the original panchromatic and MS images. The performance of this method on the original full-scale images has been greatly improved, and clear spatial and spectral details can be obtained.

[0106] Ratio transform fusion (Brovey): Brovey transform fusion is also called ratio transform fusion. The Brovey fusion algorithm normalizes the multispectral band images and then performs multiplication band operations with the panchromatic image. Its spectral information is well preserved, but it is limited by the band and can only process information from three bands.

[0107] By comparing the results of different fusion methods, it can be seen that the ICGAN proposed by the method of the present invention has excellent comprehensive performance. First, in terms of human visual perception, the spatial details and texture features of the ICGAN fused image are richer, and the color distortion is relatively light.

[0108] In the study area a, the easier-to-distinguish features are the well-shaped buildings and the small rivers around them. It can be seen that the sharpening effects of GS, Brovey, and ICGAN are very obvious. The texture information of the edge of the well-shaped building is relatively complete, and the details along the river are richer. UCGAN has a weaker ability to retain spatial information, and some areas have serious local blurring effects, which are not significantly different from low-resolution images. However, it is the best at retaining spectral information, and it greatly follows the spectral characteristics of the original low-resolution multispectral image. Brovey has the most serious color distortion, and the green farmland is given a light blue color. The overall color tone is significantly different from other fusion results.

[0109] The most obvious feature in the study area b is the large blue-white warehouse. The comparison results show that UCGAN and ICGAN retain the spectral information of the original image better than the GS method and the Brovey method, and the overall brightness is consistent with the low-resolution multi-spectral. The traditional method focuses too much on the preservation of spatial details, but does not take into account the image tone information, resulting in some features having good texture features but dim brightness and difficult to identify.

[0110] The best performance in the study area c is GS fusion, which not only keeps the spectral information consistent with the low-resolution image, but also has almost the same spatial details as the full-color image. UCGAN's reinforcement learning of spectral information results in a brighter overall tone, which is significantly different from the low-resolution image. Although Brovey fusion also retains very good detail texture features, the color distortion is serious and the visual interpretation is obviously different. ICGAN has a good overall performance, which better balances the spectral features extracted from the low-resolution image and the spatial features extracted from the full-color image, but there are large differences in the color of some objects.

[0111] The ICGAN method of the present invention has excellent comprehensive performance on the Gaofen-6 test dataset, and has a strong ability to retain the spectral information of low-resolution multispectral images and the spatial information of panchromatic images. In addition, the spatial details and texture features of the ICGAN fused image are richer, and the effect is better than other fusion methods. There is still color distortion in some aspects, but overall the color distortion of ICGAN is not obvious.

[0112] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A high-resolution remote sensing image fusion method based on ICGAN, characterized by: It includes acquiring remote sensing image data to build an ICGAN model, training the model using a generator and a discriminator based on the remote sensing image data; using the generator to generate high-resolution multispectral images, and using the discriminator to distinguish the true from the false output of the generator; finally, testing the model based on the trained generator and comparing and analyzing the test results; the specific implementation steps are as follows: S1: Obtain a remote sensing image training dataset and preprocess the low-resolution multispectral images and panchromatic images in the dataset; S11: downsampling the low-resolution multispectral image in the remote sensing image training data set by 4 times to obtain a downsampled multispectral image, and upsampling the downsampled multispectral image by 4 times to obtain an upsampled multispectral image; S12: downsample the panchromatic image in the remote sensing image training data set by 4 times to obtain a downsampled panchromatic image, and replicate the downsampled panchromatic image 4 times at the channel level to obtain a replicated panchromatic image; S2: Use the generator to receive the image information in S1 and perform feature fusion and image reorganization on it, and output the fused image; S21: The generator receives the low-resolution multispectral image, the upsampled multispectral image and the panchromatic image after channel replication in S1, uses the spectral information extraction network to extract the spectral information of the low-resolution multispectral image and the upsampled multispectral image respectively, and uses the spatial information extraction network to extract the spatial information of the replicated panchromatic image; S22: splicing and combining the spectral information and spatial information in S21 at the channel level and inputting them into the feature fusion network for feature fusion to obtain a feature map; S23: Input the convolved feature map in S22 into the image reconstruction network and output the fused image; S3: Input the two output results in S2 into the discriminator to determine whether they are true or false, then return the results and calculate the loss; S31: Divide the input image into several small local areas, perform true and false discrimination on each local area, and combine the discrimination results of all local areas as the final discrimination result; S32: Calculate adversarial loss, image reconstruction loss, image quality loss and cycle consistency loss; By combining four loss functions, the generator can generate images that are more realistic to human vision from a global perspective. The comprehensive loss function is calculated as follows: LBCGAN=αL adv +βL rec +γL QNR +δL cyc , Among them, α, β, γ, and δ are the weight factors of the four losses, L adv represents adversarial loss; L rec represents the image reconstruction loss; L QNR Indicates image quality loss; L cyc represents cycle consistency loss; QNR loss measures the spectral and spatial losses of the fused image and the image before fusion, and adversarial loss and image reconstruction loss jointly compare the spatial and spectral information of the fused image; S4: Obtain the remote sensing image test dataset and input it into the generator for testing, and output the test results.

2. The high-resolution remote sensing image fusion method based on ICGAN according to claim 1, characterized in that: The specific content of calculating adversarial loss in S32 is: In ICGAN, mean square error is introduced as adversarial loss. The adversarial loss between the generated image and the true label is calculated in the generator loss. In the discriminator loss, the adversarial loss of the real input and the real label and the adversarial loss of the false input and the false label are averaged as the loss. The adversarial loss calculation formula in the generator is: Where: D is the discriminator, Y1 is the full-color image input to the discriminator, X is the upsampled low-resolution image input to the generator, Y2 is the full-color image input to the generator after channel replication 4 times; True is the true label with the same size as the discriminator output, G represents the generator, The representative uses L2 loss; In the discriminator, the discriminator loss formula is: Among them: True is the real label, Fake is the false label, Y1 is the full-color image input to the discriminator, X1 is the original multispectral image, Y2 is the full-color image input to the generator after channel replication 4 times, X2 is the upsampled low-resolution image input to the generator; G represents the generator, The representative uses L2 loss.

3. The high-resolution remote sensing image fusion method based on ICGAN according to claim 1, characterized in that: The specific content of calculating the image reconstruction loss in S32 is: The image reconstruction loss calculation formula is as follows: L rec =||G(X,Y)-X||1, Among them, X is the upsampled low-resolution image of the input generator, Y is the full-color image of the input generator after channel replication 4 times; G represents the generator, and || ||1 represents the L1 loss.

4. The high-resolution remote sensing image fusion method based on ICGAN according to claim 1, characterized in that: The specific content of calculating the image quality loss in S32 is: The image quality loss is constructed using the quality lossless index. The image quality loss calculation formula is as follows: L QNR =1-(1-D λ )×(1-D s ), Among them, D λ is the spectral distortion, D s is the spatial distortion.

5. The high-resolution remote sensing image fusion method based on ICGAN according to claim 1, characterized in that: The specific content of calculating the cycle consistency loss in S32 is: The calculation formula of cycle consistency loss is as follows: Among them, A1 is the upsampled low-resolution image, A2 is the original multispectral image, and B is the full-color image with the channel replicated 4 times; The representative uses L2 loss.

6. The high-resolution remote sensing image fusion method based on ICGAN according to claim 1, characterized in that: Obtain the remote sensing image test dataset in S4 and input it into the generator for testing, and output the specific content of the test results: The remote sensing image test dataset is preprocessed, and the preprocessed images are input into the trained generator to output the test results.

Images