A remote sensing image dehazing method based on generative adversarial network

Abstract

The present invention provides a remote sensing image defogging method based on a generative adversarial network, comprising the following steps: S1, constructing a foggy remote sensing image data set for training; S2, building an attention-guided residual fusion adversarial generative network for remote sensing image defogging; S3, training the attention-guided residual fusion adversarial generative network; S4, applying the attention-guided residual network to defog foggy remote sensing images; the present invention combines the extracted underlying detail features with the high-level global features to improve the fineness of the defogged image, adopts the adversarial network idea, reduces the difficulty of network training, is suitable for image-generated image tasks such as remote sensing image defogging, can more accurately sense the concentration and size of haze, effectively deals with the problem of poor global defogging effect caused by uneven haze distribution concentration in the face of real haze conditions in previous methods, and can retain the original details and colors as much as possible while defogging.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a remote sensing image defogging method based on a generative adversarial network. Background Art

[0002] In recent years, remote sensing technology has been continuously improving and has been widely used in various fields. Through post-processing of remote sensing images, useful information can be extracted from limited images as much as possible. However, when taking pictures under foggy weather conditions, due to the scattering and refraction of some particles in the atmosphere, the image quality collected by the imaging device is significantly reduced, which is specifically manifested in blurred scene details, reduced contrast, surface color attenuation, etc. With the development of science and technology, the resolution of images collected by remote sensing imaging equipment is getting higher and higher. The higher the resolution of the image, the more important information it contains. Therefore, in order to make more effective use of the information contained in the image, it is very important to defog the foggy image.

[0003] At present, there are two methods for restoring remote sensing images degraded by fog: methods based on physical models and methods based on deep learning. Methods based on physical models include atmospheric scattering attenuation model, dark channel prior method, etc. The physical model method estimates the transmittance and atmospheric light value of the input fog image based on the properties of the remote sensing image signal itself, and combines the atmospheric scattering model to invert the defogging image. However, the defogging performance of this method is closely related to the estimation accuracy of transmittance and atmospheric light, and only dilutes the negative impact of fog on remote sensing images to a certain extent.

[0004] The existing remote sensing image restoration method based on physical models has a long algorithm processing time, has a poor defogging effect on continuous areas, and the restored image is prone to oversaturation.

[0005] Many existing dehazing models rely on the atmospheric scattering model to linearly model the relationship between depth of field and brightness and saturation and learn the model parameters through training. However, there are artifacts in the dehazing results of remote sensing images, which affects the judgment.

[0006] Some existing dehazing models ignore the fact that haze has different concentrations in different areas of the image. For areas with dense haze, the dehazing conversion may be insufficient, while for areas with light haze or less haze, excessive conversion may occur, resulting in problems such as color cast and blur. Therefore, a remote sensing image dehazing method based on generative adversarial network is proposed. Summary of the invention

[0007] In view of this, an embodiment of the present invention hopes to provide a remote sensing image defogging method based on a generative adversarial network to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.

[0008] The technical solution of the embodiment of the present invention is implemented as follows: a remote sensing image defogging method based on a generative adversarial network comprises the following steps:

[0009] S1. Build a foggy remote sensing image dataset for training;

[0010] S2. Build an attention-guided residual fusion adversarial generative network for remote sensing image dehazing;

[0011] S3, training attention-guided residual fusion adversarial generative network;

[0012] S4. Apply attention-guided residual fusion adversarial generative network to dehaze foggy remote sensing images.

[0013] In some embodiments, in S1, a foggy remote sensing image dataset for training is constructed. First, a total of 840 fog-free clear images are selected from the open source aerial remote sensing image dataset DOTA and the satellite remote sensing image dataset RSD46-WHU as the original data source;

[0014] The atmospheric scattering physical model was used to infer the corresponding remote sensing fog map from 840 fog-free images. The atmospheric scattering coefficients were set to [0.08, 0.12, 0.14], and the atmospheric light value was set to [0.8, 0.9].

[0015] Each clear remote sensing image generates 6 fog images with different concentrations, resulting in a total of 5040 foggy remote sensing images;

[0016] A total of 5880 images of synthetic remote sensing images with different concentrations of fog and their corresponding real fog-free clear images are used as the experimental data set of the present invention;

[0017] Finally, the experimental dataset is divided into a training set and a test set in a ratio of 8:2, that is, 4032 foggy images and 672 clear and fog-free images are used for training, and 1008 foggy images and 168 clear and fog-free images are used for testing.

[0018] In some embodiments, in S2, an attention-guided residual fusion adversarial generative network (ARF-GAN) for remote sensing image defogging is constructed;

[0019] Build a generator for generating fog-free remote sensing images that are close to real images. The generator can learn the mapping from blurry remote sensing images to their corresponding real fog-free remote sensing images. The generator has an upsampling encoder composed of a jump residual fusion block that can extract the features of foggy remote sensing images, a downsampling decoder, and a channel attention guidance mechanism that can defog by region and degree.

[0020] Build a discriminator to help the generator achieve the best fit. Provide the generator's output and its real fog-free remote sensing image as input to the discriminator for comparison. The discriminator then feeds back the quality judgment score to the generator.

[0021] The specific steps are as follows:

[0022] S2.1, construct a jump residual fusion block for extracting features of foggy remote sensing images;

[0023] S2.2, construct a channel attention guidance mechanism for defogging by region and degree;

[0024] S2.3. Build a generator for generating fog-free remote sensing images that are close to real images;

[0025] S2.4. Build a discriminator to help the generator achieve the best fitting state.

[0026] In some embodiments, in S3, the training data set in S1 is used to train the attention-guided residual fusion adversarial generative network ARF-GAN constructed in S2, and the model parameters are optimized through the back propagation algorithm;

[0027] During the training process, ARF-GAN optimizes the loss function and adjusts the weight values ​​in the network to minimize the calculated result of the loss function, thereby achieving the best convergence effect.

[0028] In some embodiments, in S4, the real remote sensing image that needs to be dehazed is input into the trained ARF-GAN, and the image is processed by the generator of ARF-GAN to output a remote sensing image with unchanged resolution, natural color, and clear fog-free.

[0029] In some embodiments, in S2.1, the specific steps of constructing a jump residual fusion block for extracting features of a foggy remote sensing image are as follows:

[0030] S2.1.1. Build a CRB module, input the remote sensing feature map Input of the module, perform Conv convolution operation with a convolution kernel size of 3×3, a step size of 1, and a padding of 1, nonlinear enhancement of the ReLU activation function, and BatchNormalization, and then output a feature map F1 with unchanged resolution. The number of channels is one quarter of the feature map Input of the input skip residual fusion block;

[0031] S2.1.2, build three CRB modules in succession, and then perform residual connection, so that the three CRB modules have the same spatial resolution, concatenate the input and the output of all previous CRB modules in the channel dimension, and use the concatenated feature map as the input of the CRB module. After being processed by the second, third and fourth CRB modules, the output is a fused feature map with the same spatial resolution and one-fourth the number of channels of the input, which is recorded as F2, F3 and F4;

[0032] S2.1.3. Finally, the feature map is connected. The outputs F1, F2, F3 and F4 of the four CRB modules of S2.1.1 and S2.1.2 are spliced ​​in the channel dimension. The spliced ​​feature map is used as the output of the jump residual fusion module. The number of channels of the Output is restored to the number of channels of the input Input, which can be expressed as:

[0033] F output =F1+F2+F3+F4

[0034] Effectively extract the features of foggy remote sensing images, enhance the network's ability to handle complex remote sensing scenes and cope with remote sensing images taken in dense fog, while maintaining a stable network structure.

[0035] In some embodiments, in S2.2, the specific steps of constructing a channel attention guidance mechanism for defogging by region and degree are as follows:

[0036] S2.2.1. Construct the channel weight matrix, and the input remote sensing feature map is F∈R C×H×W , where C, H and W represent the number of channels, height and width respectively;

[0037] 1×N, where N = the height H × width W of the original feature map;

[0038] C×N, where N = the height H × width W of the original feature map;

[0039] Let F pass through a convolution operation Conv with a convolution kernel size of 1×1, a step size of 1, and a padding value of 0 to capture the channel information in the remote sensing image, and obtain a 1×H×W matrix. After deformation (Reshape, Rs) it becomes a 1×N (N=H×W) matrix, and then transpose (Transposition, Ts) the matrix to an N×1 matrix. Finally, let this N×1 matrix pass through a Sigmoid activation function to obtain a set of channel attention weights mapped in the range of [0, 1], and a channel weight matrix T of size N×1 d ;

[0040] Construct channel weight matrix T d The output process is expressed as:

[0041] T d =Sigmoid(Ts(RS(Conv(F))))

[0042] S2.2.2. Construct a feature map weight mapping matrix, transform the three-dimensional feature map F into a two-dimensional matrix of C×N (N=H×W), and combine the two-dimensional matrix with T d After multiplication, the channel weight mapping matrix T of the feature map is obtained. D ∈R C×1×1 , that is, the weight values ​​of C channels are obtained;

[0043] Construct the weight mapping matrix T of the feature map D The output process is expressed as:

[0044] T D =Rs(Rs(F)×T d )

[0045] S2.2.3. Weight mapping matrix T D Multiply the matrix values ​​by channel with the input feature map F to obtain the weighted feature map F c ∈R C×H×W . F c It can convey to the network whether a certain area in the image is important and the degree of defogging, so that the network can focus on the dense fog area. Therefore, the output process of the channel attention guidance mechanism is expressed as:

[0046] F c =T D ×F

[0047] A channel attention guidance mechanism for defogging based on regions and degrees can focus on the detailed information in remote sensing images, determine the fog concentration in the captured remote sensing images, locate the areas in the images that need to be converted, and effectively remove fog while avoiding excessive defogging.

[0048] In some embodiments, in S2.3, the specific steps of building a generator for generating a fog-free remote sensing image close to a real image are as follows:

[0049] S2.3.1. Build an entry module for processing foggy remote sensing images as input to the generator:

[0050] Build a convolution operation Conv with a convolution kernel size of 3×3, a stride of 1, and a padding value of 1, followed by a layer of ReLU activation function. After the foggy remote sensing image input to the generator G passes through this entry module, the output is a foggy remote sensing feature map with unchanged spatial resolution and 16 channels.

[0051] S2.3.2. Build a shallow feature downsampling encoder for extracting remote sensing images:

[0052] S2.3.2.1, build a jump residual fusion block constructed in S2.1 to form the first layer of the downsampling encoder, which outputs a remote sensing image feature map X1 with unchanged resolution. Similarly, stack three jump residual fusion blocks in sequence to form the second, third, and fourth layers of the downsampling encoder;

[0053] S2.3.2.2. Create a downsampling operation between adjacent skip residual fusion blocks, specifically, perform a convolution operation with a convolution kernel size of 3×3, a step size of 2, and a zero padding value on the feature map output by the skip residual fusion block of the previous layer. Therefore, the second, third, and fourth layers of the downsampling encoder output remote sensing image feature maps with half the resolution and doubled number of channels, denoted as X2, X3, and X4 respectively;

[0054] S2.3.3. Add auxiliary generator to construct channel attention guidance mechanism for dehazing remote sensing images:

[0055] A channel attention guidance mechanism constructed by S2.2 is built in the first layer of the generator G, and X1 is input. After being processed by this mechanism, a weighted feature map X1′ about X1 is obtained. Similarly, a channel attention guidance mechanism is built in the second, third, and fourth layers, and X2, X3, and X4 are input respectively to obtain the corresponding weighted feature maps X2′, X3′, and X4′;

[0056] S2.3.4. Build an upsampling decoder for extracting deeper features of remote sensing images:

[0057] S2.3.4.1, stack the skip residual fusion blocks constructed in S2.1 four times in sequence to form the fourth, third, second, and first layers of the upsampling decoder of the generated network;

[0058] S2.3.4.2. An upsampling operation is constructed between adjacent skip residual fusion blocks. Specifically, a transposed convolution operation with a convolution kernel size of 2×2 and a stride of 2 is performed to double the resolution of the input feature map and halve the number of channels.

[0059] S2.3.4.3, let X4 and X4′ be multiplied and fused bit by bit to form a remote sensing feature map that can further highlight the thickness of fog and the color information of remote sensing scene, and then use it as the input of the jump residual fusion block of the fourth layer of the upsampling decoder. After being processed by the jump residual fusion block, a remote sensing feature map with the same resolution as the input is output. After one upsampling operation, the feature map obtains a remote sensing feature map with doubled resolution and halved number of channels, which is recorded as Y1;

[0060] S2.3.4.4, let the remote sensing feature map fused by bitwise multiplication of Y1 and X3' be used as the input of the third-layer skip residual fusion block of the upsampling decoder. After being processed by this block, the remote sensing feature map with the same resolution as the input is output. After one upsampling operation, the remote sensing feature map with doubled resolution and halved number of channels is obtained, which is recorded as Y2;

[0061] S2.3.4.5, let Y2 and X2' be multiplied and fused bit by bit, and the obtained remote sensing feature map is input into the second layer skip residual fusion block of the upsampling decoder. The remote sensing feature map input in the second layer is successively subjected to the skip residual fusion block and an upsampling operation to obtain a remote sensing feature map with doubled resolution and halved number of channels, which is recorded as Y3;

[0062] S2.3.4.6, let Y3 and X1′ be fused bit by bit and input into the skip residual fusion block of the first layer of the upsampling decoder to obtain a remote sensing feature map with unchanged resolution and number of channels, denoted as Y4;

[0063] S2.3.4. Build the network layer for outputting the results of the generator:

[0064] Build a convolution block Conv with a convolution kernel size of 3×3, a stride of 1, and a padding of 1, followed by a layer of ReLU activation function. After Y4 is calculated by Conv and ReLU, the generator completes the defogging operation of the input foggy remote sensing image and outputs a clear defogging remote sensing image with three RGB channels.

[0065] In some embodiments, in S2.4, a discriminator is built to help the generator achieve the best fitting state, and network layers with different functions are built in sequence, including a convolution layer, a ReLU activation function layer, a batch normalization layer, and a fully connected layer, and a convolution layer with a convolution kernel size of 4×4, a number of convolution kernels of 36, and a step size of 2 is built;

[0066] Build a convolution layer with a kernel size of 3×3, 72 kernels, and a stride of 2;

[0067] Build a layer of ReLU activation function to enhance the nonlinear fitting ability of the network;

[0068] Build a convolution layer with a kernel size of 3×3, 144 kernels, and a stride of 2;

[0069] Build a batch normalization layer; build a convolution layer with a convolution kernel size of 3×3, 288 convolution kernels, and a stride of 2;

[0070] Build a layer of ReLU activation function; build a convolution layer with a convolution kernel size of 3×3, 288 convolution kernels, and a stride of 2;

[0071] Build a batch normalization layer;

[0072] Build a layer of ReLU activation function; build a convolution layer with a convolution kernel size of 3×3, a convolution kernel number of 1, and a stride of 1;

[0073] Finally, a fully connected layer is added at the end of the discriminator to integrate the results and output a value to complete the image judgment calculation;

[0074] The output value 1 indicates that the remote sensing image input to the discriminator is a truly collected fog-free image, and the output value 0 indicates that the remote sensing image input to the discriminator is a fog-free image generated by the generator.

[0075] In some embodiments, in S3, the specific steps of training the attention-guided residual fusion adversarial generation network are as follows:

[0076] S3.1. Initialize the hyperparameters required for training the model, the batch size, training rounds, and initial learning rate of the training set.

[0077] Start training the model. Input the remote sensing image to be defogged into the generator in ARF-GAN. Each time, the defogged remote sensing image generated by the generator and its corresponding real fog-free remote sensing image are input into the discriminator in the adversarial network. The discriminator will output 0 or 1, and repeat this adversarial generation learning process.

[0078] S3.2. Construct a loss function for ARF-GAN to make the details of the image generated by the generator more complete, prevent the generator from tampering with the tones in the original image, and make the generated dehazed image more realistic. Therefore, the loss function L for pixel comparison is G The specific expression is as follows:

[0079]

[0080] In the formula, e is a pixel, y(e) is the pixel value of the fog-free image generated by the corresponding generator, Y(e) is the pixel value of the real fog-free image, and N is the number of training samples at one time;

[0081] The task of the discriminator is to distinguish the real haze-free image Y from the haze-free image G(x i ), that is, output 1 for Y, and the haze-free image G(x) generated by the generator i )output 0;

[0082] Therefore, the loss function of the discriminator uses the cross entropy binary classification loss function, denoted by L D ;

[0083] The loss function of ARF-GAN is the sum of the loss function of the optimized generator and the loss function of the optimized discriminator, as shown in the following formula:

[0084] L=λ1L G +λ2L D

[0085] In the formula, λ1 and λ2 are weight parameters that need to be learned as the network model is trained.

[0086] The embodiment of the present invention has the following advantages due to the adoption of the above technical solution:

[0087] 1. The present invention improves the fineness of the dehazed image by combining the extracted underlying detail features with the high-level global features, adopts the adversarial network idea, reduces the difficulty of network training, and is suitable for image-generating tasks such as remote sensing image dehazing. It can more accurately sense the concentration and size of haze, and effectively deals with the problem of poor global dehazing effect caused by uneven haze distribution concentration in real haze situations faced by previous methods. It can retain the original details and colors as much as possible while dehazing.

[0088] 2. The present invention adopts an end-to-end technical design to learn the mapping relationship between foggy remote sensing images and fog-free remote sensing images, and uses a large number of paired foggy and fog-free remote sensing image datasets as data drive.

[0089] 3. The present invention takes a short time to process remote sensing images, has a fast response, and has a good defogging effect on continuous areas.

[0090] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0092] Figure 1 is a flow chart of the present invention;

[0093] Figure 2 The structure diagram of the attention-guided residual fusion adversarial generation network of the present invention;

[0094] Figure 3 This is a structural diagram of the fusion block of the present invention;

[0095] Figure 4 It is a flow chart of the channel attention guidance mechanism of the present invention;

[0096] Figure 5 It is a structural diagram of the generator of the present invention;

[0097] Figure 6 This is a structural diagram of the discriminator of the present invention. DETAILED DESCRIPTION

[0098] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0099] It should be noted that the terms "first", "second", "symmetrical", "array", etc. are only used to distinguish between description and position description purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the definition of "first", "symmetrical", etc. can explicitly or implicitly include one or more of these features; similarly, when the quantity of certain features is not limited in the form of words such as "two" or "three", it should be noted that this feature also explicitly or implicitly includes one or more feature quantities;

[0100] In the present invention, unless otherwise clearly specified and limited, the terms such as "installation", "connection", "fixation" and the like should be understood in a broad sense; for example, it can be a fixed connection, a detachable connection, or an integral molding; it can be a mechanical connection, a direct connection, welding, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood based on the description and drawings combined with specific circumstances.

[0101] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0102] like Figure 1-Figure 6 As shown, an embodiment of the present invention provides a remote sensing image defogging method based on a generative adversarial network, comprising the following steps:

[0103] S1. Build a foggy remote sensing image dataset for training;

[0104] S2. Build an attention-guided residual fusion adversarial generative network for remote sensing image dehazing;

[0105] S3, training attention-guided residual fusion adversarial generative network;

[0106] S4. Apply attention-guided residual fusion adversarial generative network to dehaze foggy remote sensing images.

[0107] In this embodiment, specifically, in S1, a foggy remote sensing image dataset for training is constructed. First, a total of 840 fog-free clear images are selected from the open source aerial remote sensing image dataset DOTA and the satellite remote sensing image dataset RSD46-WHU as the original data source;

[0108] The atmospheric scattering physical model was used to infer the corresponding remote sensing fog map from 840 fog-free images. The atmospheric scattering coefficients were set to [0.08, 0.12, 0.14], and the atmospheric light value was set to [0.8, 0.9].

[0109] Each clear remote sensing image generates 6 fog images with different concentrations, resulting in a total of 5040 foggy remote sensing images;

[0110] A total of 5880 images of synthetic remote sensing images with different concentrations of fog and their corresponding real fog-free clear images are used as the experimental data set of the present invention;

[0111] Finally, the experimental dataset is divided into a training set and a test set in a ratio of 8:2, that is, 4032 foggy images and 672 clear and fog-free images are used for training, and 1008 foggy images and 168 clear and fog-free images are used for testing.

[0112] In this embodiment, specifically, the atmospheric scattering model formula is as follows:

[0113] I(x)=J(x)t(x)+A(1-t(x))

[0114] I(x) represents the foggy image, J(x) represents the fog-free image, A represents the atmospheric light value, and x represents the corresponding pixel;

[0115] t(x) represents the transmittance, which is expressed as t(x)=e -βd(x) , where β represents the atmospheric scattering coefficient

[0116] In this embodiment, specifically, in S2, an attention-guided residual fusion adversarial generative network ARF-GAN (Attention-guided residual fusion adversarial generative network) for remote sensing image defogging is constructed;

[0117] Build a generator for generating fog-free remote sensing images that are close to real images. The generator can learn the mapping from blurry remote sensing images to their corresponding real fog-free remote sensing images. The generator has an upsampling encoder composed of a jump residual fusion block that can extract the features of foggy remote sensing images, a downsampling decoder, and a channel attention guidance mechanism that can defog by region and degree.

[0118] Build a discriminator to help the generator achieve the best fit. Provide the generator's output and its real fog-free remote sensing image as input to the discriminator for comparison. The discriminator then feeds back the quality judgment score to the generator.

[0119] The specific steps are as follows:

[0120] S2.1, construct a jump residual fusion block for extracting features of foggy remote sensing images;

[0121] S2.2, construct a channel attention guidance mechanism for defogging by region and degree;

[0122] S2.3. Build a generator for generating fog-free remote sensing images that are close to real images;

[0123] S2.4. Build a discriminator to help the generator achieve the best fitting state.

[0124] In this embodiment, specifically, in S3, the attention-guided residual fusion adversarial generative network ARF-GAN built in S2 needs to be trained using the training data set in S1, and the model parameters are optimized through the back-propagation algorithm;

[0125] During the training process, ARF-GAN optimizes the loss function and adjusts the weight values ​​in the network to minimize the calculated result of the loss function, thereby achieving the best convergence effect.

[0126] In this embodiment, specifically, in S4, the real remote sensing image that needs to be dehazed is input into the trained ARF-GAN, and the image is processed by the generator of ARF-GAN to output a remote sensing image with unchanged resolution, natural color, and clear fog-free.

[0127] In this embodiment, specifically, in S2.1, the specific steps of constructing a jump residual fusion block for extracting features of a foggy remote sensing image are as follows:

[0128] S2.1.1. Build a CRB module, input the remote sensing feature map Input of the module, perform Conv convolution operation with a convolution kernel size of 3×3, a step size of 1, and a padding of 1, nonlinear enhancement of the ReLU activation function, and BatchNormalization, and then output a feature map F1 with unchanged resolution. The number of channels is one quarter of the feature map Input of the input skip residual fusion block;

[0129] S2.1.2, build three CRB modules in succession, and then perform residual connection, so that the three CRB modules have the same spatial resolution, concatenate the input and the output of all previous CRB modules in the channel dimension, and use the concatenated feature map as the input of the CRB module. After being processed by the second, third and fourth CRB modules, the output is a fused feature map with the same spatial resolution and one-fourth the number of channels of the input, which is recorded as F2, F3 and F4;

[0130] S2.1.3. Finally, the feature map is connected. The outputs F1, F2, F3 and F4 of the four CRB modules of S2.1.1 and S2.1.2 are spliced ​​in the channel dimension. The spliced ​​feature map is used as the output of the jump residual fusion module. The number of channels of the Output is restored to the number of channels of the input Input, which can be expressed as:

[0131] F output =F1+F2+F3+F4

[0132] Effectively extract the features of foggy remote sensing images, enhance the network's ability to handle complex remote sensing scenes and cope with remote sensing images taken in dense fog, while maintaining a stable network structure.

[0133] In this embodiment, specifically, the embodiment of S2.1:

[0134] The remote sensing feature map Input with a spatial resolution of 128×128 and 32 channels is input into the skip residual fusion block, and after passing through a CRB module, a remote sensing feature map F1 with a resolution of 128×128 and 8 channels is obtained;

[0135] Concatenate Input and F1 in the channel dimension to obtain a remote sensing feature map with a resolution of 128×128 and 40 channels, which is input into the second CRB module to obtain a remote sensing feature map F2 with a resolution of 128×128 and 8 channels;

[0136] Input, F1 and F2 are concatenated in the channel dimension to obtain a remote sensing feature map with a resolution of 128×128 and 48 channels, which is then input into the third CRB module to obtain a remote sensing feature map F3 with a resolution of 128×128 and 8 channels;

[0137] Input, F1, F2 and F3 are concatenated in the channel dimension to obtain a remote sensing feature map with a resolution of 128×128 and 56 channels, which is then input into the third CRB module to obtain a remote sensing feature map F4 with a resolution of 128×128 and 8 channels;

[0138] Concatenate F1, F2, F3 and F4 in the channel dimension to obtain a remote sensing feature map F with a resolution of 128×128 and 32 channels. output , F output It is the Output of the jump residual fusion block.

[0139] In this embodiment, specifically, the convolution operation is the most important and most common operation part of the convolutional neural network, and parameters such as step size, convolution kernel size, and padding value need to be set for it. The earliest convolutional neural network model LeNet uses 3 convolution layers. The activation function can enhance the nonlinear expression of the neural network and learn more complex information. The ReLU activation function is a nonlinear function, and its derivation is simpler, allowing the model to converge faster. Batch Normalization is also a commonly used technology in neural networks. Its main function is to accelerate the convergence speed of the neural network, improve the stability of the model, and help prevent overfitting;

[0140] The feature map is the result of one or more convolutions, pooling and other operations on the input image in the convolutional neural network. One layer of the feature map corresponds to the result of one convolution of the input image. Therefore, the feature map after convolution is equal to the number of convolution kernels, that is, the number of channels of the output feature map after convolution operation is determined by the number of convolution kernels set during the convolution operation.

[0141] Residual connection is a commonly used technique in deep neural networks. By adding the output of the previous layer directly to the input of the current layer, the network can learn the residual (i.e., the change of the current layer relative to the previous layer), making it easier to train deep neural networks.

[0142] In this embodiment, specifically, in S2.2, the specific steps of constructing a channel attention guidance mechanism for defogging by region and degree are as follows:

[0143] S2.2.1. Construct the channel weight matrix, and the input remote sensing feature map is F∈R C×H×W , where C, H and W represent the number of channels, height and width respectively;

[0144] 1×N, where N = the height H × width W of the original feature map;

[0145] C×N, where N = the height H × width W of the original feature map;

[0146] Let F pass through a convolution operation Conv with a convolution kernel size of 1×1, a step size of 1, and a padding value of 0 to capture the channel information in the remote sensing image, and obtain a 1×H×W matrix. After deformation (Reshape, Rs) it becomes a 1×N (N=H×W) matrix, and then transpose (Transposition, Ts) the matrix to an N×1 matrix. Finally, let this N×1 matrix pass through a Sigmoid activation function to obtain a set of channel attention weights mapped in the range of [0, 1], and a channel weight matrix T of size N×1 d ;

[0147] Construct channel weight matrix T d The output process is expressed as:

[0148] T d =Sigmoid(Ts(RS(Conv(F))))

[0149] S2.2.2. Construct a feature map weight mapping matrix, transform the three-dimensional feature map F into a two-dimensional matrix of C×N (N=H×W), and combine the two-dimensional matrix with T d After multiplication, the channel weight mapping matrix T of the feature map is obtained. D ∈R C×1×1 , that is, the weight values ​​of C channels are obtained;

[0150] Construct the weight mapping matrix T of the feature map D The output process is expressed as:

[0151] T D =Rs(Rs(F)×T d )

[0152] S2.2.3. Weight mapping matrix T D Multiply the matrix values ​​by channel with the input feature map F to obtain the weighted feature map F c ∈R C×H×W . F c It can convey to the network whether a certain area in the image is important and the degree of defogging, so that the network can focus on the dense fog area. Therefore, the output process of the channel attention guidance mechanism is expressed as:

[0153] F c =T D ×F

[0154] A channel attention guidance mechanism for defogging based on regions and degrees can focus on the detailed information in remote sensing images, determine the fog concentration in the captured remote sensing images, locate the areas in the images that need to be converted, and effectively remove fog while avoiding excessive defogging.

[0155] In this embodiment, specifically, matrix deformation means that the dimension of the matrix needs to change, and both matrix transposition and matrix deformation operations can be implemented by calling methods in the deep learning framework.

[0156] In this embodiment, specifically, in S2.3, the specific steps of building a generator for generating a fog-free remote sensing image close to a real image are as follows:

[0157] S2.3.1. Build an entry module for processing foggy remote sensing images as input to the generator:

[0158] Build a convolution operation Conv with a convolution kernel size of 3×3, a stride of 1, and a padding value of 1, followed by a layer of ReLU activation function. After the foggy remote sensing image input to the generator G passes through this entry module, the output is a foggy remote sensing feature map with unchanged spatial resolution and 16 channels.

[0159] S2.3.2. Build a shallow feature downsampling encoder for extracting remote sensing images:

[0160] S2.3.2.1, build a jump residual fusion block constructed in S2.1 to form the first layer of the downsampling encoder, which outputs a remote sensing image feature map X1 with unchanged resolution. Similarly, stack three jump residual fusion blocks in sequence to form the second, third, and fourth layers of the downsampling encoder;

[0161] S2.3.2.2. Create a downsampling operation between adjacent skip residual fusion blocks, specifically, perform a convolution operation with a convolution kernel size of 3×3, a step size of 2, and a zero padding value on the feature map output by the skip residual fusion block of the previous layer. Therefore, the second, third, and fourth layers of the downsampling encoder output remote sensing image feature maps with half the resolution and doubled number of channels, denoted as X2, X3, and X4 respectively;

[0162] S2.3.3. Add auxiliary generator to construct channel attention guidance mechanism for dehazing remote sensing images:

[0163] A channel attention guidance mechanism constructed by S2.2 is built in the first layer of the generator G, and X1 is input. After being processed by this mechanism, a weighted feature map X1′ about X1 is obtained. Similarly, a channel attention guidance mechanism is built in the second, third, and fourth layers, and X2, X3, and X4 are input respectively to obtain the corresponding weighted feature maps X2′, X3′, and X4′;

[0164] S2.3.4. Build an upsampling decoder for extracting deeper features of remote sensing images:

[0165] S2.3.4.1, stack the skip residual fusion blocks constructed in S2.1 four times in sequence to form the fourth, third, second, and first layers of the upsampling decoder of the generated network;

[0166] S2.3.4.2. An upsampling operation is constructed between adjacent skip residual fusion blocks. Specifically, a transposed convolution operation with a convolution kernel size of 2×2 and a stride of 2 is performed to double the resolution of the input feature map and halve the number of channels.

[0167] S2.3.4.3, let X4 and X4′ be multiplied and fused bit by bit to form a remote sensing feature map that can further highlight the thickness of fog and the color information of remote sensing scene, and then use it as the input of the jump residual fusion block of the fourth layer of the upsampling decoder. After being processed by the jump residual fusion block, a remote sensing feature map with the same resolution as the input is output. After one upsampling operation, the feature map obtains a remote sensing feature map with doubled resolution and halved number of channels, which is recorded as Y1;

[0168] S2.3.4.4, let the remote sensing feature map fused by bitwise multiplication of Y1 and X3' be used as the input of the third-layer skip residual fusion block of the upsampling decoder. After being processed by this block, the remote sensing feature map with the same resolution as the input is output. After one upsampling operation, the remote sensing feature map with doubled resolution and halved number of channels is obtained, which is recorded as Y2;

[0169] S2.3.4.5, let Y2 and X2' be multiplied and fused bit by bit, and the obtained remote sensing feature map is input into the second layer skip residual fusion block of the upsampling decoder. The remote sensing feature map input in the second layer is successively subjected to the skip residual fusion block and an upsampling operation to obtain a remote sensing feature map with doubled resolution and halved number of channels, which is recorded as Y3;

[0170] S2.3.4.6, let Y3 and X1′ be fused bit by bit and input into the skip residual fusion block of the first layer of the upsampling decoder to obtain a remote sensing feature map with unchanged resolution and number of channels, denoted as Y4;

[0171] S2.3.4. Build the network layer for outputting the results of the generator:

[0172] Build a convolution block Conv with a convolution kernel size of 3×3, a stride of 1, and a padding of 1, followed by a layer of ReLU activation function. After Y4 is calculated by Conv and ReLU, the generator completes the defogging operation of the input foggy remote sensing image and outputs a clear defogging remote sensing image with three RGB channels.

[0173] In this embodiment, specifically, an embodiment of S2.3.2 is given below:

[0174] The remote sensing feature map with 256×256 resolution and 16 channels is input into the downsampling encoder, and after passing through the skip residual fusion block of the first layer, the remote sensing feature map with unchanged resolution and number of channels is obtained;

[0175] After one downsampling, it becomes a remote sensing feature map with a resolution of 128×128 and 32 channels, which is input into the skip residual fusion block of the second layer to obtain a remote sensing feature map with unchanged resolution and number of channels;

[0176] After one downsampling, it becomes a remote sensing feature map with a resolution of 64×64 and 64 channels, which is input into the skip residual fusion block of the third layer to obtain a remote sensing feature map with unchanged resolution and number of channels;

[0177] After one downsampling, it becomes a remote sensing feature map with a resolution of 32×32 and 128 channels, which is input into the skip residual fusion block of the fourth layer to obtain a remote sensing feature map with a resolution of 32×32 and 128 channels.

[0178] In this embodiment, specifically, the following is an embodiment of constructing a jump residual fusion block for extracting foggy remote sensing image features in S2.3.3:

[0179] The 32×32 resolution, 128-channel X4 and X4′ are bitwise multiplied and fused, and then input into the skip residual fusion block of the fourth layer, and the remote sensing feature map with a resolution of 32×32 and 128 channels is output. After one upsampling operation, it becomes a 64×64 resolution, 64-channel Y1;

[0180] Y1 and X3' are fused bitwise and input into the skip residual fusion block of the third layer, and a 64×64 resolution and 64-channel remote sensing feature map is output. After one upsampling operation, it becomes a 128×128 resolution and 32-channel Y2;

[0181] Y2 and X2' are fused bitwise and input into the skip residual fusion block of the second layer, and the remote sensing feature map with a resolution of 128×128 and 32 channels is output. After one upsampling operation, it becomes Y3 with a resolution of 256×256 and 16 channels.

[0182] Y3 and X1′ are bitwise multiplied and fused and input into the skip residual fusion block of the first layer to output the remote sensing feature map Y4 with a resolution of 256×256 and 16 channels.

[0183] In this embodiment, specifically, in S2.4, a discriminator is built to help the generator achieve the best fitting state, and network layers with different functions are built in sequence, including a convolution layer, a ReLU activation function layer, a batch normalization layer, and a fully connected layer, and a convolution layer with a convolution kernel size of 4×4, a number of convolution kernels of 36, and a step size of 2 is built;

[0184] Build a convolutional layer with a kernel size of 3×3, 72 kernels, and a stride of 2;

[0185] Build a layer of ReLU activation function to enhance the nonlinear fitting ability of the network;

[0186] Build a convolution layer with a kernel size of 3×3, 144 kernels, and a stride of 2;

[0187] Build a batch normalization layer; build a convolution layer with a convolution kernel size of 3×3, 288 convolution kernels, and a stride of 2;

[0188] Build a layer of ReLU activation function; build a convolution layer with a convolution kernel size of 3×3, 288 convolution kernels, and a stride of 2;

[0189] Build a batch normalization layer;

[0190] Build a layer of ReLU activation function; build a convolution layer with a convolution kernel size of 3×3, a convolution kernel number of 1, and a stride of 1;

[0191] Finally, a fully connected layer is added at the end of the discriminator to integrate the results and output a value to complete the image judgment calculation;

[0192] The output value 1 indicates that the remote sensing image input to the discriminator is a truly collected fog-free image, and the output value 0 indicates that the remote sensing image input to the discriminator is a fog-free image generated by the generator.

[0193] In this embodiment, specifically, the step size is an important integer parameter used to define the filter in the convolutional layer in the convolutional neural network, which represents the distance that the filter slides on the input data each time;

[0194] Fully connected layers are usually used at the end of a neural network to combine and integrate the features of previous layers to generate the final output.

[0195] In this embodiment, specifically, in S3, the specific steps of training the attention-guided residual fusion adversarial generative network are as follows:

[0196] S3.1. Initialize the hyperparameters required for training the model, set the batch size, training rounds, and initial learning rate values ​​for the training set;

[0197] Start training the model. Input the remote sensing image to be defogged into the generator in ARF-GAN. Each time, the defogged remote sensing image generated by the generator and its corresponding real fog-free remote sensing image are input into the discriminator in the adversarial network. The discriminator will output 0 or 1, and repeat this adversarial generation learning process.

[0198] S3.2. Construct a loss function for ARF-GAN to make the details of the image generated by the generator more complete, prevent the generator from tampering with the tones in the original image, and make the generated dehazed image more realistic. Therefore, the loss function L for pixel comparison is G The specific expression is as follows:

[0199]

[0200] In the formula, e is a pixel, y(e) is the pixel value of the fog-free image generated by the corresponding generator, Y(e) is the pixel value of the real fog-free image, and N is the number of training samples at one time;

[0201] The task of the discriminator is to distinguish the real haze-free image Y from the haze-free image G(x i ), that is, output 1 for Y, and the haze-free image G(x) generated by the generator i )output 0;

[0202] Therefore, the loss function of the discriminator uses the cross entropy binary classification loss function, denoted by L D ;

[0203] The loss function of ARF-GAN is the sum of the loss function of the optimized generator and the loss function of the optimized discriminator, as shown in the following formula:

[0204] L=λ1L G +λ2L D

[0205] In the formula, λ1 and λ2 are weight parameters that need to be learned as the network model is trained.

[0206] In this embodiment, specifically, an embodiment of S3 is given below:

[0207] The constructed foggy remote sensing image dataset is input into the generator of the adversarial network in batches. The appropriate batch size is selected according to the hardware and memory conditions. The batch size is set to 16. The hyperparameters of the network framework are set. The epoch is set to 180, the initial learning rate is set to 0.001, and it decays to 50% of the current learning rate every 30 epochs.

[0208] The network was trained and tested on the Ubuntu platform using the Pytorch deep learning framework. The λ1 of the loss function was initially set to 2 and λ2 to 0.8.

[0209] In this embodiment, specifically, the cross entropy binary classification loss L D It is the most commonly used loss function for the discriminator in the generative adversarial neural network, as shown below:

[0210]

[0211] The back propagation algorithm is an existing optimization algorithm for training neural networks. It can continuously adjust the weights in the neural network to minimize the error between the model's predicted output and the actual label.

[0212] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of various changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A remote sensing image defogging method based on a generative adversarial network, characterized in that: The following steps are involved: S1. Build a foggy remote sensing image dataset for training; S2. Build an attention-guided residual fusion adversarial generative network for remote sensing image dehazing; Build an attention-guided residual fusion adversarial generative network (ARF-GAN) for remote sensing image dehazing. Build a generator for generating fog-free remote sensing images that are close to real images. The generator can learn the mapping from blurry remote sensing images to their corresponding real fog-free remote sensing images. The generator has an upsampling encoder composed of a jump residual fusion block that can extract the features of foggy remote sensing images, a downsampling decoder, and a channel attention guidance mechanism that can defog by region and degree. Build a discriminator to help the generator achieve the best fit. Provide the generator's output and its real fog-free remote sensing image as input to the discriminator for comparison. The discriminator then feeds back the quality judgment score to the generator. The specific steps are as follows: S2.

1. Constructing a jump residual fusion block for extracting features of foggy remote sensing images. The specific steps are as follows: S2.1.

1. Build a CRB module, input the remote sensing feature map Input of the module, perform Conv convolution operation with a convolution kernel size of 3×3, a step size of 1, and a padding of 1, nonlinear enhancement of the ReLU activation function, and BatchNormalization, and then output a feature map F1 with unchanged resolution. The number of channels is one quarter of the feature map Input of the input skip residual fusion block; S2.1.2, build three CRB modules in succession, and then perform residual connection to make the three CRB modules have the same spatial resolution, concatenate the input and the output of all previous CRB modules in the channel dimension, and use the concatenated feature map as the input of the CRB module. After being processed by the second, third and fourth CRB modules, the output is a fused feature map with the same spatial resolution and one-fourth the number of channels of the input, denoted as F2, F3 and F4; S2.1.

3. Finally, the feature map is connected. The outputs F1, F2, F3 and F4 of the four CRB modules of S2.1.1 and S2.1.2 are spliced ​​in the channel dimension. The spliced ​​feature map is used as the output of the jump residual fusion module. The number of channels of the Output is restored to the number of channels of the input Input, which can be expressed as: <h2 style=";text-align:left;direction:ltr">F<h2 style=";text-align:left;direction:ltr"> output <h2 style=";text-align:left;direction:ltr"> (F1+F2+F3+F4) Effectively extract the features of foggy remote sensing images, enhance the network's ability to handle complex remote sensing scenes and remote sensing images taken in dense fog, while maintaining a stable network structure; S2.

2. The specific steps of constructing the channel attention guidance mechanism for defogging by region and degree are as follows: S2.2.

1. Construct the channel weight matrix, and the input remote sensing feature map is F∈R C×H×W , where C, H and W represent the number of channels, height and width respectively; 1×N, where N = the height H × width W of the original feature map; C×N, where N = the height H × width W of the original feature map; Let F pass through a convolution operation Conv with a convolution kernel size of 1×1, a step size of 1, and a padding value of 0 to capture the channel information in the remote sensing image, and obtain a 1×H×W matrix. After deformation (Reshape, Rs) it becomes a 1×N matrix, and then the matrix is ​​transposed (Transposition, Ts) to an N×1 matrix. Finally, let this N×1 matrix pass through a Sigmoid activation function to obtain a set of channel attention weights mapped in the range of [0, 1], and a channel weight matrix T of size N×1 d ; Construct channel weight matrix T d The output process is expressed as: T d =Sigmoid(Ts(RS(Conv(F)))) S2.2.

2. Construct a feature map weight mapping matrix, transform the three-dimensional feature map F into a C×N two-dimensional matrix, and combine the two-dimensional matrix with T d After multiplication, the channel weight mapping matrix T of the feature map is obtained. D ∈R C×1×1 , that is, the weight values ​​of C channels are obtained; Construct the weight mapping matrix T of the feature map D The output process is expressed as: T D =Rs(Rs(F)×T d ) S2.2.

3. Weight mapping matrix T D Multiply the matrix values ​​by channel with the input feature map F to obtain the weighted feature map F c ∈R C×H×W , F c It can convey to the network whether a certain area in the image is important and the degree of defogging, so that the network can focus on the dense fog area. Therefore, the output process of the channel attention guidance mechanism is expressed as: F c =T D ×F A channel attention guidance mechanism for defogging by region and degree. This mechanism can focus on the details in the remote sensing image, determine the fog density in the captured remote sensing image, locate the area in the image that needs to be converted, effectively defogging while avoiding over-defogging; S2.

3. The specific steps of building a generator for generating fog-free remote sensing images close to real images are as follows: S2.3.

1. Build an entry module for processing foggy remote sensing images as input to the generator: Build a convolution operation Conv with a convolution kernel size of 3×3, a stride of 1, and a padding value of 1, followed by a layer of ReLU activation function. After the foggy remote sensing image input to the generator G passes through this entry module, the output is a foggy remote sensing feature map with unchanged spatial resolution and 16 channels. S2.3.

2. Build a shallow feature downsampling encoder for extracting remote sensing images: S2.3.2.1, build a jump residual fusion block constructed in S2.1 to form the first layer of the downsampling encoder, which outputs a remote sensing image feature map X1 with unchanged resolution. Similarly, stack three jump residual fusion blocks in sequence to form the second, third, and fourth layers of the downsampling encoder; S2.3.2.

2. Create a downsampling operation between adjacent skip residual fusion blocks, specifically, perform a convolution operation with a convolution kernel size of 3×3, a step size of 2, and a zero padding value on the feature map output by the skip residual fusion block of the previous layer. Therefore, the second, third, and fourth layers of the downsampling encoder output remote sensing image feature maps with half the resolution and doubled number of channels, denoted as X2, X3, and X4 respectively; S2.3.

3. Add auxiliary generator to construct channel attention guidance mechanism for dehazing remote sensing images: A channel attention guidance mechanism constructed by S2.2 is built in the first layer of the generator G, and X1 is input. After being processed by this mechanism, a weighted feature map X1' about X1 is obtained. Similarly, a channel attention guidance mechanism is built in the second, third, and fourth layers, and X2, X3, and X4 are input respectively to obtain the corresponding weighted feature maps X2', X3', and X4'; S2.3.

4. Build an upsampling decoder for extracting deeper features of remote sensing images: S2.3.4.1, stack the skip residual fusion blocks constructed in S2.1 four times in sequence to form the fourth, third, second, and first layers of the upsampling decoder of the generated network; S2.3.4.

2. An upsampling operation is constructed between adjacent skip residual fusion blocks. Specifically, a transposed convolution operation with a convolution kernel size of 2×2 and a stride of 2 is performed to double the resolution of the input feature map and halve the number of channels. S2.3.4.3, let X4 and X4' be multiplied and fused bit by bit to form a remote sensing feature map that can further highlight the thickness of fog and the color information of remote sensing scene, and then use it as the input of the jump residual fusion block of the fourth layer of the upsampling decoder. After being processed by the jump residual fusion block, a remote sensing feature map with the same resolution as the input is output. After one upsampling operation, the feature map is obtained with doubled resolution and halved number of channels, which is recorded as Y1; S2.3.4.4, let the remote sensing feature map fused by bitwise multiplication of Y1 and X3' be used as the input of the third-layer skip residual fusion block of the upsampling decoder. After being processed by this block, the remote sensing feature map with the same resolution as the input is output. After one upsampling operation, the remote sensing feature map with doubled resolution and halved number of channels is obtained, which is recorded as Y2; S2.3.4.5, let Y2 and X2' be multiplied and fused bit by bit, and the obtained remote sensing feature map is input into the second layer skip residual fusion block of the upsampling decoder. The remote sensing feature map input in the second layer is successively subjected to the skip residual fusion block and an upsampling operation to obtain a remote sensing feature map with doubled resolution and halved number of channels, which is recorded as Y3; S2.3.4.6, let Y3 and X1' be fused bit by bit multiplication, and input into the skip residual fusion block of the first layer of the upsampling decoder to obtain the remote sensing feature map with unchanged resolution and number of channels, denoted as Y4; S2.3.

4. Build the network layer for outputting the results of the generator: Build a convolution block Conv with a convolution kernel size of 3×3, a stride of 1, and a padding of 1, followed by a layer of ReLU activation function. After Y4 is calculated by Conv and ReLU, the generator completes the defogging operation of the input foggy remote sensing image and outputs a clear defogging remote sensing image with three RGB channels. S2.

4. Build a discriminator to help the generator achieve the best fitting state; S3, training attention-guided residual fusion adversarial generative network; S4. Apply attention-guided residual fusion adversarial generative network to dehaze foggy remote sensing images.

2. The remote sensing image defogging method based on a generative adversarial network according to claim 1, characterized in that: In S1, a foggy remote sensing image dataset for training is constructed. First, a total of 840 fog-free clear images are selected from the open source aerial remote sensing image dataset DOTA and the satellite remote sensing image dataset RSD46-WHU as the original data source; The atmospheric scattering physical model was used to infer the corresponding remote sensing fog map from 840 fog-free images. The atmospheric scattering coefficients were set to [0.08, 0.12, 0.14], and the atmospheric light value was set to [0.8, 0.9]. Each clear remote sensing image generates 6 fog images with different concentrations, resulting in a total of 5040 foggy remote sensing images; A total of 5880 images of synthetic remote sensing images with different concentrations of fog and their corresponding real fog-free clear images are used as the experimental data set of the present invention; Finally, the experimental dataset is divided into a training set and a test set in a ratio of 8:2, that is, 4032 foggy images and 672 clear and fog-free images are used for training, and 1008 foggy images and 168 clear and fog-free images are used for testing.

3. The remote sensing image defogging method based on generative adversarial network according to claim 1, characterized in that: In S3, the training data set in S1 is used to train the attention-guided residual fusion adversarial generative network ARF-GAN built in S2, and the model parameters are optimized through the back propagation algorithm; During the training process, ARF-GAN optimizes the loss function and adjusts the weight values ​​in the network to minimize the calculated result of the loss function, thereby achieving the best convergence effect.

4. The remote sensing image defogging method based on a generative adversarial network according to claim 1, characterized in that: In S4, the real remote sensing image to be dehazed is input into the trained ARF-GAN, and the image is processed by the generator of ARF-GAN to output a remote sensing image with unchanged resolution, natural color, and clear fog-free.

5. The remote sensing image defogging method based on generative adversarial network according to claim 1, characterized in that: In S2.4, a discriminator is built to help the generator achieve the best fitting state, and network layers with different functions are built in sequence, including a convolution layer, a ReLU activation function layer, a batch normalization layer, and a fully connected layer. A convolution layer with a convolution kernel size of 4×4, a number of convolution kernels of 36, and a step size of 2 is built; Build a convolutional layer with a kernel size of 3×3, 72 kernels, and a stride of 2; Build a layer of ReLU activation function to enhance the nonlinear fitting ability of the network; Build a convolution layer with a kernel size of 3×3, 144 kernels, and a stride of 2; Build a batch normalization layer; build a convolution layer with a convolution kernel size of 3×3, 288 convolution kernels, and a stride of 2; Build a layer of ReLU activation function; build a convolution layer with a convolution kernel size of 3×3, 288 convolution kernels, and a stride of 2; Build a batch normalization layer; Build a layer of ReLU activation function; build a convolution layer with a convolution kernel size of 3×3, a convolution kernel number of 1, and a stride of 1; Finally, a fully connected layer is added at the end of the discriminator to integrate the results and output a value to complete the image judgment calculation; The output value 1 indicates that the remote sensing image input to the discriminator is a truly collected fog-free image, and the output value 0 indicates that the remote sensing image input to the discriminator is a fog-free image generated by the generator.

6. The remote sensing image defogging method based on a generative adversarial network according to claim 1, characterized in that: In S3, the specific steps of training the attention-guided residual fusion adversarial generation network are as follows: S3.

1. Initialize the hyperparameters required for training the model, the batch size, training rounds, and initial learning rate of the training set. Start training the model. Input the remote sensing image to be defogged into the generator in ARF-GAN. Each time, the defogged remote sensing image generated by the generator and its corresponding real fog-free remote sensing image are input into the discriminator in the adversarial network. The discriminator will output 0 or 1, and repeat this adversarial generation learning process. S3.

2. Construct a loss function for ARF-GAN to make the details of the image generated by the generator more complete, prevent the generator from tampering with the tones in the original image, and make the generated dehazed image more realistic. Therefore, the loss function L for pixel contrast G The specific expression is as follows: In the formula, e is a pixel, y(e) is the pixel value of the fog-free image generated by the corresponding generator, Y(e) is the pixel value of the real fog-free image, and N is the number of training samples at one time; The task of the discriminator is to distinguish the real haze-free image Y from the haze-free image G(x i ), that is, output 1 for Y, and the fog-free image G9x generated by the generator i )output 0; Therefore, the loss function of the discriminator uses the cross entropy binary classification loss function, denoted by L D ; The loss function of ARF-GAN is the sum of the loss function of the optimized generator and the loss function of the optimized discriminator, as shown in the following formula: L=λ1L G +λ2L D In the formula, λ1 and λ2 are weight parameters that need to be learned as the network model is trained.

Images