High-resolution aerosol estimation method based on conditional generative adversarial network

Abstract

The present invention provides a high-resolution aerosol estimation method based on a conditional generative adversarial network, comprising: constructing a conditional generative adversarial network, the conditional generative adversarial network comprising a generator network and a discriminator network; the generator network improves the resolution of an input image through two stages of preliminary resolution improvement and detail quality improvement; constructing a sample set, the sample set comprising an original high-resolution AOD image, environmental feature data, an estimated high-resolution AOD image and a low-resolution AOD image; training the generator network through the sample set, the trained generator network is used to estimate the high-resolution AOD image for the low-resolution AOD image to be processed; wherein the low-resolution AOD image is used as a spatial condition, and the environmental feature data is used as an environmental condition to train the generator network. The present invention can achieve accurate estimation of high-resolution AOD images.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing information technology, and in particular to a high-resolution aerosol estimation method based on a conditional generative adversarial network. Background Art

[0002] Atmospheric aerosol is a gas dispersion system composed of solid or liquid particles suspended in the air. Aerosols have direct and indirect effects on the Earth's radiation balance and climate through complex interactions with solar radiation, clouds, and atmospheric chemicals. Harmful aerosol particles can also damage human health and increase the risk of cardiovascular and respiratory diseases. Researchers usually define the vertical integral of the atmospheric extinction coefficient as the aerosol optical depth (AOD) and use it as a key parameter in aerosol-related research.

[0003] High-precision AOD products with complete temporal and spatial coverage can promote various aerosol-related studies. Existing AOD acquisition methods can be divided into three categories: ground observation, satellite retrieval, and model simulation. The Aerosol Robotic Network (AERONET) can provide accurate global distribution observations of AOD, but the spatial coverage is limited. Simulation of AOD through atmospheric chemical models, such as the Global 3-D Atmospheric Chemical Transport Model (GEOS-Chem), can generate aerosol profiles and column concentration data with high temporal resolution, but the spatial resolution and accuracy are still limited. Satellite retrieval can obtain AOD with high spatial coverage, such as the Moderate Resolution Imaging Spectroradiometer (MODIS) and the geostationary satellite Himawari-8. However, due to the limited swath width, the influence of cloud cover, and the inherent limitations of the AOD retrieval algorithm theory, the AOD products of a single satellite sensor still have defects in accuracy and temporal and spatial integrity, but the AOD products from different satellite sensors can complement each other. Therefore, by fusing satellite AOD products from different satellite sensors, high-precision AOD products with complete temporal and spatial coverage can be obtained.

[0004] At present, scholars at home and abroad have achieved certain results in the study of estimating high-resolution aerosols. When fusing multi-sensor AOD products, the pixel values ​​of multi-source AOD products are mainly used to fill in the missing values ​​of AOD products at the same location. For example, through the functional relationship of AOD values ​​of different satellite sensors at the same geographical location, researchers have developed polynomial regression models, maximum likelihood estimation models, least squares estimation models, optimal interpolation, empirical orthogonal functions, and some simplified hybrid methods. In addition, there are some geostatistical methods, including universal kriging, geostatistical inverse modeling, and spatial statistical data fusion. However, these interpolation methods usually lead to smooth diffusion, which increases the uncertainty of AOD and cannot guarantee the accuracy of high-resolution AOD image estimation. Summary of the invention

[0005] The present invention provides a high-resolution aerosol estimation method based on a conditional generative adversarial network, which is used to solve the defect of low precision of high-resolution AOD image estimation results in the prior art, and to achieve accurate estimation of the high-resolution AOD image, so as to ensure that the obtained high-resolution AOD image is closer to the actual distribution.

[0006] The present invention provides a high-resolution aerosol estimation method based on a conditional generative adversarial network, comprising:

[0007] Constructing a conditional generative adversarial network, the conditional generative adversarial network includes a generator network and a discriminator network; the generator network improves the resolution of an input image through two stages, wherein the first stage performs a preliminary resolution improvement on the input image, and the second stage performs a detail quality improvement on the input image after the preliminary resolution improvement;

[0008] Constructing a sample set, the sample set comprising an original high-resolution AOD image, environmental feature data, an estimated high-resolution AOD image and a low-resolution AOD image; wherein the low-resolution AOD image is obtained by downsampling the original high-resolution AOD image; and the estimated high-resolution AOD image is obtained by inputting the low-resolution AOD image and the environmental feature data into the generator network;

[0009] The generator network is trained by the sample set to obtain the trained generator network, and the trained generator network is used to estimate the high-resolution AOD image for the low-resolution AOD image to be processed; wherein the low-resolution AOD image is used as a spatial condition, and the environmental feature data is used as an environmental condition to train the generator network; during the training process, the sample set is input into the discriminator network, and the generator network is supervised and trained according to the output of the discriminator network; if the preset conditions are met, the trained generator network is obtained, otherwise, the parameters of the generator network are adjusted, and the generator network is trained again based on the loss function.

[0010] According to a high-resolution aerosol estimation method based on a conditional generative adversarial network provided by the present invention, after obtaining the trained generator network, the method further includes: obtaining a low-resolution AOD image to be processed, inputting the low-resolution AOD image to be processed and the corresponding environmental feature data into the trained generator network to obtain a high-resolution AOD image; wherein the original high-resolution AOD image in the sample set adopts MODIS AOD data with high spatial and low temporal resolution; the low-resolution AOD image to be processed adopts Himawari AOD data with low spatial and high temporal resolution, and the obtained high-resolution AOD image is AOD estimation data with high spatial and high temporal resolution.

[0011] According to a high-resolution aerosol estimation method based on a conditional generative adversarial network provided by the present invention, the generator network includes an encoder-decoder network and a residual network connected in sequence;

[0012] The encoder-decoder network is used to input the low-resolution AOD image, and generate a first high-resolution AOD image according to the low-resolution AOD image, thereby completing the initial resolution improvement of the low-resolution AOD image;

[0013] The residual network is used to input the first high-resolution AOD image and the environmental feature data, correct the first high-resolution AOD image according to the environmental feature data, obtain the estimated high-resolution AOD image, and improve the detail quality of the low-resolution AOD image.

[0014] According to a high-resolution aerosol estimation method based on a conditional generative adversarial network provided by the present invention, the supervised training of the generator network according to the output of the discriminator network includes:

[0015] The sample set is input into the discriminator network, and the original high-resolution AOD image and the estimated high-resolution AOD image are judged by the discriminator network according to the downsampling conditions of the low-resolution AOD image and the environmental feature data, and the generator network is supervised and trained according to the judgment result of the discriminator network; if the discriminator network cannot distinguish between the original high-resolution AOD image and the estimated high-resolution AOD image, the trained generator network is obtained, otherwise, the training fails.

[0016] According to a high-resolution aerosol estimation method based on a conditional generative adversarial network provided by the present invention, the secondary training of the generator network based on the loss function includes:

[0017] The parameters of the discriminator network are fixed, and the parameters of the generator network are adjusted. According to the mutual adversarial feedback between the generator network and the discriminator network, the generator network and the discriminator network are updated alternately to obtain the trained generator network; wherein, the generator network and the discriminator network are updated alternately with the goal of minimizing the loss function.

[0018] According to a high-resolution aerosol estimation method based on a conditional generative adversarial network provided by the present invention, the loss function is shown as follows:

[0019] Loss_1(θ G ,θ D )=-log D(M HR , M′ LR )-log(1-D(G(M′ LR ), M′ LR ))

[0020] Loss_2(θ G ,θ D )=-log D(M HR , E)-log(1-D(G(M′ LR ), E))

[0021]

[0022] L(θ G ,θ D )=Loss SC (θ G )+λ1Loss_1(θ G ,θ D )+λ2Loss_2(θ G ,θ D )

[0023] In the formula, M′LR 、M HR , M′ HR , E are respectively the low-resolution AOD image, the original high-resolution AOD image, the estimated high-resolution AOD image, and the environmental feature data; θ G ,θ D are the parameters of the generator network and the discriminator network respectively; Loss_1(θ G ,θ D )、Loss_2(θ G ,θ D ) are respectively the adversarial loss based on the low-resolution AOD image and the adversarial loss based on the environmental data; D() is the probability value output by the discriminator network; G() represents the generator network; Loss SC (θ G ) represents the spatial content loss; n represents the number of pixels of the original high-resolution AOD image; M HR，i , M′ HR，i denote the i-th pixel in the original high-resolution AOD image and the estimated high-resolution AOD image respectively; ρ() denotes a penalty function; L(θ G ,θ D ) represents the overall loss function of the conditional generative adversarial network; λ1 and λ2 are weight coefficients respectively.

[0024] The high-resolution aerosol estimation method based on conditional generative adversarial network provided by the present invention proposes a new type of generator network. The generator network improves the resolution of the input low-resolution AOD image through two stages. The first stage performs preliminary resolution improvement, and the second stage performs detail quality improvement. In the process of training the generator network, the low-resolution AOD image is used as a spatial condition, and the environmental feature data is used as an environmental condition, so that the deep representation of the AOD data and the interaction between the AOD data and the local environmental pattern can be captured, so that more realistic details can be estimated, and the accuracy of the high-resolution AOD image estimation is effectively improved, and a high-resolution AOD image that is closer to the real distribution can be obtained. At the same time, the present invention trains the generator network in two stages. The first stage supervises the training of the generator network through the output of the discriminator network, and the second stage corrects the estimation error through a loss function containing spatial and environmental content, further improving the accuracy of the high-resolution AOD image estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is one of the flow charts of the high-resolution aerosol estimation method based on conditional generative adversarial network provided by the present invention;

[0027] Figure 2 This is the second flow chart of the high-resolution aerosol estimation method based on conditional generative adversarial network provided by the present invention;

[0028] Figure 3 It is a schematic diagram of the structure of the conditional generative adversarial network provided by the present invention;

[0029] FIG4( a) is a schematic diagram of original high temporal resolution Himawari AOD data provided by the present invention;

[0030] FIG4( b) is a schematic diagram of Himawari AOD data with high temporal and spatial resolution generated by the present invention;

[0031] Figure 5 It is a structural schematic diagram of a high-resolution aerosol estimation device based on a conditional generative adversarial network provided by the present invention;

[0032] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] Combine the following Figure 1-Figure 3 The high-resolution aerosol estimation method based on conditional generative adversarial network of the present invention is described, comprising the following steps:

[0035] S100. Construct a conditional generative adversarial network, wherein the conditional generative adversarial network includes a generator network and a discriminator network; the generator network improves the resolution of an input image through two stages, wherein the first stage performs a preliminary resolution improvement on the input image, and the second stage performs a detail quality improvement on the input image after the preliminary resolution improvement.

[0036] S200, constructing a sample set, wherein the sample set includes an original high-resolution AOD image, environmental feature data, an estimated high-resolution AOD image and a low-resolution AOD image; wherein the low-resolution AOD image is obtained by downsampling the original high-resolution AOD image; and the estimated high-resolution AOD image is obtained by inputting the low-resolution AOD image and the environmental feature data into the generator network.

[0037] S300, training the generator network through the sample set to obtain the trained generator network, and the trained generator network is used to estimate the high-resolution AOD image for the low-resolution AOD image to be processed; wherein, the low-resolution AOD image is used as a spatial condition, and the environmental feature data is used as an environmental condition to train the generator network; during the training process, the sample set is input into the discriminator network, and the generator network is supervised and trained according to the output of the discriminator network; if the preset conditions are met, the trained generator network is obtained, otherwise, the parameters of the generator network are adjusted, and the generator network is trained twice based on the loss function.

[0038] S400, obtaining a low-resolution AOD image to be processed, inputting the low-resolution AOD image to be processed and the corresponding environmental feature data into the trained generator network to obtain a high-resolution AOD image.

[0039] Wherein, in step S100, the generator network includes an encoder-decoder network and a residual network connected in sequence;

[0040] The encoder-decoder network is used to input the low-resolution AOD image, and generate a first high-resolution AOD image based on the low-resolution AOD image, thereby completing the initial resolution improvement of the low-resolution AOD image; wherein the encoder-decoder network includes an encoder and a decoder connected in sequence, the encoder includes a plurality of two-dimensional convolutional layers for converting the low-resolution AOD image into a specific spatial vector, and each of the two-dimensional convolutional layers is activated by a LeakyReLU function; the decoder includes a plurality of two-dimensional transposed convolutional layers for generating the first high-resolution AOD image based on the spatial vector, and each two-dimensional transposed convolutional layer is activated by a ReLU function; the number of the two-dimensional convolutional layers and the two-dimensional transposed convolutional layers is equal, and the convolution kernel size and stride length are correspondingly equal, that is, each encoder layer performs zero padding using a convolution with a given convolution kernel and stride length, and each decoder layer implements upsampling of the feature map by using the same transposed convolution as the encoder layer; the output of the encoder-decoder network uses a Tanh activation function. In this embodiment, the number of two-dimensional convolutional layers and two-dimensional transposed convolutional layers is 3.

[0041] The residual network is used to input the first high-resolution AOD image and the environmental feature data, and to correct the first high-resolution AOD image according to the environmental feature data to obtain the estimated high-resolution AOD image that is closer to the real details, thereby improving the detail quality of the low-resolution AOD image; wherein the residual network includes a plurality of residual blocks connected in sequence and a first convolutional layer, the residual block is used to correct the estimation error of the first high-resolution AOD image according to the environmental feature data, and output a feature map with spatial details; the first convolutional layer is used to output the estimated high-resolution AOD image according to the feature map with spatial details. In this embodiment, the number of residual blocks is 6, and the structure of each residual block is the same, including a second convolutional layer, a first batch processing normalization layer, a LeakyReLU activation function layer, a third convolutional layer, a second batch processing normalization layer, and a first connection layer connected in sequence; wherein the convolution kernel size of the second convolutional layer and the third convolutional layer is 3x3, which is used to extract features of the first high-resolution AOD image; the first batch processing normalization layer and the second batch processing normalization layer are used to prevent gradient disappearance; the ReLU activation function layer is used to increase the nonlinearity of the network to prevent gradient disappearance; the first connection layer uses skip connections to add low-dimensional image features to high-dimensional image features pixel by pixel to obtain a feature map with spatial details.

[0042] By innovating the generator network structure and introducing two-stage networks, the encoder-decoder network and the residual network, it is possible to capture the deep representation of AOD data and the interaction between AOD data and local environmental patterns, thereby estimating more realistic details.

[0043] The discriminator network includes a fourth convolutional layer, a second connection layer, a fifth convolutional layer, a sixth convolutional layer, and a seventh convolutional layer connected in sequence. The fourth convolutional layer, the fifth convolutional layer, and the sixth convolutional layer are all activated by the LeakyReLU activation function, and the seventh convolutional layer is activated by the Sigmod activation; the fourth convolutional layer is used to convolve the original high-resolution AOD image and the estimated high-resolution AOD image with the low-resolution AOD image and the environmental condition data, respectively, to obtain a feature map; the second connection layer is used to merge the convolved feature map and use it as the input of the fifth convolutional layer; the fifth convolutional layer, the sixth convolutional layer, and the seventh convolutional layer are used for further feature extraction; the LeakyReLU activation function and the Sigmod activation function are used to perform nonlinear feature transformation on the data.

[0044] In step S200, the method for acquiring the original high-resolution AOD image includes:

[0045] Obtain historical high-resolution AOD images, use an active selection window to divide each of the historical high-resolution AOD images into several image blocks, select a target image block from the image block according to a preset threshold, and use the neighboring mean to fill the missing values ​​of the target image block to obtain the original high-resolution AOD image. Among them, the historical high-resolution AOD image uses a high spatial and low temporal resolution MODIS (Moderate-resolution Imaging Spectroradiometer) AOD image (resolution: 1Km, day), and MODIS is mainly carried on Terra and Aqua. The size of the active selection window and the preset threshold are set according to actual needs. In this embodiment, the size of the active selection window is set to 32x32, and the preset threshold is greater than 95%.

[0046] The method for acquiring the low-resolution AOD image comprises:

[0047] The original high-resolution AOD image is downsampled to obtain the corresponding low-resolution AOD image; wherein the sampling rate of the downsampling is set according to actual needs. In this embodiment, the resolution of the low-resolution AOD image obtained after downsampling is: 5Km, hour.

[0048] The method for acquiring the environmental characteristic data comprises:

[0049] 3D-CNN is used to extract features from the auxiliary data to obtain the environmental feature data; wherein the auxiliary data includes but is not limited to meteorological factors and surface conditions, such as surface reflectivity, temperature, humidity, and wind speed; the size of the 3D-CNN convolution kernel is set according to actual needs, for example, 3x3x4.

[0050] The method for obtaining the estimated high-resolution AOD image comprises:

[0051] Inputting the low-resolution AOD image into the encoder-decoder network to generate a first high-resolution AOD image, thereby completing a preliminary resolution improvement of the low-resolution AOD image;

[0052] The first high-resolution AOD image and the corresponding environmental feature data are input into the residual network, and the estimation error of the first high-resolution AOD image is corrected according to the environmental feature data to obtain the estimated high-resolution AOD image that is closer to the real details, thereby improving the detail quality of the low-resolution AOD image.

[0053] In step S300, training the generator network using the sample set includes:

[0054] S301. Input a sample set including an original high-resolution AOD image, environmental feature data, an estimated high-resolution AOD image and a low-resolution AOD image into a discriminator network. The discriminator network judges the original high-resolution AOD image and the estimated high-resolution AOD image according to the spatial sampling conditions of the low-resolution AOD image and the environmental conditions of the environmental feature data, and outputs 0 or 1; wherein 1 indicates that the estimated high-resolution AOD image is discriminated as the original high-resolution AOD image, that is, the discriminator network cannot distinguish between the real and generated images, and this is continued until all the estimated high-resolution AOD images are judged as the original high-resolution AOD images, which indicates that the training of the generator network is completed, and a trained generator network is obtained, and step S400 is executed; wherein 0 indicates that the estimated high-resolution AOD image is judged as the generated high-resolution image, that is, the discriminator can distinguish between the real and generated images, which indicates that the training has failed, and step S302 is executed.

[0055] S302, fix the parameters of the discriminator network, and adjust the parameters of the generator network obtained in S100, according to the mutual confrontation feedback of the generator network and the discriminator network, alternately update the generator network and the discriminator network, continue iterative training, and obtain the trained generator network; wherein, the generator network and the discriminator network are alternately updated with the goal of minimizing the loss function. The loss function includes a spatial condition loss function, an environmental condition loss function, and an overall content loss function, as shown in the following formula:

[0056] Loss_1(θ G ,θ D )=-log D(M HR , M′ LR )-log(1-D(G(M′ LR ), M′LR ))

[0057] Loss_2(θ G ,θ D )=-log D(M HR , E)-log(1-D(G(M′ LR ), E))

[0058]

[0059] L(θ G ,θ D )=Loss SC (θ G )+λ1Loss_1(θ G ,θ D )+λ2Loss_2(θ G ,θ D )

[0060] In the formula, M′ LR 、M HR , M′ HR , E are respectively the low-resolution AOD image, the original high-resolution AOD image, the estimated high-resolution AOD image, and the environmental feature data; θ G ,θ D are the parameters of the generator network and the discriminator network respectively; Loss_1(θ G ,θ D )、Loss_2(θ G ,θ D ) are respectively the adversarial loss based on the low-resolution AOD image (i.e., the spatial condition loss function) and the adversarial loss based on the environmental data (i.e., the environmental condition loss function); D() is the probability value output by the discriminator network, which represents the probability that the input estimated high-resolution AOD image is the original high-resolution AOD image; G() represents the generator network; Loss SC (θ G ) represents the spatial content loss (i.e., the overall content loss function); n represents the number of pixels of the original high-resolution AOD image; M HR，i , M′ HR，i denote the i-th pixel in the original high-resolution AOD image and the estimated high-resolution AOD image respectively; ρ() denotes a penalty function; L(θ G ,θ D ) represents the overall loss function of the conditional generative adversarial network; λ1 and λ2 are weight coefficients respectively.

[0061] In step S400, the low-resolution AOD image to be processed uses the Himawari AOD data with low spatial and high temporal resolution (resolution: 5Km, hour), and the Himawari AOD data with low spatial and high temporal resolution is input into the trained generator network, so that the Himawari AOD estimation data with high spatial and high temporal resolution (resolution: 1Km, hour) can be obtained. Among them, the input low-resolution AOD image to be processed needs to be divided into image blocks by actively selecting the window, the effective value ratio of each input image block needs to be greater than 80%, and the missing values ​​in the image block are filled with the neighboring mean. The original Himawari AOD data and the obtained Himawari AOD data with high spatiotemporal resolution are shown in Figures 4(a) and 4(b), respectively.

[0062] In order to further illustrate the feasibility and effectiveness of the method of the present invention, the commonly used PSNR and SSIM are usually used to evaluate the quantitative results of the image. In addition, according to the characteristics of AOD data, the present invention adds the RMSE index for measurement.

[0063] Table 1 shows the calculation results of RMSE, PSNR and SSIM of 5 methods, which are: Krige, super-resolution convolutional neural network SRCNN, the method of the present invention AeroCGAN (meteorological data) adding meteorological data, the method of the present invention AeroCGAN (surface reflectance) adding surface data, and the method of the present invention AeroCGAN (3D-CNN embedding) encoding environmental features of auxiliary data through 3D-CNN.

[0064] Table 1

[0065] RMSE PSNR SSIM Krige 0.036 28.89 0.868 SRCNN 0.071 22.81 0.796 AeroCGAN (meteorological data) 0.031 30.17 0.864 AeroCGAN (surface reflectance) 0.029 30.75 0.861 AeroCGAN (3D-CNN embedding) 0.021 33.24 0.883

[0066] Table 1 shows the performance comparison of different methods with three evaluation indicators. Overall, these indicators prove that the model proposed in the present invention can further improve the performance, especially with the addition of useful auxiliary data. The PSNR and SSIME values ​​show that these methods can all estimate the distribution trend of AOD, but the RMSE value represents the accuracy difference between different methods. Compared with other methods, the estimation performance of the model obtained by encoding the auxiliary data through 3D-CNN in the present invention is significantly improved, and the best indicator value (RMSE: 0.021, PSNR: 33.24, SSIM: 0.883) is obtained. The improvement in the RMSE value proves that directly adding auxiliary data has limited improvement on the efficiency of the model, so 3D-CNN can extract highly relevant features, thereby improving the utilization of auxiliary data.

[0067] Combine the following Figure 5 The high-resolution aerosol estimation device based on a conditional generative adversarial network provided by the present invention is described. The high-resolution aerosol estimation device based on a conditional generative adversarial network described below and the high-resolution aerosol estimation method based on a conditional generative adversarial network described above can be referenced to each other.

[0068] The device includes: a network construction module 501, which is used to construct a conditional generative adversarial network, and the conditional generative adversarial network includes a generator network and a discriminator network; the generator network improves the resolution of the input image through two stages, wherein the first stage performs preliminary resolution improvement on the input image, and the second stage improves the detail quality of the input image after the preliminary resolution improvement; as a preferred solution, the generator network includes an encoder-decoder network and a residual network connected in sequence; the encoder-decoder network is used to input the low-resolution AOD image, and generate a first high-resolution AOD image based on the low-resolution AOD image, thereby completing the preliminary resolution improvement of the low-resolution AOD image; the residual network is used to input the first high-resolution AOD image and the environmental feature data, and correct the first high-resolution AOD image according to the environmental feature data to obtain the estimated high-resolution AOD image, thereby completing the improvement of the detail quality of the low-resolution AOD image.

[0069] The sample construction module 502 is used to construct a sample set, which includes an original high-resolution AOD image, environmental feature data, an estimated high-resolution AOD image and a low-resolution AOD image; wherein the low-resolution AOD image is obtained by downsampling the original high-resolution AOD image; the estimated high-resolution AOD image is obtained by inputting the low-resolution AOD image and the environmental feature data into the generator network; wherein the original high-resolution AOD image in the sample set adopts MODIS AOD data with high spatial and low temporal resolution.

[0070] The high-resolution aerosol estimation module 503 is used to train the generator network through the sample set to obtain the trained generator network, and the trained generator network is used to estimate the high-resolution AOD image for the low-resolution AOD image to be processed; the high-resolution aerosol estimation module 503 includes a first training module 504, a second training module 505, and a calculation module 506.

[0071] The first training module 504 is used to input the sample set into the discriminator network, and supervise the generator network according to the output of the discriminator network; if the preset conditions are met, the trained generator network is obtained, otherwise, the training fails. As a preferred solution, the training process of the first training module 504 includes: inputting the sample set into the discriminator network, judging the original high-resolution AOD image and the estimated high-resolution AOD image through the discriminator network according to the downsampling conditions of the low-resolution AOD image and the environmental feature data, and supervise the generator network according to the judgment result of the discriminator network; if the discriminator network cannot distinguish between the original high-resolution AOD image and the estimated high-resolution AOD image, the trained generator network is obtained, otherwise, the training fails.

[0072] The second training module 505 is used to adjust the parameters of the generator network when the training of the first training module fails, and perform secondary training on the generator network based on the loss function to obtain the trained generator network. As a preferred solution, the training process of the second training module 505 includes: fixing the parameters of the discriminator network, adjusting the parameters of the generator network, and alternately updating the generator network and the discriminator network according to the mutual adversarial feedback between the generator network and the discriminator network to obtain the trained generator network; wherein, the generator network and the discriminator network are alternately updated with the goal of minimizing the loss function. The loss function is shown as follows:

[0073] Loss_1(θ G ,θ D )=-log D(M HR , M′ LR )-log(1-D(G(M′ LR ), M′ LR ))

[0074] Loss_2(θ G ,θ D )=-log D(M HR , E)-log(1-D(G(M′ LR ), E))

[0075]

[0076] L(θ G ,θ D )=Loss SC (θ G )+λ1Loss_1(θ G ,θ D)+λ2Loss_2(θ G ,θ D )

[0077] In the formula, M′ LR 、M HR , M′ HR , E are respectively the low-resolution AOD image, the original high-resolution AOD image, the estimated high-resolution AOD image, and the environmental feature data; θ G ,θ D are the parameters of the generator network and the discriminator network respectively; Loss_1(θ G ,θ D )、Loss_2(θ G ,θ D ) are respectively the adversarial loss based on the low-resolution AOD image and the adversarial loss based on the environmental data; D() is the probability value output by the discriminator network; G() represents the generator network; Loss SC (θ G ) represents the spatial content loss; n represents the number of pixels of the original high-resolution AOD image; M HR，i , M′ HR，i denote the i-th pixel in the original high-resolution AOD image and the estimated high-resolution AOD image respectively; ρ() denotes a penalty function; L(θ G ,θ D ) represents the overall loss function of the conditional generative adversarial network; λ1 and λ2 are weight coefficients respectively.

[0078] The calculation module 506 is used to estimate the high-resolution AOD image of the low-resolution AOD image to be processed according to the trained generator network. As a preferred solution, the calculation module 506 inputs the low-resolution AOD image to be processed and the corresponding environmental feature data into the trained generator network to obtain a high-resolution AOD image; wherein the low-resolution AOD image to be processed adopts Himawari AOD data with low spatial and high temporal resolution, and the obtained high-resolution AOD image is AOD estimation data with high spatial and high temporal resolution.

[0079] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute a high-resolution aerosol estimation method based on a conditional generative adversarial network, the method comprising:

[0080] Constructing a conditional generative adversarial network, the conditional generative adversarial network includes a generator network and a discriminator network; the generator network improves the resolution of an input image through two stages, wherein the first stage performs a preliminary resolution improvement on the input image, and the second stage performs a detail quality improvement on the input image after the preliminary resolution improvement;

[0081] Constructing a sample set, the sample set comprising an original high-resolution AOD image, environmental feature data, an estimated high-resolution AOD image and a low-resolution AOD image; wherein the low-resolution AOD image is obtained by downsampling the original high-resolution AOD image; and the estimated high-resolution AOD image is obtained by inputting the low-resolution AOD image and the environmental feature data into the generator network;

[0082] The generator network is trained by the sample set to obtain the trained generator network, and the trained generator network is used to estimate the high-resolution AOD image for the low-resolution AOD image to be processed; wherein the low-resolution AOD image is used as a spatial condition, and the environmental feature data is used as an environmental condition to train the generator network; during the training process, the sample set is input into the discriminator network, and the generator network is supervised and trained according to the output of the discriminator network; if the preset conditions are met, the trained generator network is obtained, otherwise, the parameters of the generator network are adjusted, and the generator network is trained twice based on the loss function.

[0083] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0084] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the high-resolution aerosol estimation method based on a conditional generative adversarial network provided by the above methods, the method comprising:

[0085] Constructing a conditional generative adversarial network, the conditional generative adversarial network includes a generator network and a discriminator network; the generator network improves the resolution of an input image through two stages, wherein the first stage performs a preliminary resolution improvement on the input image, and the second stage performs a detail quality improvement on the input image after the preliminary resolution improvement;

[0086] Constructing a sample set, the sample set comprising an original high-resolution AOD image, environmental feature data, an estimated high-resolution AOD image and a low-resolution AOD image; wherein the low-resolution AOD image is obtained by downsampling the original high-resolution AOD image; and the estimated high-resolution AOD image is obtained by inputting the low-resolution AOD image and the environmental feature data into the generator network;

[0087] The generator network is trained by the sample set to obtain the trained generator network, and the trained generator network is used to estimate the high-resolution AOD image for the low-resolution AOD image to be processed; wherein the low-resolution AOD image is used as a spatial condition, and the environmental feature data is used as an environmental condition to train the generator network; during the training process, the sample set is input into the discriminator network, and the generator network is supervised and trained according to the output of the discriminator network; if the preset conditions are met, the trained generator network is obtained, otherwise, the parameters of the generator network are adjusted, and the generator network is trained again based on the loss function.

[0088] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the high-resolution aerosol estimation method based on a conditional generative adversarial network provided by the above methods, the method comprising:

[0089] Constructing a conditional generative adversarial network, the conditional generative adversarial network includes a generator network and a discriminator network; the generator network improves the resolution of an input image through two stages, wherein the first stage performs a preliminary resolution improvement on the input image, and the second stage performs a detail quality improvement on the input image after the preliminary resolution improvement;

[0090] Constructing a sample set, the sample set comprising an original high-resolution AOD image, environmental feature data, an estimated high-resolution AOD image and a low-resolution AOD image; wherein the low-resolution AOD image is obtained by downsampling the original high-resolution AOD image; and the estimated high-resolution AOD image is obtained by inputting the low-resolution AOD image and the environmental feature data into the generator network;

[0091] The generator network is trained by the sample set to obtain the trained generator network, and the trained generator network is used to estimate the high-resolution AOD image for the low-resolution AOD image to be processed; wherein the low-resolution AOD image is used as a spatial condition, and the environmental feature data is used as an environmental condition to train the generator network; during the training process, the sample set is input into the discriminator network, and the generator network is supervised and trained according to the output of the discriminator network; if the preset conditions are met, the trained generator network is obtained, otherwise, the parameters of the generator network are adjusted, and the generator network is trained again based on the loss function.

[0092] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0093] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A high-resolution aerosol estimation method based on conditional generative adversarial networks, characterized in that: include: Constructing a conditional generative adversarial network, the conditional generative adversarial network includes a generator network and a discriminator network; the generator network improves the resolution of an input image through two stages, wherein the first stage performs a preliminary resolution improvement on the input image, and the second stage performs a detail quality improvement on the input image after the preliminary resolution improvement; Constructing a sample set, the sample set comprising an original high-resolution AOD image, environmental feature data, an estimated high-resolution AOD image and a low-resolution AOD image; wherein the low-resolution AOD image is obtained by downsampling the original high-resolution AOD image; and the estimated high-resolution AOD image is obtained by inputting the low-resolution AOD image and the environmental feature data into the generator network; The generator network is trained by the sample set to obtain the trained generator network, and the trained generator network is used to estimate the high-resolution AOD image for the low-resolution AOD image to be processed; wherein the low-resolution AOD image is used as a spatial condition, and the environmental feature data is used as an environmental condition to train the generator network; during the training process, the sample set is input into the discriminator network, and the generator network is supervised and trained according to the output of the discriminator network; if the preset conditions are met, the trained generator network is obtained, otherwise, the parameters of the generator network are adjusted, and the generator network is trained twice based on the loss function; After obtaining the trained generator network, the method further includes: obtaining a low-resolution AOD image to be processed, inputting the low-resolution AOD image to be processed and the corresponding environmental feature data into the trained generator network to obtain a high-resolution AOD image; wherein the original high-resolution AOD image in the sample set adopts MODIS AOD data with high spatial and low temporal resolution; the low-resolution AOD image to be processed adopts Himawari AOD data with low spatial and high temporal resolution, and the obtained high-resolution AOD image is AOD estimation data with high spatial and high temporal resolution; The generator network includes an encoder-decoder network and a residual network connected in sequence; The encoder-decoder network is used to input the low-resolution AOD image, and generate a first high-resolution AOD image according to the low-resolution AOD image, thereby completing the initial resolution improvement of the low-resolution AOD image; The residual network is used to input the first high-resolution AOD image and the environmental feature data, correct the first high-resolution AOD image according to the environmental feature data, obtain the estimated high-resolution AOD image, and improve the detail quality of the low-resolution AOD image.

2. The high-resolution aerosol estimation method based on conditional generative adversarial network according to claim 1, characterized in that: The supervised training of the generator network according to the output of the discriminator network comprises: The sample set is input into the discriminator network, and the original high-resolution AOD image and the estimated high-resolution AOD image are judged by the discriminator network according to the downsampling conditions of the low-resolution AOD image and the environmental feature data, and the generator network is supervised and trained according to the judgment result of the discriminator network; if the discriminator network cannot distinguish between the original high-resolution AOD image and the estimated high-resolution AOD image, the trained generator network is obtained, otherwise, the training fails.

3. The high-resolution aerosol estimation method based on conditional generative adversarial network according to claim 1, characterized in that: The second training of the generator network based on the loss function includes: The parameters of the discriminator network are fixed, and the parameters of the generator network are adjusted. According to the mutual adversarial feedback between the generator network and the discriminator network, the generator network and the discriminator network are updated alternately to obtain the trained generator network; wherein, the generator network and the discriminator network are updated alternately with the goal of minimizing the loss function.

4. The high-resolution aerosol estimation method based on conditional generative adversarial network according to any one of claims 1 to 3, characterized in that: The loss function is shown as follows: Loss_1(θ G ,θ D )0-log D(M HR ,M′ LR )-log(1-D(G(M′ LR ),M′ LR ))Loss_2(θ G ,θ D )0-log D(M HR ,E)-log(1-D(G(M′ LR ),E)) L(θ G ,the D )=Loss SC (i G )+λ1Loss_1(θ G ,the D )+λ2Loss_2(θ G ,the D ) In the formula, M′ LR 、M HR , M′ HR , E are respectively the low-resolution AOD image, the original high-resolution AOD image, the estimated high-resolution AOD image, and the environmental feature data; θ G ,θ D are the parameters of the generator network and the discriminator network respectively; Loss_1(θ G ,θ D )、Loss_2(θ G ,θ D ) are respectively the adversarial loss based on the low-resolution AOD image and the adversarial loss based on the environmental feature data; D() is the probability value output by the discriminator network; G() represents the generator network; Loss SC (θ G ) represents the spatial content loss; n represents the number of pixels of the original high-resolution AOD image; M HR，i , M′ HR，i denote the i-th pixel in the original high-resolution AOD image and the estimated high-resolution AOD image respectively; ρ() denotes a penalty function; L(θ G ,θ D ) represents the overall loss function of the conditional generative adversarial network; λ1 and λ2 are weight coefficients respectively.

Images