Super-resolution night-light image reconstruction method based on improved SRGAN

Abstract

The present invention discloses a super-resolution night light image reconstruction method based on improved SRGAN, comprising the following steps: (1) screening out night light images with an annual cloud cover of less than 10% and downloading them; (2) performing preprocessing operations on the night light images; (3) constructing an improved SRGAN network model; (4) constructing an overall loss function and training the model; (5) inputting the preprocessed night light image into the model, and outputting a corresponding high-resolution night light reconstruction image. The present invention, on the basis of making full use of night light remote sensing image data, adopts an improved residual network based on super-resolution, further improving the resolution of satellite night light image data, which is of great significance for the subsequent application of night light image analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of night light remote sensing image data processing, and in particular to a super-resolution night light image reconstruction method based on improved SRGAN. Background Art

[0002] Most of the remote sensing data used by people are taken during the day, with sufficient light, and can clearly reflect the surface conditions, such as the distribution of urban buildings, vegetation coverage, river direction, etc. At the same time, there are also some remote sensing data that are taken at night. With economic development and technological progress, the darkness of the night can no longer restrain humans. From candles to fluorescent lamps to neon lights, human society at night also exudes dazzling light. When there are no clouds at night, remote sensing technology is used to observe the surface and capture various visible light radiation sources such as lights and fire points to form night light remote sensing data. By studying night light data, we can understand human activities, estimate socioeconomic parameters, and monitor oil and gas field combustion. In general, the research direction of night light data is also very broad.

[0003] There are two types of night light remote sensing data that are widely used in current research: DMSP / OLS (Defense Meteorological Satellite Program's Operational Linescan System) and NPP / VIIRS (Suomi National Polar-Orbiting Partnership / the Visible Infrared Imaging Radiometer Suite). DMSP / OLS is the first generation of night light data. In the last century, NASA and the National Geophysical Data Center of the National Oceanic and Atmospheric Administration jointly launched the Defense Meteorological Satellite Program. The DMSP / OLS data comes from the linear scanning system sensor carried by this program. The resolution is about 1km, which can clearly reflect lights, fires, etc. NPP / VIIRS is the second generation of night light data, which is detected by the visible infrared imaging radiometer carried by the National Polar-Orbiting Partnership satellite of the United States. The resolution is about 500m, which is more accurate than DMSP / OLS. Both data have their own advantages and disadvantages. DMSP / OLS has a longer time series (1992-2013), which is obtained by 6 different satellites, namely F10, F12, F14, F15, F16, and F18. Because different satellites are used, there are compatibility and stability issues, and preprocessing methods are required before use. NPP / VIIRS can reflect more details, but the time span is not as good as DMSP / OLS. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a super-resolution night light image reconstruction method based on improved SRGAN, which further improves the resolution of satellite night light image data and is of great significance for the subsequent application of night light image analysis.

[0005] In order to solve the above technical problems, the present invention provides a super-resolution night light image reconstruction method based on improved SRGAN, comprising the following steps:

[0006] (1) Filter out night light images with annual cloud cover below 10% and download them;

[0007] (2) Preprocessing the night light image;

[0008] (3) Construct an improved SRGAN network model;

[0009] (4) Construct the overall loss function and train the model;

[0010] (5) The preprocessed night light image is input into the improved SRGAN network model, and the corresponding high-resolution night light reconstructed image is output.

[0011] Preferably, in step (2), the preprocessing operations include radiation calibration, atmospheric correction, geometric correction, denoising and temporary light removal.

[0012] Preferably, in step (2), the preprocessing operation of the night light image is specifically: cropping the night light data, loading all raster data, setting the image grayscale value DN value less than 0 to 0, converting the coordinate system to WGS1984, importing stable images of some areas around the control point for georeferencing, and entering the formula in the raster calculator for radiance conversion:

[0013]

[0014] Where L is the radiation brightness value after absolute radiation correction, the unit is W / (m 2 ·sr·μm), DN is the image gray value.

[0015] Preferably, in step (3), the improved SRGAN network model includes a generator and a discriminator. The generator learns the distribution of real samples to generate highly similar data to pass to the discriminator. The discriminator performs binary classification on the generated fake samples to determine whether they are real data to guide the training of the generator.

[0016] Preferably, in step (3), the generator network structure is: a 9×9 convolution layer to extract features from the input image, 16 layers of improved residual convolution, each residual block includes two 1×1 convolution layers and one 3×3 convolution layer, 2 times 2x deconvolution scaling, tanh scaling, and finally generating the result, removing the batch normalization layer that increases the network parameters, adding a GCNet module after each residual block and the shallow and deep layers of the discriminator, and selecting the Leaky ReLU as the activation function.

[0017] Preferably, in step (3), the discriminator network structure is: both the shallow layer and the deep layer adopt the form of convolution layer + Leaky ReLU, and a GCNet module is added at the end, and then a binary classification output is obtained through a fully connected layer + Sigmoid for the generator to select data.

[0018] Preferably, in step (4), the loss function is composed of the generative adversarial loss and the mean square error loss MSE, and the generative adversarial loss is

[0019]

[0020] in, represents the probability that the generated image is a true high-resolution image. It represents the high-resolution image generated by reconstruction;

[0021]

[0022] Where W and H represent the width and height of the image respectively. represents high-resolution image, I LR Represents a low-resolution image.

[0023] The overall loss function is

[0024] L ALL =L GAN +L MSE .

[0025] The beneficial effects of the present invention are as follows: the present invention replaces the residual module of the SRGAN model with an improved residual module, thereby improving the performance of the network, realizing the combined application of urban lighting development and deep learning, and the night light images generated by the improved SRGAN model have higher resolution; the batch normalization layer in the generator network is removed, thereby reducing the computational complexity, and reducing the computational complexity can greatly shorten the training and testing time of the network; improving the resolution of night light images can expand the scope of application fields, and can be applied in some fields that require higher resolution, such as the recognition and extraction of urban road lights. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the method flow of the present invention.

[0027] Figure 2 It is a schematic diagram of the generator structure of the present invention.

[0028] Figure 3 Schematic diagram of the discriminator structure of the present invention.

[0029] Figure 4 It is a schematic diagram of the structural comparison of the original residual network, deep convolutional residual network and improved residual network of the present invention. DETAILED DESCRIPTION

[0030] like Figure 1-4 As shown, a super-resolution night light image reconstruction method based on improved SRGAN includes the following steps:

[0031] (1) Filter and download the NPP images of Shenzhen in February 2020 from NASA's official website;

[0032] (2) Preprocess the downloaded original NPP images of Shenzhen City with an annual cloud cover below 10%, including radiometric calibration, atmospheric correction, geometric correction, denoising, and temporary light removal;

[0033] (3) Constructing an improved SRGAN network model: This mainly includes replacing the original residual module with an improved residual module, removing unnecessary BN layers to improve network training efficiency, adding GCNet modules after each residual module of the generator and the deep and shallow layers of the discriminator, and training the model using the DIV2K dataset;

[0034] (4) Using the Shenzhen NPP image preprocessed in step (2) as the input of the improved SRGAN model obtained in step (3);

[0035] (5) The output result of the improved SRGAN model in (4) is the NPP data of Shenzhen City with improved resolution.

[0036] The specific steps for preprocessing the original NPP image of Shenzhen City in step (2) are as follows:

[0037] (2.1) Perform radiometric calibration on low-cloud NPP images;

[0038] (2.2) Perform atmospheric correction on the images obtained by NPP;

[0039] (2.3) Assign 0 to the values ​​of the image DN value obtained by BPP that is less than 0, and assign 0 to the points with abnormal DN value in the temporary lighting;

[0040] (2.4) The image obtained in step (2.3) is cropped to obtain the NPP preprocessed image of the training sample city, i.e., Shenzhen.

[0041] The specific steps for training the improved SRGAN model in step (3) are as follows:

[0042] (3.1) Experimental data preparation: The experimental data used is the public dataset DIV2K, which includes 800 training images, 100 validation images, and 100 test images. The resolution of each image is 2k.

[0043] (3.2) Model training: The loss function consists of the generative adversarial loss and the MSE loss:

[0044] L ALL =L GAN +L MSE

[0045] Generate adversarial loss:

[0046]

[0047] in, represents the probability that the generated image is a true high-resolution image. It represents the high-resolution image generated by reconstruction;

[0048] MSE loss:

[0049]

[0050] Where W and H represent the width and height of the image respectively. represents high-resolution image, I LR Represents a low-resolution image.

[0051] The residual module contains 16 residual blocks, 64 channels, and a GCNet module is set at the back. The training set images are cropped to 88*88, the batch sampling size is 64, the number of iterations is set to 100, the Adam optimizer is used, and the learning rate is set to 0.0001, which is called the improved SRGAN model.

Claims

1. A super-resolution night light image reconstruction method based on improved SRGAN, characterized in that: The steps include: (1) Filter out night light images with annual cloud cover below 10% and download them; (2) Preprocessing the night light image; (3) Construct an improved SRGAN network model; the improved SRGAN network model includes a generator and a discriminator. The generator learns the distribution of real samples to generate highly similar data and pass it to the discriminator. The discriminator performs binary classification on the generated fake samples to determine whether they are real data to guide the training of the generator. The generator network structure is: a 9×9 convolution layer extracts features from the input image, 16 layers of improved residual convolution, each residual block includes two 1×1 convolution layers and one 3×3 convolution layer, through 2 times 2-fold deconvolution scaling and tanh scaling, finally generate the result, remove the batch normalization layer that increases the number of network parameters, add GCNet modules after each residual block and the shallow and deep layers of the discriminator, and select LeakyReLU as the activation function; the discriminator network structure is: the shallow and deep layers are both in the form of convolution layer + Leaky ReLU, and the GCNet module is added at the end, and then the binary classification output is obtained through the fully connected layer + Sigmoid for the generator to select data; (4) Construct the overall loss function and train the model; the loss function consists of the generative adversarial loss and the mean square error loss MSE. The generative adversarial loss is: in, represents the probability that the generated image is a true high-resolution image. It represents the high-resolution image generated by reconstruction; Where W and H represent the width and height of the image respectively. represents high-resolution image, I LR Represents a low-resolution image; The overall loss function is L ALL =L GAN +L MSE ; (5) The preprocessed night light image is input into the improved SRGAN network model, and the corresponding high-resolution night light reconstructed image is output.

2. The super-resolution night light image reconstruction method based on improved SRGAN as claimed in claim 1, characterized in that: In step (2), the preprocessing operations include radiometric calibration, atmospheric correction, geometric correction, denoising and temporary light removal.

3. The super-resolution night light image reconstruction method based on improved SRGAN as claimed in claim 1, characterized in that: In step (2), the preprocessing operation of the night light image is as follows: crop the night light data, load all raster data, set the image grayscale value DN value less than 0 to 0, convert the coordinate system to WGS1984, import some stable images of the area around the control point for georeferencing, and enter the formula in the raster calculator for radiation brightness conversion: Where L is the radiation brightness value after absolute radiation correction, the unit is W / (m 2 ·sr·μm), DN is the image gray value.

Images