Remote sensing image surface temperature downscaling method and system based on deep learning technology

Abstract

The invention discloses a remote sensing image land surface temperature downscaling method and system based on a deep learning technology, and the method comprises the steps: calculating each remote sensing index and land surface temperature data based on a Landsat-8 image, carrying out the correlation analysis, selecting three remote sensing indexes which have the highest correlation with the land surface temperature and are not collinear, and carrying out the downscaling of the land surface temperature. Constructing a first deep learning model; calculating various remote sensing indexes based on the Sentinel-2 image, selecting three remote sensing indexes which have the highest correlation with the surface temperature and are not collinear, and constructing a second deep learning model; simulating the characteristics of a Landsat-8 remote sensing index by using the Sentinel-2 remote sensing index, fusing multi-source remote sensing data, and inputting the fused data into a first deep learning model to obtain a downscaled surface temperature image; the method is advanced in algorithm, high in precision, small in model data size, capable of effectively reducing the calculation time, and meanwhile suitable for downscaling of the surface temperature LST data of different input ranges.

Description

Technical Field

[0001] The present invention relates to a method and system for downscaling land surface temperature, and particularly to a method and system for downscaling land surface temperature of remote sensing images based on deep learning technology. Background Art

[0002] In recent years, global cities have faced severe challenges of rising land surface temperature and deteriorating ecological environment. The increase in human activities and the industrialization process have led to the expansion of urban built-up areas, the reduction of vegetation cover, and the increase in pollution emissions, which in turn have caused problems such as the urban heat island effect and air pollution. Studying the changes in land surface temperature and its relationship with socio-economic and ecological environment factors is of great significance for alleviating urban environmental problems.

[0003] Using remote sensing technology to obtain land surface temperature data has become an important means to study this problem. The Landsat-8 remote sensing satellite can accurately measure the land surface temperature due to its high-precision thermal infrared sensor, and has a high spatial resolution and temporal resolution, providing a powerful data source for monitoring the changes in urban land surface temperature. However, the spatial resolution of Landsat-8 still cannot fully depict the subtle temperature changes within the urban area. The Sentinel-2 remote sensing satellite has the ability to capture the details of urban land surface temperature with its higher spatial resolution and more frequent observation periods, but lacks a thermal infrared sensor and cannot directly obtain land surface temperature data.

[0004] In view of the limitations of existing satellite data, downscaling technology has become an effective means. However, traditional downscaling algorithms often rely on complex algorithm structures and a large amount of experimental data, with limited computational efficiency, and there is a problem that it is difficult to balance the acquisition of higher accuracy and computational efficiency. Summary of the Invention

[0005] Object of the Invention: The object of the present invention is to provide a method for downscaling land surface temperature of remote sensing images based on deep learning technology to improve the accuracy of land surface temperature downscaling and generate high-resolution land surface temperature images. On the other hand, a system for downscaling land surface temperature of remote sensing images based on deep learning technology is provided.

[0006] Technical Solution: A method for downscaling land surface temperature of remote sensing images based on deep learning technology according to the present invention includes the following steps:

[0007] (1) Obtain Landsat-8 and Sentinel-2 remote sensing image data, and preprocess the data;

[0008] (2) Calculate remote sensing indices and land surface temperature LST based on Landsat-8 remote sensing image data. The remote sensing indices include Enhanced Vegetation Index (EVI), Normalized Difference Vegetation Index (NDVI), Fractional Vegetation Cover (FVC), Normalized Difference Water Index (NDWI), and Normalized Difference Built-up Index (NDBI).

[0009] (3) Calculate the Pearson correlation coefficients of the remote sensing indices and land surface temperature LST described in step (2), and select three remote sensing indices with the highest correlation and no collinearity.

[0010] (4) Construct a first deep learning model with the three remote sensing indices described in step (3) and the land surface temperature LST described in step (2) through a simplified SENET model.

[0011] (5) Calculate remote sensing indices based on Sentinel-2 remote sensing image data, calculate the correlation coefficients with the land surface temperature LST described in step (2), and select three remote sensing indices with the highest correlation and no collinearity.

[0012] (6) Construct a second deep learning model with the three remote sensing indices described in step (3) and the three remote sensing indices described in step (5). The second deep model selects a RESNET model with a residual connection block and a simplified SE block and a SENET model.

[0013] (7) Construct a comprehensive calculation model based on the first deep learning model and the second deep learning model to generate a land surface temperature LST image of the downscaled Sentinel-2 remote sensing image.

[0014] Preferably, the preprocessing described in step (1) includes radiometric calibration, atmospheric correction, and removal of data in cloud-covered areas for Landsat-8 and Sentinel-2 remote sensing image data.

[0015] Preferably, the calculation formulas for the remote sensing indices and land surface temperature LST described in step (2) are as follows:

[0016]

[0017] Among them, NIR represents the reflectance of the near-infrared band, RED represents the reflectance of the red band, BLUE represents the reflectance of the blue band, NDVI represents the NDVI value of the pixel, NDVI soil represents the maximum NDVI value of pure bare soil, NDVI vegetation represents the minimum NDVI value of pure vegetation in the image, GREEN represents the reflectance of the green band, MIR represents the reflectance of the mid-infrared band, T LST represents the land surface temperature, BT represents the brightness temperature, λ represents the wavelength, C2 represents the second radiation constant, and ε represents the land surface emissivity.

[0018] Preferably, the specific steps of constructing the first deep learning model in step 4 are as follows:

[0019] (41) Randomly select the values of grid points without missing values in the data area as the data set, extract the three remote sensing indices of each corresponding point as the model input, synthesize the training matrix, and use the land surface temperature LST of each corresponding point as the model output;

[0020] (42) Set the training set, validation set and test set, randomly divide the corresponding ratio as 8:1:1, and select the streamlined SENET model with SEblock for the model.

[0021] Preferably, the specific method for synthesizing the training matrix in step 41 is as follows:

[0022]

[0023] Among them, input (a,b,…,n) represents the deep learning remote sensing index matrix, and the component index is the three remote sensing indices from period a to period n.

[0024] Preferably, the working process expression of the streamlined SENET model in step 42 is as follows:

[0025]

[0026] S = F x (Z, W) = σ(g(Z, W));

[0027]

[0028] Among them, Z represents the output after the Squeeze operation, W and H are the sizes of the input matrix, and x(i, j) represents the element in the i-th row and j-th column of the matrix; S represents the output after the Excitation operation, σ represents the sigmoid function, and g represents the linear layer; represents the output after the tensor U operation, U represents the two-dimensional matrix, and S represents the learned weight; L2(MSE) represents the loss function, n represents the sample size of the training data set, y sr represents the predicted value obtained through model training, y hr represents the true value obtained through model training; P i represents the model-simulated land surface temperature data, represents the average value of the model-simulated land surface temperature data, M i represents the actual land surface temperature data, represents the average value of the actual land surface temperature data.

[0029] Preferably, the specific steps of constructing the second deep learning model in step 6 are as follows:

[0030] (61) Randomly select the grid points of Landsat-8 remote sensing images without missing values in the data area, obtain the geographical location information of the points, and extract the grid points of Sentinel-2 remote sensing images according to the corresponding positions.

[0031] (62) Take the central grid of the extracted Sentinel-2 remote sensing image and its surrounding grid data as the input according to the spatial position correspondence relationship, and the grid points at the corresponding positions of the Landsat-8 remote sensing image as the output.

[0032] (63) Set the training set, validation set, and test set, and randomly divide the corresponding ratio as 8:1:1.

[0033] (64) Use the three remote sensing indices described in step 3 and the three remote sensing indices described in step 5 to construct a deep learning model of the sensor relationship. The model selects the RESNET model and SENET model with residual connection blocks and simplified SE blocks.

[0034] A remote sensing image land surface temperature downscaling system based on deep learning technology, including:

[0035] A data acquisition module for obtaining Landsat-8 and Sentinel-2 remote sensing image data;

[0036] A data preprocessing module for performing radiometric calibration, atmospheric correction, and data processing for removing cloud cover on the acquired data.

[0037] An index calculation module for calculating remote sensing indices and land surface temperature LST based on Landsat-8 and Sentinel-2 image data respectively.

[0038] A model construction module for constructing a first deep learning model, a second deep learning model, and a comprehensive calculation model;

[0039] An image generation module for generating a Sentinel-2 remote sensing image land surface temperature LST image with completed downscaling by the comprehensive calculation model.

[0040] A computer device includes one or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and are configured to be executed by the one or more processors. When the program is executed by the processor, it implements the steps of a remote sensing image land surface temperature downscaling method according to any one of claims 1-7.

[0041] A computer-readable storage medium stores a computer program thereon. When the computer program is executed by a processor, it implements the steps of a method for downscaling land surface temperature of remote sensing images based on deep learning technology as described in any one of claims 1-7.

[0042] Beneficial effects: Compared with the prior art, the present invention has the following remarkable advantages: 1. By comprehensively utilizing Landsat-8 and Sentinel-2 remote sensing image data, the accuracy of land surface temperature downscaling is effectively improved, and high-resolution land surface temperature images that are more in line with the actual situation can be generated; 2. The streamlined SENET and RESNET architectures are adopted to improve the data processing efficiency, and it has strong practicability and operability; 3. By constructing a deep learning model to learn the feature relationship between the two, it helps to more deeply explore the variation law of land surface temperature at different spatial scales and its relationship with environmental factors, and has important application value in the fields of ecological environment monitoring, social and economic development, and energy resource management. Brief description of the drawings

[0043] Figure 1 It is a schematic flow diagram of the present invention;

[0044] Figure 2 It is a schematic diagram of the structure of the SENET module of the present invention;

[0045] Figure 3 It is a schematic diagram of the structure of the RSENET module of the present invention. Detailed implementation manners

[0046] Next, in conjunction with the drawings, the technical solutions of the present invention will be described in detail.

[0047] (1) Data preparation: Based on the Google Earth Engine data cloud platform, Landsat-8 remote sensing image data of the same time or similar time of the Landsat-8 OLI / TIRS sensor in the target area and multi-spectral image data of the Sentinel-2 sensor in the target area are extracted; the Landsat-8 image data includes 5 visible light bands, 1 near-infrared band, 2 short-wave infrared bands of the OLI sensor, and 2 thermal infrared bands of the TIRS sensor; the Sentinel-2 image includes visible light, red edge, near-infrared, and short-wave infrared bands.

[0048] Among them, the preset resolution of the Landsat-8 image is 30m resolution, and the preset resolution of the Sentinel-2 image is 10m resolution.

[0049] (2) Preprocess the acquired Landsat-8 and Sentinel-2 remote sensing image data: complete cloud and occlusion removal, radiometric calibration, and atmospheric correction processing on the platform.

[0050] (3) Index and land surface temperature calculation:

[0051] 1. Calculate various remote sensing indices based on the coastal blue B1, blue B2, green B3, red B4, near-infrared B5, shortwave infrared 1 B6, and shortwave infrared 2 B7 bands of the Landsat-8-OLI sensor;

[0052] Calculate the enhanced vegetation index EVI:

[0053]

[0054] Calculate the normalized difference vegetation index NDVI:

[0055]

[0056] Calculate the fractional vegetation cover FVC:

[0057]

[0058] Calculate the normalized difference water index NDWI:

[0059]

[0060] Calculate the normalized difference built-up index NDBI:

[0061]

[0062] Retrieve and calculate the land surface temperature LST based on the thermal infrared 1 B10 and thermal infrared 2 B11 bands of the Landsat-8-TIRS sensor:

[0063]

[0064] Among them, NIR represents the reflectance of the near-infrared band, RED represents the reflectance of the red band, BLUE represents the reflectance of the blue band, NDVI represents the NDVI value of the pixel, NDVI soil represents the maximum NDVI value of pure bare soil, NDVI vegetation represents the minimum NDVI value of pure vegetation in the image, GREEN represents the reflectance of the green band, MIR represents the reflectance of the mid-infrared band, T LST represents the land surface temperature, BT represents the brightness temperature, λ represents the wavelength, C2 represents the second radiation constant, and ε represents the land surface emissivity.

[0065] (4) Correlation analysis and model construction:

[0066] 1. Calculate the Pearson correlation coefficients of the Enhanced Vegetation Index (EVI), Normalized Difference Vegetation Index (NDVI), Fractional Vegetation Cover (FVC), Normalized Difference Water Index (NDWI), Normalized Difference Built-up Index (NDBI), and Land Surface Temperature (LST) obtained from Landsat-8 remote sensing image data, find the three indices with the highest correlation with the Land Surface Temperature (LST), and exclude the collinearity of the three indices. The Pearson correlation coefficient formula is as follows:

[0067]

[0068] where x i and y i represent the observed values of variable x and variable y, and represent the means of variable x and variable y.

[0069] 2. Construct the first deep learning model:

[0070] Use the three remote sensing indices selected in step 3, denoted as Index 1, Index 2, and Index 3, and the Land Surface Temperature (LST) calculated in step 2 to construct the first deep learning model. Randomly select the values of 5000 grid points without missing values in the data area as the dataset. Extract Index 1, Index 2, and Index 3 of each corresponding point as the model's corresponding inputs, and the Land Surface Temperature (LST) of each corresponding point as the model's output. Set the training, validation, and test datasets, and randomly divide the dataset ratio as training:validation:test = 8:1:1; the model is a simplified SENET model using the SE block;

[0071] Concatenate Index 1, Index 2, and Index 3 of the grid at the same coordinate position as input features to synthesize a training matrix, which is used as the input data of the neural network. The synthesis process is as follows:

[0072]

[0073] where input (a,b,…,n) represents the deep learning remote sensing index matrix, and its components are the indices from period a to period n;

[0074] The working process expression of the simplified SENET model is as follows:

[0075]

[0076] S = F x (Z, W) = σ(g(Z, W));

[0077]

[0078] Set the loss function for model training, and select the L2 (MSE) loss as the loss function of the model to train the model to narrow the gap between the predicted value and the true value:

[0079]

[0080] Calculate R based on the deep learning land surface temperature LST calculated by the model and the land surface temperature LST retrieved by the Landsat-8 satellite 2 and RMSE to verify the accuracy of the model. The calculation formulas are as follows:

[0081]

[0082] where Z represents the output after the Squeeze operation, W and H are the sizes of the input matrix, x(i,j) represents the element in the i-th row and j-th column of the matrix; S represents the output after the Excitation operation, σ represents the sigmoid function, and g represents the linear layer; represents the output after the tensor U operation, U represents a two-dimensional matrix, and S represents the learned weight; L2 (MSE) represents the loss function, n represents the sample size of the training data set, and y sr represents the predicted value obtained after model training, and y hr represents the true value obtained after model training; P i represents the model-simulated land surface temperature data, represents the average value of the model-simulated land surface temperature data, M i represents the actual land surface temperature data, represents the average value of the actual land surface temperature data.

[0083] (5) Export the trained first deep learning model, calculate the Pearson correlation coefficients between the enhanced vegetation index EVI, normalized difference vegetation index NDVI, vegetation coverage FVC, normalized difference water index NDWI, normalized difference built-up index NDBI and land surface temperature LST obtained from Sentinel-2 remote sensing image data, find the three indices with the highest correlation with the land surface temperature LST among them, and exclude the collinearity of the three indices, denoted as Index 1, Index 2 and Index 3.

[0084] (6) Use the calculated Landsat-8 remote sensing image indices 1, 2, and 3 and the calculated Sentinel-2 remote sensing image indices 1, 2, and 3 to construct a deep learning model for sensor relationships. Randomly select 5000 grid points of Landsat-8 remote sensing images without missing values in the data area, obtain the geographical location information of the points, extract the grid points of Sentinel-2 remote sensing images according to the corresponding positions, set a 3×3 convolution to extract the data of 8 grid points around this position, ensure that there are no missing values in the 8 grid points, otherwise resample 1 point as a supplement until there is data for all 5000 points and the 8 points around each point. Take the central grid of the extracted Sentinel-2 remote sensing image and the 8 surrounding grid data as the input according to the spatial position correspondence relationship, and the grid points at the corresponding positions of the Landsat-8 remote sensing image as the output. Set the training, validation, and test data sets, and randomly divide the data set ratio as training:validation:test = 8:1:1. The model is a streamlined RES-SENET model using residual blocks and SE blocks. Save the model weights with the best actual effect during the training process for subsequent calculations;

[0085] The structure of the streamlined RES-SENET model is:

[0086] x l+1 = x l + F(x l , W l );

[0087] Among them, x l represents the model input data, and x l+1 represents the input data of the next stage of the model. F(x l , W l ) represents the residual mapping to be learned by the model.

[0088] After connecting the SENET module to the output residual module, continue to train the model. Set the loss function for model training and select the L2 loss as the loss function for the model to train the model to narrow the gap between the predicted value and the true value. Record the metrics of the loss function when training the model, plot the change graph of the loss function and the result changes of the validation set and the test set. Set the upper limit of the training rounds to 500 rounds. When the loss value gradually decreases as the training rounds increase until the change is flat and hardly changes anymore, it can be considered that the model effect is better at this time. Save the model weights at this time to save the neural network model parameters. If the loss value shows up and down vibrations or extreme upward situations, consider adjusting the model training parameters such as the learning rate and decay rate to optimize the model. Save the model weights with the best actual effect during the training process for subsequent calculations. Calculate R based on the Landsat-8 remote sensing image index simulated by the deep learning Sentinel-2 remote sensing image index data calculated by the model and the real Landsat-8 remote sensing image index data. 2 and RMSE to verify the accuracy of the model.

[0089] (7) Construct a comprehensive calculation model based on the trained first deep learning model and the second deep learning model. Use the sliding window method to sequentially extract the Sentinel-2 remote sensing image index data of the row and column and its surrounding 8 points according to the raster row and column numbers, input the data into the trained second deep learning model to learn the characteristics of the Landsat-8 remote sensing image index data and output the simulated Landsat-8-Sentinel-2 remote sensing image index data. Input the Landsat-8-Sentinel-2 remote sensing image index data with learned characteristics into the trained first deep learning model to let the input Index 1, Index 2, and Index 3 learn the characteristics of the land surface temperature LST. Use the sliding window method to sequentially scan the Landsat-8-Sentinel-2 remote sensing image data to generate the downscaled Sentinel-2 remote sensing image land surface temperature image.

Claims

1. A downscaling method for land surface temperature of remote sensing images based on deep learning technology, characterized in that, It includes the following steps: (1) Obtain Landsat-8 and Sentinel-2 remote sensing image data and preprocess the data; (2) Calculate remote sensing indices and land surface temperature LST based on Landsat-8 remote sensing image data. The remote sensing indices include enhanced vegetation index EVI, normalized difference vegetation index NDVI, fractional vegetation cover FVC, normalized difference water index NDWI, and normalized difference built-up index NDBI; (3) Calculate the Pearson correlation coefficients of the remote sensing indices and land surface temperature LST described in step (2), and select three remote sensing indices with the highest correlation and non-collinearity; (4) Construct a first deep learning model by using the three remote sensing indices described in step (3) and the land surface temperature LST described in step (2) through a simplified SENET model; (5) Calculate remote sensing indices based on Sentinel-2 remote sensing image data, calculate the correlation coefficients with the land surface temperature LST described in step (2), and select three remote sensing indices with the highest correlation and non-collinearity; (6) Construct a second deep learning model by using the three remote sensing indices described in step (3) and the three remote sensing indices described in step (5). The second deep model selects the RESNET model with residual connection block and simplified SE block and the SENET model; (7) Construct a comprehensive calculation model based on the first deep learning model and the second deep learning model to generate a Sentinel-2 remote sensing image land surface temperature LST image with completed downscaling.

2. The surface temperature downscaling method according to claim 1, wherein The preprocessing described in step (1) includes radiometric calibration, atmospheric correction of Landsat-8 and Sentinel-2 remote sensing image data, and removing data in areas blocked by clouds.

3. The surface temperature downscaling method according to claim 1, characterized in that The calculation formulas of the remote sensing indices and land surface temperature LST described in step (2) are as follows: Among them, NIR represents the reflectance in the near-infrared band, RED represents the reflectance in the red band, BLUE represents the reflectance in the blue band, NDVI represents the NDVI value of the pixel, NDVI soil represents the maximum NDVI value of pure bare soil, NDVI vegetation represents the minimum NDVI value of pure vegetation in the image, GREEN represents the reflectance in the green band, MIR represents the reflectance in the mid-infrared band, T LST represents the surface temperature, BT represents the brightness temperature, λ represents the wavelength, C2 represents the second radiation constant, and ε represents the surface emissivity.

4. The method for downscaling surface temperature according to claim 1, wherein The specific steps for constructing the first deep learning model described in step (4) are as follows: (41) Randomly select the values of grid points without missing values in the data area as the data set, extract the three remote sensing indices of each corresponding point as the model input, synthesize the training matrix, and use the land surface temperature LST of each corresponding point as the model output; (42) Set the training set, validation set, and test set, randomly divide them according to the corresponding ratio of 8:1:1, and the model selects the simplified SENET model with SE block.

5. The downscaling method of surface temperature according to claim 4, wherein The specific process of synthesizing the training matrix described in step (41) is as follows: Among them, input (a,b,…,n) represents the deep learning remote sensing index matrix, and the component index is three remote sensing indexes from period a to period n.

6. The method for downscaling land surface temperature according to claim 4, wherein The working process expression of the simplified SENET model described in step (42) is as follows: S = F x (Z, W) = σ(g(Z, W)); Among them, Z represents the output after the Squeeze operation, W and H are the sizes of the input matrix, and x(i,j) represents the element in the i-th row and j-th column of the matrix; S represents the output after the Excitation operation, σ represents the sigmoid function, and g represents the linear layer; represents the output after the tensor U operation, U represents a two-dimensional matrix, and S represents the learned weight; L2(MSE) represents the loss function, n represents the sample size of the training data set, and y sr represents the predicted value obtained after model training, and y hr represents the true value obtained after model training; P i represents the model's simulated land surface temperature data, represents the average value of the model's simulated land surface temperature data, and M i represents the actual land surface temperature data, represents the average value of the actual land surface temperature data.

7. The method for downscaling land surface temperature according to claim 1, wherein The specific steps for constructing the second deep learning model described in step (6) are as follows: (61) Randomly select the grid points of Landsat-8 remote sensing images without missing values in the data area, obtain the geographical location information of the points, and extract the grid points of Sentinel-2 remote sensing images according to the corresponding positions; (62) Use the central grid and its surrounding grid data of the extracted Sentinel-2 remote sensing image as the input according to the spatial position correspondence relationship, and use the grid points at the corresponding positions of the Landsat-8 remote sensing image as the output; (63) Set the training set, validation set, and test set, and randomly divide them according to the corresponding ratio of 8:1:1; (64)Construct a deep learning model of sensor relationship using the three remote sensing indices described in step 3 and the three remote sensing indices described in step 5. The model selects the RESNET model with residual connection blocks and streamlined SE blocks and the SENET model.

8. A downscaling system for land surface temperature of remote sensing images based on deep learning technology, characterized in that, Including: A data acquisition module for obtaining Landsat-8 and Sentinel-2 remote sensing image data; A data preprocessing module for performing radiometric calibration, atmospheric correction, and cloud occlusion removal data processing on the acquired data; An index calculation module for calculating remote sensing indices and land surface temperature LST based on Landsat-8 and Sentinel-2 image data respectively; A model construction module for constructing a first deep learning model, a second deep learning model, and a comprehensive calculation model; An image generation module for generating a Sentinel-2 remote sensing image land surface temperature LST image with completed downscaling by the comprehensive calculation model.

9. A computer device, characterized in that, Including one or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and are configured to be executed by the one or more processors. When the program is executed by the processor, it implements the steps of a method for downscaling the land surface temperature of remote sensing images based on deep learning technology as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of a method for downscaling the land surface temperature of remote sensing images based on deep learning technology as described in any one of claims 1-7.

Images