A deep learning satellite-borne microwave radiometer remote sensing brightness temperature super-resolution reconstruction method

Abstract

The application belongs to the technical field of processing of satellite-borne microwave radiometer remote sensing brightness temperature, and discloses a deep learning satellite-borne microwave radiometer remote sensing brightness temperature super-resolution reconstruction method, which comprises the following steps: constructing a microwave brightness temperature super-resolution data set; constructing a microwave brightness temperature super-resolution model MW-SRUNet; the encoder part performs up-sampling on the original brightness temperature through deconvolution, extracts the earth surface feature brightness temperature information through convolution and pooling, and generates a feature matrix; the decoder part restores the brightness temperature to the size of the original brightness temperature through multiple deconvolutions; a self-defined loss function is used for training the model in the area where the earth surface brightness temperature changes sharply, the influence of abnormal values on the model training is reduced, a self-defined evaluation function is used for evaluating the model training process, and the super-resolution performance of the model is intuitively understood; and the output of the model is the reconstructed high-resolution microwave brightness temperature. The application solves the problem of low microwave brightness temperature resolution, and realizes 4 times super-resolution reconstruction of the microwave brightness temperature.

Description

Technical Field

[0001] This invention belongs to the field of brightness temperature processing technology for spaceborne microwave radiometers, and specifically relates to a deep learning method for super-resolution reconstruction of brightness temperature in spaceborne microwave radiometers. Background Technology

[0002] High-resolution Earth surface brightness temperature data are crucial for studying Earth system changes. However, due to limitations in antenna size and satellite weight, the resolution of brightness temperature data obtained from spaceborne microwave radiometers is typically low, hindering the full utilization of the data. Therefore, there is a need to improve the resolution of microwave remote sensing brightness temperature.

[0003] There are two main approaches to improving the spatial resolution of brightness temperature data from microwave radiometers: Firstly, from a payload physics perspective, increasing the aperture of the payload radiometer's antenna can result in a payload antenna with high main beam efficiency and a smaller half-power width. However, the aperture of spaceborne microwave radiometer antennas is limited by platform space, and due to the special nature of aerospace equipment, it is characterized by long development cycles and high costs. Furthermore, large antennas increase satellite weight, and deformation and jitter of large antennas during satellite operation can affect the quality of microwave radiometer brightness temperature data. From a satellite system design perspective, the antenna aperture cannot be increased indefinitely to improve spatial resolution, and hardware is also nearing its limits due to industrial technology constraints. Secondly, from a data processing perspective, improving the spatial resolution of microwave radiometer brightness temperature can be achieved. Various super-resolution algorithms can be used to recover high-resolution brightness temperatures through deconvolution techniques, i.e., super-resolution brightness temperature reconstruction technology. This technology can obtain higher-resolution brightness temperatures without changing the physical antenna size. However, existing super-resolution brightness temperature reconstruction methods perform poorly when processing microwave brightness temperatures and cannot achieve high-resolution reconstruction. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a deep learning-based method for super-resolution reconstruction of brightness temperature from a spaceborne microwave radiometer, effectively improving the resolution of microwave brightness temperature and achieving 4x super-resolution reconstruction.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0006] A deep learning-based method for super-resolution reconstruction of brightness temperature from a spaceborne microwave radiometer includes the following steps:

[0007] S1. Construct a microwave brightness temperature super-resolution dataset;

[0008] S2. Constructing a microwave brightness temperature super-resolution model MW-SRUNet (Microwave Super Resolution UNet) based on the microwave brightness temperature super-resolution dataset from step S1: The UNet network is introduced into the microwave brightness temperature super-resolution model. The microwave brightness temperature super-resolution model MW-SRUNet includes an encoder part and a decoder part. In the encoder part, the original brightness temperature is first upsampled through deconvolution, and then the brightness temperature information of ground objects on the Earth's surface is extracted through convolution and pooling to generate a feature matrix. The network structure of the decoder part is a mirror image of the encoder structure, and the brightness temperature is restored to the original brightness temperature size through deconvolution.

[0009] S3. Input the original low-resolution microwave brightness temperature into the microwave brightness temperature super-resolution model MW-SRUNet constructed in step S2. The output of the model is the reconstructed high-resolution microwave brightness temperature.

[0010] Furthermore, the dataset in step S1 is a high-low resolution brightness temperature matching dataset, where the low-resolution brightness temperature is the original brightness temperature of the microwave radiometer, and the high-resolution brightness temperature is the corresponding region resolution-enhanced brightness temperature, specifically including:

[0011] S101: Mask the global microwave brightness temperature, retaining only the area with the brightness temperature value;

[0012] S102: Divide the global microwave brightness temperature into blocks to establish a super-resolution dataset. Where: the low-resolution brightness temperature blocks are divided with a pixel size of n1×n1, and the corresponding high-resolution brightness temperature block size is n2×n2, n2=4n1.

[0013] S103: Perform sequential statistical filtering on the cut global microwave brightness temperatures to batch extract the brightness temperatures, and set the extraction step length to reduce the repetition between brightness temperature blocks;

[0014] S104: The preprocessed global microwave brightness temperature blocks are iteratively connected according to the measurement time, so that the data is stored in the form of a three-dimensional matrix. The first dimension is the number of samples, and the second and third dimensions are the length and width of the brightness temperature. A high-low resolution brightness temperature matching dataset is constructed for the microwave brightness temperature super-resolution model MW-SRUNet.

[0015] Furthermore, step S2 specifically includes:

[0016] S201: During the training of the microwave brightness temperature super-resolution model MW-SRUNet, 70% of the microwave brightness temperature super-resolution dataset constructed in step S1 is used for training, 20% is used as a validation set to validate the parameters of the iteration, the model with the smallest error in the validation set is saved, and the remaining 10% of the data is used as the prediction set of the model.

[0017] S202: The microwave brightness temperature super-resolution model MW-SRUNet is implemented based on the Tensorflow-Keras framework. The optimization method uses Adam, and the callback function uses Reduce LR On Plateau for adaptive adjustment of the learning rate.

[0018] S203: A custom loss function is used to train the model in regions with drastic changes in brightness and temperature on the Earth's surface. This reduces the impact of outliers on model training and improves network convergence. The specific formula for the loss function is as follows: In the formula, N is the number of pixels in the brightness temperature, and I i It is a high-resolution brightness temperature sampled from real data. This indicates the high-resolution brightness temperature of the reconstruction.

[0019] S204: Use a custom evaluation function to evaluate the model training process to gain a more intuitive understanding of the super-resolution performance during model training. The specific formula for the evaluation function is as follows: In the formula, L represents the maximum value among the pixel values ​​in the brightness temperature, and MSE represents the mean square error between the high-resolution brightness temperature and the low-resolution brightness temperature.

[0020] Furthermore, the microwave brightness temperature super-resolution model MW-SRUNet has a low-resolution brightness temperature of 24×24 pixels as its input layer and a high-resolution brightness temperature of 96×96 pixels as its output layer. In the encoder part, the original brightness temperature is first upsampled by 4 times through two 2×2 deconvolutions, and then the brightness temperature information of ground objects on the Earth's surface is extracted through four convolutions and pooling to generate a feature matrix. In the decoder part, the brightness temperature is restored to 96×96 through four deconvolutions to obtain a 4-fold super-resolution reconstructed brightness temperature of microwave brightness temperature.

[0021] Compared with the prior art, the advantages of the present invention are as follows:

[0022] (1) The microwave brightness temperature super-resolution dataset constructed in this invention can be used for global microwave brightness temperature super-resolution reconstruction, providing data support for future microwave brightness temperature super-resolution work and expanding the application research of spaceborne microwave radiometer brightness temperature data in Earth system changes.

[0023] (2) The microwave brightness temperature super-resolution model MW-SRUNet of the present invention is based on the UNet network design. It increases the number of network layers to learn deeper brightness temperature features of Earth's surface objects. It introduces deconvolution layers to adaptively upsample the brightness temperature and directly uses low-resolution brightness temperature as the network input to construct an end-to-end network structure, thereby enabling more explicit learning of the original information of brightness temperature of Earth's surface objects. Through skip connections, the feature matrix in the downsampling process is fused during the upsampling process. This method helps to reduce information loss caused by the reduction in resolution in the downsampling data stream. The model is trained using a custom loss function for regions with drastic changes in brightness temperature on the Earth's surface to reduce the impact of large outliers on model training and to improve the convergence of the network. A custom evaluation function is used to evaluate the model training process to more intuitively understand the super-resolution performance of the model during training.

[0024] (3) This invention solves the problem of low microwave brightness temperature resolution and successfully improves the microwave brightness temperature resolution from 36km to 9km, achieving a 4-fold super-resolution reconstruction of microwave brightness temperature. It has a wide range of applications and can improve the brightness temperature resolution of both land and ocean. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart illustrating the deep learning-based spaceborne microwave radiometer remote sensing brightness temperature super-resolution reconstruction method in an embodiment of the present invention.

[0027] Figure 2 This is a network structure diagram of the microwave brightness temperature super-resolution model MW-SRUNet in an embodiment of the present invention. Detailed Implementation

[0028] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0029] Example 1

[0030] Combination Figure 1 , Figure 2 A deep learning-based method for super-resolution reconstruction of brightness temperature from a spaceborne microwave radiometer includes the following steps:

[0031] S1. Construct a microwave brightness temperature super-resolution dataset.

[0032] This dataset is a high-to-low resolution brightness temperature matching dataset. The low-resolution brightness temperature is the original brightness temperature from the microwave radiometer, and the high-resolution brightness temperature is the corresponding region-enhanced brightness temperature. Specifically, it includes:

[0033] S101: Mask the global microwave brightness temperature, retaining only the areas with brightness temperature values.

[0034] S102: Divide the valuable regions of global microwave brightness temperature into blocks to establish a super-resolution dataset. Specifically: if the low-resolution brightness temperature blocks are divided with a pixel size of n1×n1, then the corresponding high-resolution brightness temperature block size is n2×n2, where n2=4n1.

[0035] S103: Perform sequential statistical filtering on the global microwave brightness temperature of the sliced ​​blocks to batch extract the brightness temperature, and set the extraction step length to reduce the repetition between brightness temperature blocks.

[0036] S104: The preprocessed global microwave brightness temperature blocks are iteratively connected according to measurement time, so that the data is stored in the form of a three-dimensional matrix. The first dimension is the number of samples, and the second and third dimensions are the length and width of the brightness temperature. A high- and low-resolution brightness temperature matching dataset is constructed for the microwave brightness temperature super-resolution model MW-SRUNet (Microwave Super Resolution UNet).

[0037] S2. Based on the microwave brightness temperature super-resolution dataset from step S1, construct a deep convolutional neural network microwave brightness temperature super-resolution model MW-SRUNet (Microwave Super Resolution UNet).

[0038] This invention introduces the UNet network into the microwave brightness temperature super-resolution model to achieve the microwave brightness temperature super-resolution task, establishing the microwave brightness temperature super-resolution model MW-SRUNet. Figure 2 As shown, the microwave brightness temperature super-resolution model MW-SRUNet includes an encoder part and a decoder part. In the encoder part, the original brightness temperature (i.e., low-resolution brightness temperature) is first upsampled by deconvolution, and then the brightness temperature information of ground objects on the Earth's surface is extracted by convolution and pooling to generate a feature matrix. The network structure of the decoder part is a mirror image of the encoder structure, and the brightness temperature is restored to the size of the original brightness temperature by deconvolution.

[0039] The following are the specific implementation steps: The microwave brightness temperature super-resolution model MW-SRUNet has a low-resolution brightness temperature of 24×24 pixels as its input layer and a high-resolution brightness temperature of 96×96 pixels as its output layer. In the encoder part, the original brightness temperature is first upsampled by 4 times through two 2×2 deconvolutions. Then, the brightness temperature information of ground objects on the Earth's surface is extracted through four convolutions and pooling to generate a feature matrix (the convolutions in the figure without annotations are all 3*3 convolutions). The network structure of the decoder part is a mirror image of the encoder structure. In the decoder part, the brightness temperature is restored to 96×96 through four deconvolutions to obtain a 4-fold super-resolution reconstructed brightness temperature of microwave brightness temperature.

[0040] Step S2 involves the training details of the model during its construction:

[0041] S201: During the training of the microwave brightness temperature super-resolution model MW-SRUNet, 70% of the microwave brightness temperature super-resolution dataset constructed in step S1 is used for training, 20% is used as a validation set to validate the parameters of the iteration, the model with the smallest error in the validation set is saved, and the remaining 10% of the data is used as the prediction set of the model.

[0042] S202: The microwave brightness temperature super-resolution model MW-SRUNet is implemented based on the Tensorflow-Keras framework. The optimization method uses Adam, and the callback function uses Reduce LR On Plateau for adaptive adjustment of the learning rate.

[0043] S203: A custom loss function is used to train the model in regions with drastic changes in brightness and temperature on the Earth's surface. This reduces the impact of large outliers on model training and improves network convergence. The specific formula for the loss function is as follows: In the formula, N is the number of pixels in the brightness temperature, and I i It is a high-resolution brightness temperature sampled from real data. This indicates the high-resolution brightness temperature of the reconstruction.

[0044] S204: Use a custom evaluation function to evaluate the model training process to gain a more intuitive understanding of the super-resolution performance during model training. The specific formula for the evaluation function is as follows: In the formula, L represents the maximum value among the pixel values ​​in the brightness temperature, and MSE represents the mean square error between the high-resolution brightness temperature and the low-resolution brightness temperature.

[0045] S3. Input the original low-resolution microwave brightness temperature into the microwave brightness temperature super-resolution model MW-SRUNet constructed in step S2. The output of the model is the reconstructed high-resolution microwave brightness temperature.

[0046] Example 2

[0047] This embodiment uses the Soil Moisture Active Passive (SMAP) satellite launched by NASA as an example to quantitatively compare the proposed deep learning-based spaceborne microwave radiometer remote sensing brightness temperature super-resolution reconstruction method with the traditional bicubic interpolation method and the SRCNN method (as shown in Table 1). The evaluation metrics for the quantitative comparison include brightness temperature quality assessment and quantitative remote sensing regression accuracy assessment. Brightness temperature quality assessment includes Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM), which are calculated as follows: In the formula, L represents the maximum value among the pixel values ​​in the brightness temperature, and MSE represents the mean square error between the high-resolution brightness temperature and the low-resolution brightness temperature. In the formula, This indicates a comparison of brightness between the reconstructed image and the reference image. Indicates a comparison of contrast. This is a structural comparison function, where α, β, and γ are control parameters that adjust the relative importance. In practice, to simplify calculations, the control parameters are generally set to α = β = γ = 1. Quantitative remote sensing regression accuracy assessment includes the root mean square error (RMSE) and the Pearson linear correlation coefficient (PLCC), which are calculated as follows: In the formula, N is the number of pixels in the brightness temperature, and I(i) is the high-resolution brightness temperature sampled from the real data. This indicates the high-resolution brightness temperature of the reconstruction.

[0048] Table 1 Evaluation indicators of reconstruction results of different super-resolution methods

[0049]

[0050]

[0051] It can be observed that the MW-SRUNet reconstruction results have an average PSNR of 32.12 and an SSIM of 0.9726, which are superior to the traditional interpolation methods' 22.89 / 0.8358 and the shallow CNN methods' 23.1850 / 0.8636. The average RMSE is 5.62K, significantly lower than the traditional methods' 15.36K / 14.53K and the shallow CNN methods' 14.53K. The average PLCC is 0.9884, superior to the traditional methods' 0.9507 / 0.9551 and the shallow CNN methods' 14.53K. In all evaluation metrics of the reconstruction results, the MW-SRUNet algorithm outperforms both the traditional bicubic interpolation method and the shallow CNN method.

[0052] In summary, this invention can solve the problem of low brightness temperature resolution in traditional spaceborne microwave radiometers, thereby improving the spatial resolution of microwave radiometer brightness temperature without requiring improvements to the payload antenna design, but only from the perspective of brightness temperature data processing.

[0053] The deep learning-based spaceborne microwave radiometer brightness temperature super-resolution reconstruction method of this invention effectively solves the problem of low microwave brightness temperature resolution, successfully improving the spatial resolution of 36km brightness temperature to 9km, achieving a 4-fold super-resolution reconstruction of microwave brightness temperature. This provides a new method and approach for acquiring high-resolution microwave remote sensing brightness temperature data and expands the application research of spaceborne microwave radiometer brightness temperature data in Earth system change.

[0054] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should be protected by the present invention.

Claims

1. A method for super-resolution reconstruction of brightness temperature from a deep learning-based spaceborne microwave radiometer, characterized in that... Includes the following steps: S1. Construct a microwave brightness temperature super-resolution dataset; S2. Construct a microwave brightness temperature super-resolution model MW-SRUNet based on the microwave brightness temperature super-resolution dataset from step S1: Introduce the UNet network into the microwave brightness temperature super-resolution model. The microwave brightness temperature super-resolution model MW-SRUNet includes an encoder part and a decoder part. In the encoder part, the original brightness temperature is first upsampled by deconvolution, and then the brightness temperature information of ground objects on the Earth's surface is extracted by convolution and pooling to generate a feature matrix. The network structure of the decoder is a mirror image of the encoder structure, and the brightness temperature is restored to the original brightness temperature size through multiple deconvolutions. S3. Input the original low-resolution microwave brightness temperature into the microwave brightness temperature super-resolution model MW-SRUNet constructed in step S2. The output of the model is the reconstructed high-resolution microwave brightness temperature.

2. The method for super-resolution reconstruction of brightness temperature using a deep learning-based spaceborne microwave radiometer according to claim 1, characterized in that, The dataset in step S1 is a high-low resolution brightness temperature matching dataset, where the low-resolution brightness temperature is the original brightness temperature of the microwave radiometer, and the high-resolution brightness temperature is the corresponding region resolution-enhanced brightness temperature, specifically including: S101: Mask the global microwave brightness temperature, retaining only the area with the brightness temperature value; S102: Divide the global microwave brightness temperature into blocks to establish a super-resolution dataset. Where: the low-resolution brightness temperature blocks are divided with a pixel size of n1×n1, and the corresponding high-resolution brightness temperature block size is n2×n2, n2=4n1. S103: Perform sequential statistical filtering on the cut global microwave brightness temperatures to batch extract the brightness temperatures, and set the extraction step length to reduce the repetition between brightness temperature blocks; S104: The preprocessed global microwave brightness temperature blocks are iteratively connected according to the measurement time, so that the data is stored in the form of a three-dimensional matrix. The first dimension is the number of samples, and the second and third dimensions are the length and width of the brightness temperature. A high-low resolution brightness temperature matching dataset is constructed for the microwave brightness temperature super-resolution model MW-SRUNet.

3. The method for super-resolution reconstruction of brightness temperature from a deep learning-based spaceborne microwave radiometer according to claim 1, characterized in that, S2 specifically includes: S201: During the training of the microwave brightness temperature super-resolution model MW-SRUNet, 70% of the microwave brightness temperature super-resolution dataset constructed in step S1 is used for training, 20% is used as a validation set to validate the parameters of the iteration, the model with the smallest error in the validation set is saved, and the remaining 10% of the data is used as the prediction set of the model. S202: The microwave brightness temperature super-resolution model MW-SRUNet is implemented based on the Tensorflow-Keras framework. The optimization method uses Adam, and the callback function uses Reduce LR On Plateau for adaptive adjustment of the learning rate. S203: A custom loss function is used to train the model in regions with drastic changes in brightness and temperature on the Earth's surface. This reduces the impact of outliers on model training and improves network convergence. The specific formula for the loss function is as follows: In the formula, N is the number of pixels in the brightness temperature, and I i It is a high-resolution brightness temperature sampled from real data. This indicates the high-resolution brightness temperature of the reconstruction. S204: Use a custom evaluation function to evaluate the model training process to gain a more intuitive understanding of the super-resolution performance during model training. The specific formula for the evaluation function is as follows: In the formula, L represents the maximum value among the pixel values ​​in the brightness temperature, and MSE represents the mean square error between the high-resolution brightness temperature and the low-resolution brightness temperature.

4. The method for super-resolution reconstruction of brightness temperature using a deep learning-based spaceborne microwave radiometer according to claim 1, characterized in that, The microwave brightness temperature super-resolution model MW-SRUNet has a low-resolution brightness temperature of 24×24 pixels as its input layer and a high-resolution brightness temperature of 96×96 pixels as its output layer. In the encoder part, the original brightness temperature is first upsampled by 4 times through two 2×2 deconvolutions, and then the brightness temperature information of ground objects on the Earth's surface is extracted through four convolutions and pooling to generate a feature matrix. In the decoder part, the brightness temperature is restored to 96×96 through four deconvolutions to obtain a 4-fold super-resolution reconstructed brightness temperature of microwave brightness temperature.

Images