Multi-scale progressive surface temperature fusion downscaling method and device

Abstract

The invention relates to the technical field of remote sensing image intelligent processing, in particular to a multi-scale progressive surface temperature fusion downscaling method and device, and the method comprises the steps: preprocessing a surface temperature image in a multi-scale image library; constructing a multi-scale training data set containing the surface temperature and at least one auxiliary parameter; constructing a surface temperature downscaling network based on multi-parameter fusion, and constructing a domain transformation network based on heterogeneous high-frequency information guidance; training a surface temperature downscaling network and a domain transformation network based on a progressive dual-network joint training strategy; finely adjusting the downscaling model meeting a preset low-medium resolution condition; and performing low-medium-high progressive downscaling on the low-resolution surface temperature image to be processed to generate a multi-scale progressive surface temperature meeting a target high-resolution condition. The method fully considers the problem of descending scale difference of different resolutions, achieves the maintenance of the physical characteristics of the surface thermal field, and improves the processing efficiency of large-area remote sensing data.

Description

Technical Field

[0001] This invention relates to the field of intelligent processing technology for remote sensing images, and in particular to a multi-scale progressive surface temperature fusion downscaling method and apparatus. Background Technology

[0002] Satellite remote sensing technology is the most effective means of rapidly acquiring regional or global-scale land surface temperature data, playing a crucial role in research on agricultural drought monitoring, urban heat island effect, global hydrothermal cycle, and climate change. However, due to hardware limitations of thermal infrared sensors, existing land surface temperature products are typically only at the hundred-meter resolution, posing challenges to obtaining higher spatial resolution land surface temperature data and failing to meet the needs of refined applications. Spatial downscaling techniques are an effective way to obtain higher spatial resolution land surface temperature data.

[0003] Existing spatial downscaling techniques for land surface temperature (LST) are mainly based on statistical regression, surface energy balance, and geographic weighted regression. Among these, statistical regression is widely used in LST spatial downscaling research due to its simplicity and high accuracy. Early regression algorithms primarily focused on fitting linear or simple nonlinear statistical relationships between LST and remote sensing indices, but they were insufficient in handling complex nonlinear features and spatial heterogeneity. With the rapid development of deep learning technology, its powerful feature extraction and nonlinear modeling capabilities have been introduced into LST downscaling research, significantly improving the expressive power and accuracy of the models. However, most existing deep learning downscaling methods assume that the mapping relationship between LST and auxiliary parameters remains consistent across different scales. Typically, the mapping relationship is learned at a low-resolution scale and directly applied to high-resolution auxiliary parameters to predict high-resolution LST. This assumption ignores the local heterogeneity caused by scale differences between high and low resolutions, limiting the accuracy of the downscaling results. Summary of the Invention

[0004] This invention provides a multi-scale progressive surface temperature fusion downscaling method and apparatus to solve the problems of existing deep learning downscaling methods ignoring the local heterogeneity caused by the scale difference between high and low resolution, which limits the accuracy of downscaling results.

[0005] A first aspect of this invention provides a multi-scale progressive land surface temperature fusion downscaling method, comprising the following steps: acquiring at least one of the following products: a land surface temperature product from a target satellite meeting preset low-resolution conditions, a land surface temperature product from a reference satellite meeting preset medium-resolution conditions, and an auxiliary parameter product from the reference satellite meeting preset high-resolution conditions; resampling the at least one product to generate a multi-scale image library corresponding to the resolution of the at least one product, and preprocessing the land surface temperature images in the multi-scale image library to generate preprocessed images; constructing a multi-scale training dataset containing land surface temperature and at least one auxiliary parameter using the preprocessed images; constructing a land surface temperature downscaling network based on multi-parameter fusion based on the multi-scale training dataset, and constructing a domain transformation network guided by heterogeneous high-frequency information based on the multi-scale training dataset; and using the multi-scale images based on a progressive dual-network joint training strategy. The land surface temperature downscaling network and the domain transformation network are trained using a training dataset that meets preset low-to-medium resolution conditions to generate a downscaling model that meets preset low-to-medium resolution conditions and a downscaling result that meets preset medium resolution conditions. The downscaling model that meets preset low-to-medium resolution conditions is then fine-tuned using a dataset that meets preset medium-to-high resolution conditions in the multi-scale training dataset to generate a downscaling model that meets preset medium-to-high resolution conditions and a downscaling result that meets preset high resolution conditions. Based on the downscaling model that meets preset low-to-medium resolution conditions, the downscaling result that meets preset medium resolution conditions, the downscaling model that meets preset medium-to-high resolution conditions, and the downscaling result that meets preset high resolution conditions, a low-to-medium-to-high progressive downscaling is performed on the low-resolution land surface temperature image to be processed to generate a multi-scale progressive land surface temperature image that meets the target high resolution conditions.

[0006] Optionally, in one embodiment of the present invention, the preprocessing of the land surface temperature images in the multi-scale image library to generate preprocessed images includes: calculating the normalized vegetation index, normalized water index, and normalized building index using the reflectance images of the target satellite; performing statistical characteristic correction on the land surface temperature data of the reference satellite to generate a land surface temperature image of the reference satellite that meets the target resolution conditions; and preprocessing the land surface temperature images of the target satellite, the land surface temperature images of the reference satellite, and images of the digital elevation model, the normalized vegetation index, the normalized water index, and the normalized building index to generate the preprocessed images.

[0007] Optionally, in one embodiment of the present invention, the formula for calculating the surface temperature image of the reference satellite that satisfies the target resolution condition is:

[0008] in, This represents a medium-resolution reference satellite surface temperature image after histogram matching. This represents the original medium-resolution reference satellite surface temperature image. and These represent the standard deviations of the low-resolution target satellite surface temperature image and the original medium-resolution reference satellite surface temperature image, respectively. and These represent the average values ​​of the low-resolution target satellite surface temperature image and the original medium-resolution reference satellite surface temperature image, respectively.

[0009] Optionally, in one embodiment of the present invention, constructing a multi-scale training dataset containing land surface temperature and at least one auxiliary parameter using the preprocessed images includes: cropping the multi-scale image library into at least one set of image patches; generating a medium-to-high resolution dataset in the multi-scale training dataset based on the target satellite land surface temperature image patch satisfying a preset low-resolution condition, the reference satellite land surface temperature image patch satisfying a preset medium-resolution condition, the digital elevation model image patch satisfying a preset medium-resolution condition, the normalized vegetation index image patch, the normalized water index image patch, and the normalized building index image patch in the at least one set of image patches; and generating a high-resolution auxiliary parameter dataset in the multi-scale training dataset based on the digital elevation model image patch, the normalized vegetation index image patch, the normalized water index image patch, and the normalized building index image patch satisfying a preset high-resolution condition in the at least one set of image patches.

[0010] Optionally, in one embodiment of the present invention, the step of constructing a land surface temperature downscaling network based on multi-parameter fusion according to the multi-scale training dataset includes: extracting preliminary features from land surface temperature satisfying a preset coarse resolution condition and auxiliary land surface parameters satisfying a preset fine resolution condition; constructing a multi-scale feature extraction module, and using the multi-scale feature extraction module to extract multi-scale features of the preliminary features of the auxiliary land surface parameters satisfying the preset fine resolution condition; constructing a lightweight CNN module, and using the lightweight CNN module to extract local features from the land surface temperature satisfying the preset coarse resolution condition and the multi-scale features, and compressing the local features to generate a compressed result; constructing a lightweight Transformer module, and using the lightweight Transformer module to extract features from the compressed result to generate global features; and constructing a downscaling reconstruction module, and using the downscaling reconstruction module to process the global features to generate a downscaled land surface temperature.

[0011] Optionally, in one embodiment of the present invention, the formula for extracting the preliminary features is:

[0012]

[0013] in, and These represent the preliminary characteristics of the surface temperature that meets the preset coarse resolution condition and the surface parameter that meets the preset fine resolution condition, respectively. This represents the surface temperature that meets the preset coarse resolution conditions. This indicates a channel addition operation. This represents the convolution operation. , and , These are the weights and biases for the two convolution operations, respectively. For digital elevation models, Normalized Difference Vegetation Index (NDVI) The normalized water index, The normalized building index; The formula for constructing the multi-scale features is as follows:

[0014] in, The output feature map of the multi-scale features, This indicates the convolutional attention operation. Indicates size is × and × The feature map obtained by combining the features obtained from convolution. Indicates size is × and × The feature map obtained by combining the features obtained from convolution; The calculation formulas for the feature maps are as follows:

[0015]

[0016] in, This represents the ReLU activation function. , , , and , They are respectively the size of × , × , × The weights and biases in the convolution operation; The formulas for extracting local features and calculating channel compression are as follows:

[0017]

[0018]

[0019] in, and These are the local features obtained by the lightweight CNN module through surface temperature and multi-scale feature processing that meet the preset coarse resolution conditions. These are the channel-compressed feature maps of the outputs of two lightweight CNN architectures. This indicates a 2x upsampling operation. , and , These represent the weight and bias terms of the two lightweight CNN modules, respectively. and These are the weight and bias terms for the channel compression convolution operation, respectively. The calculation formula for the lightweight Transformer module is as follows:

[0020] in, This is the output feature map of the lightweight Transformer module. The outputs of the two lightweight CNN architectures are compressed feature maps across channels. The number of channel segments, Indicates the scaling factor. , , This represents the matrix obtained by linear projection of the input feature map; The formulas for calculating the matrix are as follows:

[0021]

[0022]

[0023] in, ,in For the segmentation operation; The formula for calculating the downscaled surface temperature is as follows:

[0024] in, The surface temperature output by the downscaling reconstruction module. This represents the surface temperature that meets the preset coarse resolution conditions. This indicates a 2x upsampling operation. , and , These are the weights and biases for single-channel and three-channel convolutions, respectively. Optionally, in one embodiment of the present invention, the step of constructing a domain transformation network guided by heterogeneous high-frequency information based on the multi-scale training dataset includes: extracting high-frequency information of land surface temperature from the downscaled land surface temperature using Laplacian high-pass filtering; extracting local features from the high-frequency information of land surface temperature using the lightweight CNN module; extracting global features from the local features using the lightweight Transformer module; and processing the feature map using single-channel convolution based on the global features, and constructing the domain transformation network guided by heterogeneous high-frequency information using residual connections.

[0025] Optionally, in one embodiment of the present invention, the formula for extracting high-frequency surface temperature information is:

[0026] in, This indicates the results of high-frequency information extraction of land surface temperature. This represents a Laplace high-pass filter, where the filter kernel is a matrix. , The surface temperature output by the downscaling reconstruction module; The matrix Defined as:

[0027] The formula for calculating the local features is:

[0028] in, This represents local features extracted from the high-frequency information of the land surface temperature. This refers to the lightweight CNN module. This represents local features extracted from the high-frequency information of the land surface temperature; The formula for calculating the global features is as follows:

[0029] in, This represents the global features extracted from the local features. This refers to the lightweight Transformer module. This represents local features extracted from the high-frequency information of the land surface temperature; The formula for reconstructing high-frequency information in the domain transform network is:

[0030] in, The high-frequency information prediction results of NDVI obtained after domain transformation are shown. and These represent the weight and bias terms of a single-channel convolution, respectively. This represents the global features extracted from the local features.

[0031] Optionally, in one embodiment of the present invention, the calculation formulas for the surface temperature downscaling network and the domain transformation network are respectively:

[0032]

[0033] in, and Let represent the loss functions of the surface temperature downscaling network and the domain transformation network, respectively. This represents the surface temperature fidelity loss under the preset coarse resolution conditions. This represents the frequency domain consistency loss that satisfies the preset fine-resolution condition. This represents the cross-modal frequency domain alignment loss. For neural network parameters, These are all empirical coefficients used to adjust the weights of each sub-loss function; The formula for calculating the surface temperature fidelity loss that meets the preset coarse resolution condition is as follows:

[0034] in, This indicates the number of training image pairs. Indicates the first Coarse-resolution surface temperature images from the image pair. Represents network 1, This indicates resampling to a coarse-resolution data scale; The formula for calculating the frequency domain consistency loss that satisfies the preset fine resolution condition is as follows:

[0035] in, This indicates the number of training image pairs. This indicates a Laplace high-pass filter. Indicates the first Reference fine-resolution surface temperature image in the image pair; The formula for calculating the cross-modal frequency domain alignment loss is as follows:

[0036] in, This refers to network 1. Represents network 2, Indicates the first For the fine-resolution NDVI image in the image pair, Indicates the first Coarse-resolution surface temperature images from the image pair.

[0037] A second aspect of the present invention provides a multi-scale progressive land surface temperature fusion and downscaling device, comprising: an acquisition module, configured to acquire at least one of the following: a land surface temperature product from a target satellite meeting preset low-resolution conditions, a land surface temperature product from a reference satellite meeting preset medium-resolution conditions, and an auxiliary parameter product from the reference satellite meeting preset high-resolution conditions; a preprocessing module, configured to resample the at least one product to generate a multi-scale image library corresponding to the resolution of the at least one product, and preprocess the land surface temperature images in the multi-scale image library to generate preprocessed images; a data construction module, configured to construct a multi-scale training dataset containing land surface temperature and at least one auxiliary parameter using the preprocessed images; a network construction module, configured to construct a land surface temperature downscaling network based on multi-parameter fusion based on the multi-scale training dataset, and construct a domain transformation network guided by heterogeneous high-frequency information based on the multi-scale training dataset; and a training module, configured to perform progressive dual-network joint training. The strategy involves training the land temperature downscaling network and the domain transformation network using a training dataset that meets preset low-to-medium resolution conditions from the multi-scale training dataset, to generate a downscaling model that meets preset low-to-medium resolution conditions and a downscaling result that meets preset medium resolution conditions. A generation module is used to fine-tune the downscaling model that meets preset low-to-medium resolution conditions using a dataset that meets preset medium-to-high resolution conditions from the multi-scale training dataset, to generate a downscaling model that meets preset medium-to-high resolution conditions and a downscaling result that meets preset high resolution conditions. A downscaling processing module is used to perform low-to-medium-to-high progressive downscaling on the low-resolution land temperature image to be processed based on the downscaling model that meets preset low-to-medium resolution conditions, the downscaling result that meets preset medium-to-high resolution conditions, and the downscaling model that meets preset medium-to-high resolution conditions, to generate a multi-scale progressive land temperature image that meets the target high resolution conditions.

[0038] Optionally, in one embodiment of the present invention, the preprocessing module includes: a calculation unit for calculating the normalized vegetation index, normalized water index, and normalized building index using the reflectance image of the target satellite; a correction unit for performing statistical characteristic correction on the land surface temperature data of the reference satellite to generate a land surface temperature image of the reference satellite that meets the target resolution conditions; and a preprocessing unit for preprocessing the land surface temperature image of the target satellite, the land surface temperature image of the reference satellite, the digital elevation model, the normalized vegetation index, the normalized water index, and the normalized building index to generate the preprocessed image.

[0039] Optionally, in one embodiment of the present invention, the formula for calculating the surface temperature image of the reference satellite that satisfies the target resolution condition is:

[0040] in, This represents a medium-resolution reference satellite surface temperature image after histogram matching. This represents the original medium-resolution reference satellite surface temperature image. and These represent the standard deviations of the low-resolution target satellite surface temperature image and the original medium-resolution reference satellite surface temperature image, respectively. and These represent the average values ​​of the low-resolution target satellite surface temperature image and the original medium-resolution reference satellite surface temperature image, respectively.

[0041] Optionally, in one embodiment of the present invention, the data construction module includes: a cropping unit, configured to crop the multi-scale image library into at least one set of image blocks; a first generation unit, configured to generate a medium-to-high resolution dataset in the multi-scale training dataset based on the target satellite surface temperature image block that meets preset low-resolution conditions, the reference satellite surface temperature image block that meets preset medium-resolution conditions, the digital elevation model image block that meets preset medium-resolution conditions, the normalized vegetation index image block, the normalized water index image block, and the normalized building index image block in the at least one set of image blocks; and a second generation unit, configured to generate a high-resolution auxiliary parameter dataset in the multi-scale training dataset based on the digital elevation model image block, the normalized vegetation index image block, the normalized water index image block, and the normalized building index image block in the at least one set of image blocks that meets preset high-resolution conditions.

[0042] Optionally, in one embodiment of the present invention, the network construction module includes: a feature extraction unit, configured to extract preliminary features from land surface temperature satisfying a preset coarse resolution condition and auxiliary land surface parameters satisfying a preset fine resolution condition; a multi-scale feature extraction unit, configured to construct a multi-scale feature extraction module and use the multi-scale feature extraction module to extract multi-scale features of the preliminary features of the auxiliary land surface parameters satisfying the preset fine resolution condition; a first local feature extraction unit, configured to construct a lightweight CNN module and use the lightweight CNN module to extract local features from the land surface temperature satisfying the preset coarse resolution condition and the multi-scale features, and to perform channel compression on the local features to generate a compressed result; a first global feature extraction unit, configured to construct a lightweight Transformer module and use the lightweight Transformer module to extract features from the compressed result to generate global features; and a first reconstruction unit, configured to construct a downscaling reconstruction module and use the downscaling reconstruction module to process the global features to generate a downscaled land surface temperature.

[0043] Optionally, in one embodiment of the present invention, the formula for extracting the preliminary features is:

[0044]

[0045] in, and These represent the preliminary characteristics of the surface temperature that meets the preset coarse resolution condition and the surface parameter that meets the preset fine resolution condition, respectively. This represents the surface temperature that meets the preset coarse resolution conditions. This indicates a channel addition operation. This represents the convolution operation. , and , These are the weights and biases for the two convolution operations, respectively. For digital elevation models, Normalized Difference Vegetation Index (NDVI) The normalized water index, The normalized building index; The formula for constructing the multi-scale features is as follows:

[0046] in, The output feature map of the multi-scale features, This indicates the convolutional attention operation. Indicates size is × and × The feature map obtained by combining the features obtained from convolution. Indicates size is × and × The feature map obtained by combining the features obtained from convolution; The calculation formulas for the feature maps are as follows:

[0047]

[0048] in, This represents the ReLU activation function. , , , and , They are respectively the size of × , × , × The weights and biases in the convolution operation; The formulas for extracting local features and calculating channel compression are as follows:

[0049]

[0050]

[0051] in, and These are the local features obtained by the lightweight CNN module through surface temperature and multi-scale feature processing that meet the preset coarse resolution conditions. These are the channel-compressed feature maps of the outputs of two lightweight CNN architectures. This indicates a 2x upsampling operation. , and , These represent the weight and bias terms of the two lightweight CNN modules, respectively. and These are the weight and bias terms for the channel compression convolution operation, respectively. The calculation formula for the lightweight Transformer module is as follows:

[0052] in, This is the output feature map of the lightweight Transformer module. The outputs of the two lightweight CNN architectures are compressed feature maps across channels. The number of channel segments, Indicates the scaling factor. , , This represents the matrix obtained by linear projection of the input feature map; The formulas for calculating the matrix are as follows:

[0053]

[0054]

[0055] in, ,in For the segmentation operation; The formula for calculating the downscaled surface temperature is as follows:

[0056] in, The surface temperature output by the downscaling reconstruction module. This represents the surface temperature that meets the preset coarse resolution conditions. This indicates a 2x upsampling operation. , and , These are the weights and biases for single-channel and three-channel convolutions, respectively. Optionally, in one embodiment of the present invention, the network construction module includes: an information extraction unit, used to extract high-frequency information of land surface temperature from the downscaled land surface temperature using Laplacian high-pass filtering; a second local feature extraction unit, used to extract local features from the high-frequency information of land surface temperature using the lightweight CNN module; a second global feature extraction unit, used to extract global features from the local features using the lightweight Transformer module; and a second reconstruction unit, used to process the feature map based on the global features using single-channel convolution and to construct the domain transformation network guided by heterogeneous high-frequency information using residual connections.

[0057] Optionally, in one embodiment of the present invention, the formula for extracting high-frequency surface temperature information is:

[0058] in, This indicates the results of high-frequency information extraction of land surface temperature. This represents a Laplace high-pass filter, where the filter kernel is a matrix. , The surface temperature output by the downscaling reconstruction module; The matrix Defined as:

[0059] The formula for calculating the local features is:

[0060] in, This represents local features extracted from the high-frequency information of the land surface temperature. This refers to the lightweight CNN module. This represents local features extracted from the high-frequency information of the land surface temperature; The formula for calculating the global features is as follows:

[0061] in, This represents the global features extracted from the local features. This refers to the lightweight Transformer module. This represents local features extracted from the high-frequency information of the land surface temperature; The formula for reconstructing high-frequency information in the domain transform network is:

[0062] in, The high-frequency information prediction results of NDVI obtained after domain transformation are shown. and These represent the weight and bias terms of a single-channel convolution, respectively. This represents the global features extracted from the local features.

[0063] Optionally, in one embodiment of the present invention, the calculation formulas for the surface temperature downscaling network and the domain transformation network are respectively:

[0064]

[0065] in, and Let represent the loss functions of the surface temperature downscaling network and the domain transformation network, respectively. This represents the surface temperature fidelity loss under the preset coarse resolution conditions. This represents the frequency domain consistency loss that satisfies the preset fine-resolution condition. This represents the cross-modal frequency domain alignment loss. For neural network parameters, These are all empirical coefficients used to adjust the weights of each sub-loss function; The formula for calculating the surface temperature fidelity loss that meets the preset coarse resolution condition is as follows:

[0066] in, This indicates the number of training image pairs. Indicates the first Coarse-resolution surface temperature images from the image pair. Represents network 1, This indicates resampling to a coarse-resolution data scale; The formula for calculating the frequency domain consistency loss that satisfies the preset fine resolution condition is as follows:

[0067] in, This indicates the number of training image pairs. This indicates a Laplace high-pass filter. Indicates the first Reference fine-resolution surface temperature image in the image pair; The formula for calculating the cross-modal frequency domain alignment loss is as follows:

[0068] in, This refers to network 1. Represents network 2, Indicates the first For the fine-resolution NDVI image in the image pair, Indicates the first Coarse-resolution surface temperature images from the image pair.

[0069] This invention can construct a multi-scale dataset containing land surface temperature and multi-source auxiliary parameters using multi-source remote sensing data. At a low-resolution scale, a land surface temperature downscaling network is used to fuse heterogeneous features and downscale, obtaining a medium-resolution land surface temperature. High-frequency information is extracted from the medium-resolution result, and a domain transformation network is used to transform the high-frequency information of the heterogeneous data, enhancing the spatial detail representation. At a high-resolution scale, the network is fine-tuned to address the scale difference issue and applied to the medium-resolution downscaling result to obtain a high-resolution land surface temperature. This invention fully considers the downscaling differences between different resolutions, preserving the physical properties of the land surface thermal field and improving the processing efficiency of large-area remote sensing data. Therefore, it solves the problems of existing deep learning downscaling methods ignoring the local heterogeneity caused by the scale difference between high and low resolutions, which limits the accuracy of downscaling results.

[0070] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0071] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart of a multi-scale progressive surface temperature fusion downscaling method provided according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall process of a multi-scale progressive surface temperature fusion downscaling method according to an embodiment of the present invention. Figure 3 A schematic diagram of the network framework for a multi-scale progressive land surface temperature fusion and downscaling method according to an embodiment of the present invention; Figure 4 This is a schematic diagram showing the results of a multi-scale progressive surface temperature fusion downscaling method according to an embodiment of the present invention; Figure 5 This is a schematic diagram of a multi-scale progressive surface temperature fusion downscaling device provided according to an embodiment of the present invention.

[0072] Among them, 10 is a multi-scale progressive surface temperature fusion and downscaling device; 100 is an acquisition module; 200 is a preprocessing module; 300 is a data construction module; 400 is a network construction module; 500 is a training module; 600 is a generation module; and 700 is a downscaling processing module. Detailed Implementation

[0073] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0074] The following describes a multi-scale progressive land surface temperature fusion and downscaling method and apparatus according to embodiments of the present invention with reference to the accompanying drawings. Addressing the problem mentioned in the background art that existing deep learning downscaling methods ignore the local heterogeneity caused by scale differences between high and low resolutions, thus limiting the accuracy of downscaling results, the present invention provides a multi-scale progressive land surface temperature fusion and downscaling method. In this method, a multi-scale dataset containing land surface temperature and multi-source auxiliary parameters can be constructed using multi-source remote sensing data. At the low-resolution scale, a land surface temperature downscaling network is used to perform heterogeneous feature fusion downscaling to obtain a medium-resolution land surface temperature. High-frequency information is extracted from the medium-resolution result, and a domain transformation network is used to convert the high-frequency information of the heterogeneous data, enhancing the spatial detail representation. At the high-resolution scale, the network is fine-tuned to address the scale difference problem and applied to the medium-resolution downscaling result to obtain a high-resolution land surface temperature. The present invention fully considers the downscaling differences at different resolutions, achieving preservation of the physical properties of the land surface thermal field and improving the processing efficiency of large-area remote sensing data. Therefore, it solves the problem that existing deep learning downscaling methods ignore the local heterogeneity caused by scale differences between high and low resolutions, thus limiting the accuracy of downscaling results.

[0075] Specifically, Figure 1 This is a schematic flowchart of a multi-scale progressive surface temperature fusion downscaling method provided in an embodiment of the present invention.

[0076] like Figure 1 As shown, this multi-scale progressive surface temperature fusion downscaling method includes the following steps: In step S101, at least one of the following products is obtained: a surface temperature product of the target satellite that meets the preset low-resolution condition, a surface temperature product of the reference satellite that meets the preset medium-resolution condition, and an auxiliary parameter product of the reference satellite that meets the preset high-resolution condition.

[0077] In practical implementation, embodiments of this invention can acquire low-resolution land surface temperature products from the target satellite, medium-resolution land surface temperature products from the reference satellite, and high-resolution auxiliary parameter products, including Normalized Difference Vegetation Index (NDVI), Normalized Difference Water Index (NDWI), Normalized Difference Building Index (NDBI), and Digital Elevation Model (DEM). This invention can acquire Landsat 9 land surface temperature (100 m) and reflectance imagery (30 m), Landsat 7 land surface temperature imagery (60 m), and SRTM 1 DEM imagery (30 m), and calculate NDVI, NDWI, and NDBI using the Landsat 9 reflectance imagery.

[0078] This invention can overcome the limitations of downscaling single temperature data by fusing low-resolution satellite surface temperature with high-resolution multi-source heterogeneous surface parameters and combining the feature interaction mechanism of dual networks to fully explore the physical correlation and complementarity between parameters.

[0079] In step S102, at least one product is resampled to generate a multi-scale image library with a resolution corresponding to at least one product, and the land surface temperature image in the multi-scale image library is preprocessed to generate a preprocessed image.

[0080] In this embodiment of the invention, a multi-scale image library with low, medium and high resolution can be constructed based on resampling. Statistical characteristic corrections are performed on the surface temperature images of the reference satellite, the required surface parameters are calculated, and the preprocessed data is resampled to multiple scales to generate preprocessed images.

[0081] Optionally, in one embodiment of the present invention, preprocessing surface temperature images in a multi-scale image library to generate preprocessed images includes: calculating normalized vegetation index, normalized water index, and normalized building index using reflectance images of the target satellite; performing statistical characteristic correction on surface temperature data of a reference satellite to generate a surface temperature image of the reference satellite that meets the target resolution conditions; and preprocessing the surface temperature images of the target satellite, the surface temperature images of the reference satellite, and images of the digital elevation model, normalized vegetation index, normalized water index, and normalized building index to generate preprocessed images.

[0082] In actual implementation, such as Figure 2 As shown, embodiments of the present invention may include the following steps: (1) Calculate NDVI, NDWI, and NDBI using the reflectivity image of the target satellite.

[0083] (2) Histogram matching is used to perform statistical characteristic correction on the land surface temperature data of the reference satellite to reduce system bias. In one embodiment of the present invention, histogram matching can be used to perform statistical characteristic correction on the Landsat 7 land surface temperature image. The calculation formula for the land surface temperature image of the reference satellite that meets the target resolution condition is as follows:

[0084] in, This represents a medium-resolution reference satellite surface temperature image after histogram matching. This represents the original medium-resolution reference satellite surface temperature image. and These represent the standard deviations of the low-resolution target satellite surface temperature image and the original medium-resolution reference satellite surface temperature image, respectively. and These represent the average values ​​of the low-resolution target satellite surface temperature image and the original medium-resolution reference satellite surface temperature image, respectively.

[0085] (3) Preprocess the surface temperature images of the target satellite and the reference satellite, as well as the DEM, NDVI, NDWI and NDBI images, including georegistration, terrain correction and cropping, to obtain images covering the same area.

[0086] (4) Construct a multi-scale image library, resample the target satellite image obtained by the above processing to medium resolution, and resample the DEM, NDVI, NDWI and NDBI images to medium and high resolution, that is: the present invention can resample the Landsat 9 surface temperature obtained by the above processing to 120 m, and resample the DEM, NDVI, NDWI and NDBI to 60 m and 30 m.

[0087] In step S103, a multi-scale training dataset containing land surface temperature and at least one auxiliary parameter is constructed using the preprocessed images.

[0088] In actual implementation, embodiments of the present invention can use preprocessed images to construct a multi-scale training dataset containing land surface temperature and various auxiliary parameters, including a low-resolution land surface temperature-medium resolution auxiliary parameter training dataset (denoted as low-medium resolution dataset) and a high-resolution auxiliary parameter dataset.

[0089] Optionally, in one embodiment of the present invention, a multi-scale training dataset containing land surface temperature and at least one auxiliary parameter is constructed using preprocessed images, comprising: cropping the multi-scale image library into at least one set of image blocks; generating a medium-to-high resolution dataset in the multi-scale training dataset based on the target satellite land surface temperature image block satisfying a preset low-resolution condition, the reference satellite land surface temperature image block satisfying a preset medium-resolution condition, the digital elevation model image block satisfying a preset medium-resolution condition, the normalized vegetation index image block, the normalized water index image block, and the normalized building index image block in the at least one set of image blocks; and generating a high-resolution auxiliary parameter dataset in the multi-scale training dataset based on the digital elevation model image block, the normalized vegetation index image block, the normalized water index image block, and the normalized building index image block satisfying a preset high-resolution condition in the at least one set of image blocks.

[0090] Specifically, embodiments of the present invention can crop a multi-scale image library into multiple sets of image patches, and construct a multi-scale dataset based on these image patches. The low-resolution target satellite surface temperature image patch, the medium-resolution reference satellite surface temperature image patch, and the medium-resolution DEM, NDVI, NDWI, and NDBI image patches constitute a low-to-medium resolution dataset. The high-resolution DEM, NDVI, NDWI, and NDBI image patches constitute a high-resolution auxiliary parameter dataset. Specifically, using the multi-scale image library from the above steps, the 120 m Landsat 9 surface temperature image, the 60 m Landsat 7 surface temperature image, and the 60 m (30 m) DEM, NDVI, NDWI, and NDBI auxiliary data are cropped into image patches of 32×32 and 64×64 sizes covering the same ground features, respectively, to construct a 120 m surface temperature-60 m auxiliary parameter dataset (denoted as the 120 m-60 m dataset) and a 30 m auxiliary parameter dataset.

[0091] It should be noted that, since this invention relates to downscaling in two stages, low-to-medium resolution and medium-to-high resolution, the relative concepts of "coarse resolution" and "fine resolution" will be used in the following description to represent the lower and higher resolutions in the two downscaling stages, respectively.

[0092] In step S104, a surface temperature downscaling network based on multi-parameter fusion is constructed based on the multi-scale training dataset, and a domain transformation network guided by heterogeneous high-frequency information is constructed based on the multi-scale training dataset.

[0093] In actual implementation, such as Figure 3As shown, embodiments of the present invention can construct a surface temperature downscaling network based on multi-parameter fusion based on a multi-scale training dataset, and construct a domain transformation network guided by heterogeneous high-frequency information based on the multi-scale training dataset. The downscaling network and the domain transformation network are jointly trained using a CNN-Transformer hybrid architecture, which breaks through the limitations of a single model, takes into account both global feature association and local texture fidelity, and solves the problem of detail distortion caused by insufficient heterogeneous data fusion.

[0094] Optionally, in one embodiment of the present invention, constructing a land surface temperature downscaling network based on multi-parameter fusion according to a multi-scale training dataset includes: extracting preliminary features from land surface temperature satisfying a preset coarse resolution condition and auxiliary land surface parameters satisfying a preset fine resolution condition; constructing a multi-scale feature extraction module and using the multi-scale feature extraction module to extract multi-scale features of the preliminary features of the auxiliary land surface parameters satisfying the preset fine resolution condition; constructing a lightweight CNN module and using the lightweight CNN module to extract local features from the land surface temperature satisfying the preset coarse resolution condition and the multi-scale features, and compressing the local features to generate a compressed result; constructing a lightweight Transformer module and using the lightweight Transformer module to extract features from the compressed result to generate global features; and constructing a downscaling reconstruction module and using the downscaling reconstruction module to process the global features to generate a downscaled land surface temperature.

[0095] It is understood that the preset coarse resolution condition and the preset fine resolution condition in the embodiments of the present invention correspond to the lower resolution and higher resolution in the two downscaling stages, respectively.

[0096] Specifically, embodiments of the present invention may include the following steps: (1) Preliminary features are extracted from the input coarse-resolution surface temperature and fine-resolution auxiliary surface parameters using convolution, wherein, in one embodiment of the present invention, the formula for extracting the preliminary features is:

[0097]

[0098] in, and These represent the preliminary characteristics of surface temperature and surface parameters that meet the preset coarse resolution conditions, respectively. This represents the surface temperature that meets the preset coarse resolution conditions. This indicates a channel addition operation. This represents the convolution operation. , and , These are the weights and biases for the two convolution operations, respectively. For digital elevation models, Normalized Difference Vegetation Index (NDVI) The normalized water index, This is the normalized building index.

[0099] (2) Construct a multi-scale feature extraction module, which utilizes features of different sizes respectively. × , × , × The convolutional kernel extracts fine-resolution surface parameters at three scales; for example, this module utilizes kernels of different sizes. × , × , × The convolutional kernel extracts fine-resolution surface parameters at three scales, combines features from adjacent scales, and simultaneously considers channel and spatial correlations between cross-scale features. This module embeds a convolutional attention module to dynamically optimize the channel and spatial dimension weights of the feature maps. The multi-scale convolutional kernel size... , , The formula for constructing multi-scale features is:

[0100] in, The output feature map is a multi-scale feature map. This indicates the convolutional attention operation. Indicates size is × and × The feature map obtained by combining the features obtained from convolution. Indicates size is × and × The feature map obtained by combining the features obtained from convolution; and The calculation methods, i.e., the formulas for calculating the feature map, are as follows:

[0101]

[0102] in, This represents the ReLU activation function. , , , and , They are respectively the size of × , × , × The weights and biases in the convolution operation, for example... , , , and , The sizes are respectively × , × , × The weights and biases in the convolution operation.

[0103] (3) Construct a lightweight CNN module. High-frequency information from the image is extracted using a high-frequency filtering module, and local features are further extracted through residual feature blocks. This module effectively preserves image details while reducing the feature map size during convolution operations. The module also incorporates a convolutional attention module. The lightweight CNN module is used to extract local features from the input coarse-resolution surface temperature and the multi-scale features extracted in step (2), and these features are superimposed on the channels. Convolution operations are then applied to compress the channels to generate compressed convolution results. The formulas for extracting local features and calculating channel compression are as follows:

[0104]

[0105]

[0106] in, and These are the local features obtained by the lightweight CNN module through surface temperature and multi-scale feature processing, respectively, meeting the preset coarse resolution conditions. These are the channel-compressed feature maps of the outputs of two lightweight CNN architectures. This indicates a 2x upsampling operation. , and , These represent the weight and bias terms of the two lightweight CNN modules, respectively. and These are the weight and bias terms for the channel compression convolution operation, respectively.

[0107] (4) Construct a lightweight Transformer module. Two efficient Transformer modules are used to capture long-term dependencies in local regions with low computational cost. The efficient Transformer uses the encoder structure of the standard Transformer and utilizes an efficient multi-head attention mechanism to compute Q, K, and V in blocks, thereby improving computational efficiency. The lightweight Transformer module extracts features from the output of step (3) and fully mines global autocorrelation information. The calculation formula of the lightweight Transformer module is:

[0108] in, This is the output feature map of the lightweight Transformer module. These are the channel-compressed feature maps of the outputs of two lightweight CNN architectures. The number of channel segments, Indicates the scaling factor. , , This represents the matrix obtained by linear projection of the input feature map; The formulas for calculating matrices are as follows:

[0109]

[0110]

[0111] in, ,in For the segmentation operation, In the segmentation operation ; (5) Construct a downscaling reconstruction module. Use a three-channel convolution and a single-channel convolution to process the features output from step (4) to obtain the downscaled land surface temperature. Use residual connections to introduce the upsampling results of the coarse-resolution land surface temperature to prevent information loss. The formula for calculating the downscaled land surface temperature is:

[0112] in, The surface temperature output by the downscaling reconstruction module. This represents the surface temperature that meets the preset coarse resolution conditions. This indicates a 2x upsampling operation. , and , These are the weights and biases for single-channel and three-channel convolutions, respectively.

[0113] Optionally, in one embodiment of the present invention, a domain transformation network guided by heterogeneous high-frequency information is constructed based on a multi-scale training dataset, including: extracting high-frequency information of land surface temperature from the downscaled land surface temperature using Laplacian high-pass filtering; extracting local features from the high-frequency information of land surface temperature using a lightweight CNN module; extracting global features from the local features using a lightweight Transformer module; and, based on the global features, processing the feature map using single-channel convolution and employing residual connections to construct a domain transformation network guided by heterogeneous high-frequency information.

[0114] The construction of a domain transformation network guided by heterogeneous high-frequency information includes: (1) High-frequency information of the land surface temperature obtained from the downscaling reconstruction module is extracted using Laplace high-pass filtering. In one embodiment of the present invention, the formula for extracting high-frequency information of land surface temperature in the domain transform network is:

[0115] in, This indicates the results of high-frequency information extraction of land surface temperature. This represents a Laplace high-pass filter, where the filter kernel is a matrix. , The surface temperature output by the downscaling reconstruction module; matrix Defined as:

[0116] (2) Construct a lightweight CNN module to extract local features from the high-frequency surface temperature information obtained in step (1) above. This module can be represented as:

[0117] in, This indicates the local features extracted by the module from high-frequency information on land surface temperature. This indicates a lightweight CNN module. This represents local features extracted from the high-frequency information of the land surface temperature; (3) Construct a lightweight Transformer module to further extract global features from the local features extracted in step (1) above. This module can be represented as:

[0118] in, This represents the global features obtained by this module. This indicates a lightweight Transformer module. This represents local features extracted from the high-frequency information of the land surface temperature; (4) Construct a high-frequency information reconstruction module, using a single-channel convolution to process the feature map obtained in step (2) above, and using residual connections to reduce information loss. The high-frequency information reconstruction formula in the domain transform network is:

[0119] in, The high-frequency information prediction results of NDVI obtained after domain transformation are shown. and These represent the weight and bias terms of a single-channel convolution, respectively. This represents the global features extracted from the local features.

[0120] The embodiments of the present invention can use a CNN-Transformer hybrid architecture to construct a downscaling network and a domain transformation network for joint training, which breaks through the limitations of a single model, takes into account both global feature association and local texture fidelity, and solves the problem of detail distortion caused by insufficient fusion of heterogeneous data.

[0121] In step S105, based on the progressive dual-network joint training strategy, the surface temperature downscaling network and the domain transformation network are trained using the training dataset that meets the preset low-to-medium resolution conditions in the multi-scale training dataset, so as to generate a downscaling model that meets the preset low-to-medium resolution conditions and a downscaling result that meets the preset medium resolution conditions.

[0122] In practical implementation, embodiments of the present invention can employ a progressive dual-network joint training strategy, constructing a multi-scale cross-modal loss function to jointly train the downscaling network and the domain transformation network. The low-to-medium resolution training dataset constructed in step S103 is used to train the surface temperature downscaling network and the domain transformation network constructed in step S104 until convergence, yielding a low-to-medium resolution downscaling model and a medium resolution downscaling result. For example, the 120 m - 60 m dataset constructed in step S103 is used to train the network constructed in step S104 until convergence, yielding a 120 m - 60 m downscaling model and a 60 m downscaling result. The 60 m downscaling result, combined with the 30 m auxiliary parameter dataset constructed in step S103, is used to fine-tune the 120 m - 60 m model, yielding a 60 m - 30 m downscaling model and a 30 m downscaling result.

[0123] (1) Specifically, a multi-scale cross-modal loss function is constructed to jointly train the downscaling network and the domain transformation network. The downscaling network adopts a multi-scale cross-modal loss function, which consists of three parts: a coarse-resolution surface temperature fidelity loss to maintain the accuracy of the surface temperature background information, a fine-resolution frequency domain consistency loss to maintain the richness of spatial details during downscaling, and a cross-modal frequency domain alignment loss to further enhance the ability to express spatial details. The cross-modal frequency domain alignment loss is also the loss function of the domain transformation network. The loss functions of the downscaling network and the domain transformation network are each defined in the following form, wherein, in one embodiment of the present invention, the calculation formulas of the surface temperature downscaling network and the domain transformation network are respectively:

[0124]

[0125] in, and Let represent the loss functions of the surface temperature downscaling network and the domain transformation network, respectively. This represents the surface temperature fidelity loss under the preset coarse resolution conditions. This represents the frequency domain consistency loss that satisfies the preset fine-resolution condition. This represents the cross-modal frequency domain alignment loss. For neural network parameters, These are all empirical coefficients used to adjust the weights of each sub-loss function. The values ​​were set to 0.7, 0.3, and 1 for low-to-medium resolution downscaling, and to 0.7, 0, and 1 for medium-to-high resolution downscaling. The formula for calculating the surface temperature fidelity loss under the preset coarse resolution condition is as follows:

[0126] in, This indicates the number of training image pairs. Indicates the first Coarse-resolution surface temperature images from the image pair. Represents network 1, This indicates resampling to a coarse-resolution data scale; The formula for calculating the frequency domain consistency loss that satisfies the preset fine-resolution condition is as follows:

[0127] in, This indicates the number of training image pairs. This indicates a Laplace high-pass filter. Indicates the first Reference fine-resolution surface temperature image in the image pair; The formula for calculating the cross-modal frequency domain alignment loss is:

[0128] in, This refers to network 1. Represents network 2, Indicates the first For the fine-resolution NDVI image in the image pair, Indicates the first Coarse-resolution surface temperature images from the image pair.

[0129] (2) Using the low-to-medium resolution training dataset and multi-scale cross-modal loss function constructed in the above steps, the downscaling and domain transformation network are jointly trained until convergence, and the trained low-to-medium resolution downscaling model is obtained. The medium resolution downscaling result can be obtained from this model.

[0130] The embodiments of the present invention construct a resolution-adaptive progressive processing link through a phased progressive training strategy, avoiding model transfer failure caused by differences in data distribution at different resolutions, so that the algorithm can maintain downscaling accuracy in multi-resolution scenarios, and is especially suitable for large-area multi-scale remote sensing data analysis.

[0131] In step S106, the downscaling model that satisfies the preset low-to-medium resolution condition is fine-tuned using the dataset that satisfies the preset medium-to-high resolution condition in the multi-scale training dataset, so as to generate a downscaling model that satisfies the preset medium-to-high resolution condition and a downscaling result that satisfies the preset high resolution condition.

[0132] In actual implementation, the embodiments of the present invention can use medium-high resolution datasets to fine-tune the low-medium resolution downscaling model to obtain medium-high resolution downscaling model and high resolution downscaling results. That is, the embodiments of the present invention can use the medium resolution downscaling results obtained in the above steps combined with the high resolution auxiliary parameter dataset constructed in the above steps to fine-tune the low-medium resolution downscaling model obtained in step (2) to obtain medium-high resolution downscaling model, so as to avoid the scale inconsistency problem caused by direct model transfer, and thus obtain high resolution downscaling results.

[0133] In this embodiment of the invention, the 120 m - 60 m training dataset constructed in step S103 is used to jointly train the downscaling and domain transformation network constructed in step S104 until convergence using a multi-scale cross-modal loss function, resulting in a trained 120 m - 60 m downscaling model. This model can then yield a 60 m downscaling result. The 60 m downscaling result is then combined with a 30 m auxiliary parameter dataset to fine-tune the 120 m - 60 m downscaling model, resulting in a 60 m - 30 m downscaling model. This avoids the scaling inconsistency problem caused by direct model transfer, thus obtaining a 30 m downscaling result.

[0134] In step S107, based on the downscaling model that satisfies the preset low-to-medium resolution conditions, the downscaling result that satisfies the preset medium resolution conditions, the downscaling model that satisfies the preset medium-to-high resolution conditions, and the downscaling result that satisfies the preset high resolution conditions, the low-resolution land surface temperature image to be processed is downscaled in a low-to-medium-to-high progressive manner to generate a multi-scale progressive land surface temperature that satisfies the target high resolution conditions.

[0135] In practical implementation, embodiments of the present invention can utilize the downscaling network trained and fine-tuned in step S106 to perform low-to-medium-to-high progressive downscaling on the low-resolution land surface temperature image to obtain high-resolution land surface temperature. Embodiments of the present invention can also utilize the low-to-medium resolution downscaling model trained in step S106 to downscale the low-resolution land surface temperature image to obtain medium-resolution land surface temperature, and then use the medium-to-high resolution downscaling model to downscale the medium-resolution land surface temperature to obtain high-resolution land surface temperature.

[0136] Among them, such as Figure 4 As shown, in this embodiment of the invention, the 120 m-60 m downscaling model trained in steps S105 and S106 can be used to downscale the 120 m surface temperature image to be processed to obtain a 60 m surface temperature, and then the 60 m-30 m downscaling model can be used to downscale the 60 m surface temperature to obtain a 30 m surface temperature.

[0137] This invention provides a progressive unsupervised downscaling method for multi-scale land surface temperature based on heterogeneous high-frequency constraints to obtain high-resolution land surface temperature images that reflect the true thermal state of the land surface. It also develops an unsupervised downscaling model for the original high-resolution scale to fit the nonlinear relationship between land surface temperature and auxiliary parameters, and uses high-frequency information of high-resolution heterogeneous auxiliary parameters as constraints to achieve fine downscaling.

[0138] The multi-scale progressive land surface temperature fusion and downscaling method proposed in this invention can construct a multi-scale dataset containing land surface temperature and multi-source auxiliary parameters using multi-source remote sensing data. At a low-resolution scale, a land surface temperature downscaling network is used to fuse and downscale heterogeneous features to obtain a medium-resolution land surface temperature. High-frequency information is extracted from the medium-resolution result, and a domain transformation network is used to transform the high-frequency information of the heterogeneous data to enhance the spatial detail representation. At a high-resolution scale, the network is fine-tuned to address the scale difference problem and applied to the medium-resolution downscaling result to obtain a high-resolution land surface temperature. This invention fully considers the downscaling differences between different resolutions, preserving the physical properties of the land surface thermal field and improving the processing efficiency of large-area remote sensing data. Therefore, it solves the problem that existing deep learning downscaling methods ignore the local heterogeneity caused by the scale difference between high and low resolutions, which limits the accuracy of the downscaling results.

[0139] Next, referring to the accompanying drawings, a multi-scale progressive surface temperature fusion downscaling device proposed according to an embodiment of the present invention is described.

[0140] Figure 5 This is a schematic diagram of the structure of the multi-scale progressive surface temperature fusion downscaling device according to an embodiment of the present invention.

[0141] like Figure 5 As shown, the multi-scale progressive surface temperature fusion downscaling device 10 includes: an acquisition module 100, a preprocessing module 200, a data construction module 300, a network construction module 400, a training module 500, a generation module 600, and a downscaling processing module 700.

[0142] Specifically, the acquisition module 100 is used to acquire at least one of the following products: a surface temperature product of the target satellite that meets the preset low-resolution conditions, a surface temperature product of the reference satellite that meets the preset medium-resolution conditions, and an auxiliary parameter product of the reference satellite that meets the preset high-resolution conditions.

[0143] The preprocessing module 200 is used to resample at least one product to generate a multi-scale image library with a resolution corresponding to at least one product, and preprocess the surface temperature images in the multi-scale image library to generate preprocessed images.

[0144] Data construction module 300 is used to construct a multi-scale training dataset containing land surface temperature and at least one auxiliary parameter using preprocessed images.

[0145] The network construction module 400 is used to construct a surface temperature downscaling network based on multi-parameter fusion based on a multi-scale training dataset, and to construct a domain transformation network guided by heterogeneous high-frequency information based on a multi-scale training dataset.

[0146] Training module 500 is used to train the land surface temperature downscaling network and the domain transformation network based on a progressive dual-network joint training strategy. It utilizes a multi-scale training dataset that meets the preset low-to-medium resolution conditions to generate a downscaling model that meets the preset low-to-medium resolution conditions and a downscaling result that meets the preset medium resolution conditions.

[0147] The generation module 600 is used to fine-tune the downscaling model that meets the preset low-to-medium resolution conditions using the dataset that meets the preset medium-to-high resolution conditions in the multi-scale training dataset, so as to generate a downscaling model that meets the preset medium-to-high resolution conditions and a downscaling result that meets the preset high resolution conditions.

[0148] The downscaling module 700 is used to perform low-to-medium-to-high progressive downscaling on the low-resolution land temperature image to be processed based on the downscaling model that meets the preset low-to-medium resolution conditions, the downscaling result that meets the preset medium resolution conditions, the downscaling model that meets the preset medium-to-high resolution conditions, and the downscaling result that meets the preset high resolution conditions, so as to generate a multi-scale progressive land temperature that meets the target high resolution conditions.

[0149] Optionally, in one embodiment of the present invention, the preprocessing module 200 includes: a calculation unit, a correction unit, and a preprocessing unit.

[0150] The calculation unit is used to calculate the normalized vegetation index, normalized water index, and normalized building index using the reflectivity image of the target satellite.

[0151] The correction unit is used to perform statistical characteristic correction on the surface temperature data of the reference satellite in order to generate a surface temperature image of the reference satellite that meets the target resolution requirements.

[0152] The preprocessing unit is used to preprocess the surface temperature images of the target satellite, the surface temperature images of the reference satellite, and the images of the digital elevation model, normalized vegetation index, normalized water index, and normalized building index to generate preprocessed images.

[0153] Optionally, in one embodiment of the present invention, the formula for calculating the surface temperature image of the reference satellite that satisfies the target resolution condition is:

[0154] in, This represents a medium-resolution reference satellite surface temperature image after histogram matching. This represents the original medium-resolution reference satellite surface temperature image. and These represent the standard deviations of the low-resolution target satellite surface temperature image and the original medium-resolution reference satellite surface temperature image, respectively. and These represent the average values ​​of the low-resolution target satellite surface temperature image and the original medium-resolution reference satellite surface temperature image, respectively.

[0155] Optionally, in one embodiment of the present invention, the data construction module 300 includes: a trimming unit, a first generation unit, and a second generation unit.

[0156] The cropping unit is used to crop the multi-scale image library into at least one set of image blocks.

[0157] The first generation unit is used to generate a medium-to-high resolution dataset from a multi-scale training dataset based on at least one set of image blocks, including a target satellite surface temperature image block that meets preset low-resolution conditions, a reference satellite surface temperature image block that meets preset medium-resolution conditions, a digital elevation model image block that meets preset medium-resolution conditions, a normalized vegetation index image block, a normalized water index image block, and a normalized building index image block.

[0158] The second generation unit is used to generate a high-resolution auxiliary parameter dataset in the multi-scale training dataset based on at least one set of image blocks, including digital elevation model image blocks, normalized vegetation index image blocks, normalized water index image blocks, and normalized building index image blocks that meet preset high-resolution conditions.

[0159] Optionally, in one embodiment of the present invention, the network construction module 400 includes: a feature extraction unit, a multi-scale feature extraction unit, a first local feature extraction unit, a first global feature extraction unit, and a first reconstruction unit.

[0160] The feature extraction unit is used to extract preliminary features from surface temperature that meets preset coarse resolution conditions and auxiliary surface parameters that meet preset fine resolution conditions.

[0161] The multi-scale feature extraction unit is used to construct the multi-scale feature extraction module and use the multi-scale feature extraction module to extract the preliminary features of auxiliary surface parameters that meet the preset fine resolution conditions.

[0162] The first local feature extraction unit is used to construct a lightweight CNN module, and use the lightweight CNN module to extract local features from the surface temperature and multi-scale features that meet the preset coarse resolution conditions, and compress the local features to generate compressed results.

[0163] The first global feature extraction unit is used to build a lightweight Transformer module, which extracts features from the compressed results to generate global features.

[0164] The first reconstruction unit is used to construct a downscale reconstruction module and use the downscale reconstruction module to process global features to generate downscaled surface temperatures.

[0165] Optionally, in one embodiment of the present invention, the formula for extracting the preliminary features is:

[0166]

[0167] in, and These represent the preliminary characteristics of the surface temperature that meets the preset coarse resolution condition and the surface parameter that meets the preset fine resolution condition, respectively. This represents the surface temperature that meets the preset coarse resolution conditions. This indicates a channel addition operation. This represents the convolution operation. , and , These are the weights and biases for the two convolution operations, respectively. For digital elevation models, Normalized Difference Vegetation Index (NDVI) The normalized water index, The normalized building index; The formula for constructing the multi-scale features is as follows:

[0168] in, The output feature map of the multi-scale features, This indicates the convolutional attention operation. Indicates size is × and × The feature map obtained by combining the features obtained from convolution. Indicates size is × and × The feature map obtained by combining the features obtained from convolution; The calculation formulas for the feature maps are as follows:

[0169]

[0170] in, This represents the ReLU activation function. , , , and , They are respectively the size of × , × , × The weights and biases in the convolution operation; The formulas for extracting local features and calculating channel compression are as follows:

[0171]

[0172]

[0173] in, and These are the local features obtained by the lightweight CNN module through surface temperature and multi-scale feature processing that meet the preset coarse resolution conditions. These are the channel-compressed feature maps of the outputs of two lightweight CNN architectures. This indicates a 2x upsampling operation. , and , These represent the weight and bias terms of the two lightweight CNN modules, respectively. and These are the weight and bias terms for the channel compression convolution operation, respectively. The calculation formula for the lightweight Transformer module is as follows:

[0174] in, This is the output feature map of the lightweight Transformer module. The outputs of the two lightweight CNN architectures are compressed feature maps across channels. The number of channel segments, Indicates the scaling factor. , , This represents the matrix obtained by linear projection of the input feature map; The formulas for calculating the matrix are as follows:

[0175]

[0176]

[0177] in, ,in For the segmentation operation; The formula for calculating the downscaled surface temperature is as follows:

[0178] in, The surface temperature output by the downscaling reconstruction module. This represents the surface temperature that meets the preset coarse resolution conditions. This indicates a 2x upsampling operation. , and , These are the weights and biases for single-channel and three-channel convolutions, respectively. Optionally, in one embodiment of the present invention, the network construction module 400 includes: an information extraction unit, a local feature extraction unit, a global feature extraction unit, and a processing unit.

[0179] The information extraction unit is used to extract high-frequency surface temperature information from the downscaled surface temperature using Laplace high-pass filtering.

[0180] The second local feature extraction unit is used to extract local features from high-frequency surface temperature information using a lightweight CNN module.

[0181] The second global feature extraction unit is used to extract global features from local features using the lightweight Transformer module.

[0182] The second reconstruction unit is used to process the feature map based on global features using single-channel convolution and to construct a domain transformation network guided by heterogeneous high-frequency information using residual connections.

[0183] Optionally, in one embodiment of the present invention, the formula for extracting high-frequency surface temperature information is:

[0184] in, This indicates the results of high-frequency information extraction of land surface temperature. This represents a Laplace high-pass filter, where the filter kernel is a matrix. , The surface temperature output by the downscaling reconstruction module; The matrix Defined as:

[0185] The formula for calculating the local features is:

[0186] in, This represents local features extracted from the high-frequency information of the land surface temperature. This refers to the lightweight CNN module. This represents local features extracted from the high-frequency information of the land surface temperature; The formula for calculating the global features is as follows:

[0187] in, This represents the global features extracted from the local features. This refers to the lightweight Transformer module. This represents local features extracted from the high-frequency information of the land surface temperature; The formula for reconstructing high-frequency information in the domain transform network is:

[0188] in, The high-frequency information prediction results of NDVI obtained after domain transformation are shown. and These represent the weight and bias terms of a single-channel convolution, respectively. This represents the global features extracted from the local features.

[0189] Optionally, in one embodiment of the present invention, the calculation formulas for the surface temperature downscaling network and the domain transformation network are respectively:

[0190]

[0191] in, and Let represent the loss functions of the surface temperature downscaling network and the domain transformation network, respectively. This represents the surface temperature fidelity loss under the preset coarse resolution conditions. This represents the frequency domain consistency loss that satisfies the preset fine-resolution condition. This represents the cross-modal frequency domain alignment loss. For neural network parameters, These are all empirical coefficients used to adjust the weights of each sub-loss function; The formula for calculating the surface temperature fidelity loss that meets the preset coarse resolution condition is as follows:

[0192] in, This indicates the number of training image pairs. Indicates the first Coarse-resolution surface temperature images from the image pair. Represents network 1, This indicates resampling to a coarse-resolution data scale; The formula for calculating the frequency domain consistency loss that satisfies the preset fine resolution condition is as follows:

[0193] in, This indicates the number of training image pairs. This indicates a Laplace high-pass filter. Indicates the first Reference fine-resolution surface temperature image in the image pair; The formula for calculating the cross-modal frequency domain alignment loss is as follows:

[0194] in, This refers to network 1. Represents network 2, Indicates the first For the fine-resolution NDVI image in the image pair, Indicates the first Coarse-resolution surface temperature images from the image pair.

[0195] It should be noted that the foregoing explanation of the embodiment of the multi-scale progressive land surface temperature fusion downscaling method also applies to the multi-scale progressive land surface temperature fusion downscaling device of this embodiment, and will not be repeated here.

[0196] The multi-scale progressive land surface temperature fusion and downscaling device proposed in this invention can construct a multi-scale dataset containing land surface temperature and multi-source auxiliary parameters using multi-source remote sensing data. At a low-resolution scale, a land surface temperature downscaling network is used to perform heterogeneous feature fusion and downscaling to obtain a medium-resolution land surface temperature. High-frequency information is extracted from the medium-resolution result, and a domain transformation network is used to transform the high-frequency information of the heterogeneous data to enhance the spatial detail representation. At a high-resolution scale, the network is fine-tuned to address the scale difference problem and applied to the medium-resolution downscaling result to obtain a high-resolution land surface temperature. This invention fully considers the downscaling differences at different resolutions, preserving the physical properties of the land surface thermal field and improving the processing efficiency of large-area remote sensing data. Therefore, it solves the problem that existing deep learning downscaling methods ignore the local heterogeneity caused by the scale difference between high and low resolutions, which limits the accuracy of the downscaling results.

[0197] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0198] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0199] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

Claims

1. A multi-scale progressive surface temperature fusion downscaling method, characterized in that, Includes the following steps: Acquire at least one of the following: a surface temperature product of the target satellite that meets preset low-resolution conditions, a surface temperature product of the reference satellite that meets preset medium-resolution conditions, and an auxiliary parameter product of the reference satellite that meets preset high-resolution conditions. The at least one product is resampled to generate a multi-scale image library corresponding to the resolution of the at least one product, and the land surface temperature images in the multi-scale image library are preprocessed to generate preprocessed images. A multi-scale training dataset containing land surface temperature and at least one auxiliary parameter is constructed using the preprocessed images; A surface temperature downscaling network based on multi-parameter fusion is constructed based on the multi-scale training dataset, and a domain transformation network guided by heterogeneous high-frequency information is constructed based on the multi-scale training dataset. Based on the progressive dual-network joint training strategy, the surface temperature downscaling network and the domain transformation network are trained using the training dataset in the multi-scale training dataset that meets the preset low-to-medium resolution conditions, so as to generate a downscaling model that meets the preset low-to-medium resolution conditions and a downscaling result that meets the preset medium resolution conditions. The downscaling model that satisfies the preset low-to-medium resolution condition is fine-tuned using the dataset that satisfies the preset medium-to-high resolution condition in the multi-scale training dataset, so as to generate a downscaling model that satisfies the preset medium-to-high resolution condition and a downscaling result that satisfies the preset high resolution condition. Based on the downscaling model that satisfies the preset low-to-medium resolution conditions, the downscaling result that satisfies the preset medium resolution conditions, the downscaling model that satisfies the preset medium-to-high resolution conditions, and the downscaling result that satisfies the preset high resolution conditions, the low-resolution land surface temperature image to be processed is subjected to low-to-medium-to-high progressive downscaling to generate a multi-scale progressive land surface temperature that satisfies the target high resolution conditions.

2. The method according to claim 1, characterized in that, The preprocessing of the surface temperature images in the multi-scale image library to generate preprocessed images includes: The normalized vegetation index, normalized water index, and normalized building index are calculated using the reflectivity imagery of the target satellite. Statistical characteristic corrections are performed on the surface temperature data of the reference satellite to generate a surface temperature image of the reference satellite that meets the target resolution requirements. The surface temperature images of the target satellite, the surface temperature images and digital elevation models of the reference satellite, the normalized vegetation index, the normalized water index, and the normalized building index are preprocessed to generate the preprocessed images.

3. The method according to claim 2, characterized in that, The formula for calculating the surface temperature image of the reference satellite that meets the target resolution condition is as follows: in, This represents a medium-resolution reference satellite surface temperature image after histogram matching. This represents the original medium-resolution reference satellite surface temperature image. and These represent the standard deviations of the low-resolution target satellite surface temperature image and the original medium-resolution reference satellite surface temperature image, respectively. and These represent the average values ​​of the low-resolution target satellite surface temperature image and the original medium-resolution reference satellite surface temperature image, respectively.

4. The method according to claim 2, characterized in that, The construction of a multi-scale training dataset containing land surface temperature and at least one auxiliary parameter using the preprocessed images includes: The multi-scale image library is cropped into at least one set of image blocks; Based on the target satellite surface temperature image block that meets the preset low resolution condition, the reference satellite surface temperature image block that meets the preset medium resolution condition, the digital elevation model image block that meets the preset medium resolution condition, the normalized vegetation index image block, the normalized water index image block, and the normalized building index image block in the at least one set of image blocks, a medium-to-high resolution dataset is generated in the multi-scale training dataset. Based on the digital elevation model image blocks, normalized vegetation index image blocks, normalized water index image blocks, and normalized building index image blocks that meet the preset high resolution conditions in the at least one set of image blocks, a high-resolution auxiliary parameter dataset is generated in the multi-scale training dataset.

5. The method according to claim 1, characterized in that, The construction of a surface temperature downscaling network based on multi-parameter fusion using the multi-scale training dataset includes: Preliminary features are extracted from surface temperature that meets the preset coarse resolution conditions and auxiliary surface parameters that meet the preset fine resolution conditions. A multi-scale feature extraction module is constructed, and the multi-scale features of the preliminary features of the auxiliary surface parameters that meet the preset fine resolution conditions are extracted using the multi-scale feature extraction module. A lightweight CNN module is constructed, and the lightweight CNN module is used to extract the local features of the surface temperature and the multi-scale features that meet the preset coarse resolution conditions. The local features are then compressed to generate a compressed result. A lightweight Transformer module is constructed, and the lightweight Transformer module is used to extract features from the compression result to generate global features; A downscaling reconstruction module is constructed, and the global features are processed using the downscaling reconstruction module to generate the downscaled surface temperature.

6. The method according to claim 5, characterized in that, The formula for extracting the preliminary features is as follows: in, and These represent the preliminary characteristics of the surface temperature that meets the preset coarse resolution condition and the surface parameter that meets the preset fine resolution condition, respectively. This represents the surface temperature that meets the preset coarse resolution conditions. This indicates a channel addition operation. This represents the convolution operation. , and , These are the weights and biases for the two convolution operations, respectively. For digital elevation models, Normalized Difference Vegetation Index (NDVI) The normalized water index, The normalized building index; The formula for constructing the multi-scale features is as follows: in, The output feature map of the multi-scale features, This indicates the convolutional attention operation. Indicates size is × and × The feature map obtained by combining the features obtained from convolution. Indicates size is × and × The feature map obtained by combining the features obtained from convolution; The calculation formulas for the feature maps are as follows: in, This represents the ReLU activation function. , , , and , They are respectively the size of × , × , × The weights and biases in the convolution operation; The formulas for extracting local features and calculating channel compression are as follows: in, and These are the local features obtained by the lightweight CNN module through surface temperature and multi-scale feature processing that meet the preset coarse resolution conditions. These are the channel-compressed feature maps of the outputs of two lightweight CNN architectures. This indicates a 2x upsampling operation. , and , These represent the weight and bias terms of the two lightweight CNN modules, respectively. and These are the weight and bias terms for the channel compression convolution operation, respectively. The calculation formula for the lightweight Transformer module is as follows: in, This is the output feature map of the lightweight Transformer module. The outputs of the two lightweight CNN architectures are compressed feature maps across channels. The number of channel segments, Indicates the scaling factor. , , This represents the matrix obtained by linear projection of the input feature map; The formulas for calculating the matrix are as follows: in, ,in For the segmentation operation; The formula for calculating the downscaled surface temperature is as follows: in, The surface temperature output by the downscaling reconstruction module. This represents the surface temperature that meets the preset coarse resolution conditions. This indicates a 2x upsampling operation. , and , These are the weights and biases for single-channel and three-channel convolutions, respectively.

7. The method according to claim 6, characterized in that, The construction of a domain transformation network guided by heterogeneous high-frequency information based on the multi-scale training dataset includes: High-frequency information of land surface temperature is extracted from the downscaled land surface temperature using Laplace high-pass filtering. The lightweight CNN module is used to extract local features from the high-frequency information of the land surface temperature. The lightweight Transformer module is used to extract global features from the local features; Based on the global features, the feature map is processed using single-channel convolution, and residual connections are used to construct the domain transformation network guided by heterogeneous high-frequency information.

8. The method according to claim 7, characterized in that, The formula for extracting high-frequency surface temperature information is as follows: in, This indicates the results of high-frequency information extraction of land surface temperature. This represents a Laplace high-pass filter, where the filter kernel is a matrix. , The surface temperature output by the downscaling reconstruction module; The matrix Defined as: The formula for calculating the local features is: in, This represents local features extracted from the high-frequency information of the land surface temperature. This refers to the lightweight CNN module. This represents local features extracted from the high-frequency information of the land surface temperature; The formula for calculating the global features is as follows: in, This represents the global features extracted from the local features. This refers to the lightweight Transformer module. This represents local features extracted from the high-frequency information of the land surface temperature; The formula for reconstructing high-frequency information in the domain transform network is: in, The high-frequency information prediction results of NDVI obtained after domain transformation are shown. and These represent the weight and bias terms of a single-channel convolution, respectively. This represents the global features extracted from the local features.

9. The method according to claim 1, characterized in that, The calculation formulas for the surface temperature downscaling network and the domain transformation network are as follows: in, and Let represent the loss functions of the surface temperature downscaling network and the domain transformation network, respectively. This represents the surface temperature fidelity loss under the preset coarse resolution conditions. This represents the frequency domain consistency loss that satisfies the preset fine-resolution condition. This represents the cross-modal frequency domain alignment loss. For neural network parameters, These are all empirical coefficients used to adjust the weights of each sub-loss function; The formula for calculating the surface temperature fidelity loss that meets the preset coarse resolution condition is as follows: in, This indicates the number of training image pairs. Indicates the first Coarse-resolution surface temperature images from the image pair. Represents network 1, This indicates resampling to a coarse-resolution data scale; The formula for calculating the frequency domain consistency loss that satisfies the preset fine resolution condition is as follows: in, This indicates the number of training image pairs. This indicates a Laplace high-pass filter. Indicates the first Reference fine-resolution surface temperature image in the image pair; The formula for calculating the cross-modal frequency domain alignment loss is as follows: in, This refers to network 1. Represents network 2, Indicates the first For the fine-resolution NDVI image in the image pair, Indicates the first Coarse-resolution surface temperature images from the image pair.

10. A multi-scale progressive surface temperature fusion downscaling device, characterized in that, include: The acquisition module is used to acquire at least one of the following products: a surface temperature product of the target satellite that meets the preset low-resolution conditions, a surface temperature product of the reference satellite that meets the preset medium-resolution conditions, and an auxiliary parameter product of the reference satellite that meets the preset high-resolution conditions. A preprocessing module is used to resample the at least one product to generate a multi-scale image library corresponding to the resolution of the at least one product, and preprocess the land surface temperature images in the multi-scale image library to generate preprocessed images. The data construction module is used to construct a multi-scale training dataset containing land surface temperature and at least one auxiliary parameter using the preprocessed images; The network construction module is used to construct a surface temperature downscaling network based on multi-parameter fusion based on the multi-scale training dataset, and to construct a domain transformation network guided by heterogeneous high-frequency information based on the multi-scale training dataset. The training module is used to train the land surface temperature downscaling network and the domain transformation network based on a progressive dual-network joint training strategy, using the training dataset in the multi-scale training dataset that meets the preset low-to-medium resolution conditions, so as to generate a downscaling model that meets the preset low-to-medium resolution conditions and a downscaling result that meets the preset medium resolution conditions. The generation module is used to fine-tune the downscaling model that satisfies the preset low-to-medium resolution condition using the dataset that satisfies the preset medium-to-high resolution condition in the multi-scale training dataset, so as to generate a downscaling model that satisfies the preset medium-to-high resolution condition and a downscaling result that satisfies the preset high resolution condition. The downscaling module is used to perform low-to-medium-to-high progressive downscaling on the low-resolution land temperature image to be processed based on the downscaling model that meets the preset low-to-medium resolution conditions, the downscaling result that meets the preset medium resolution conditions, the downscaling model that meets the preset medium-to-high resolution conditions, and the downscaling result that meets the preset high resolution conditions, so as to generate a multi-scale progressive land temperature that meets the target high resolution conditions.

Images