A method for fusion of geostationary satellite and polar-orbiting satellite data based on deep learning

Abstract

The present invention provides a method for fusing geostationary satellite and polar-orbiting satellite data based on deep learning, which relates to satellite data fusion technology, specifically including: downloading satellite data, satellite data and geographic information data; calibrating and correcting the original data; determining the polar-orbiting satellite sensor channel closest to each geostationary satellite sensor channel, and judging the similarity between the geostationary satellite and the polar-orbiting satellite's closest sensor channels using the spectral angle according to the spectral response function; unifying the geostationary satellite and polar-orbiting satellite data into a longitude and latitude grid; using the method of minimizing the probability distribution distance to perform image registration; constructing a Swin Transformer model and forming an intelligent downscaling model; obtaining spectral channel data with high temporal and spatial resolution, and forming high temporal and spatial resolution multi-spectral channel data covering the entire geostationary satellite scanning domain. The technical solution of the present invention overcomes the problem that the prior art cannot conduct comprehensive and accurate earth observation.

Description

Technical Field

[0001] The present invention relates to the technical field of satellite data fusion, and in particular to a method for fusing geostationary satellite and polar-orbiting satellite data based on deep learning. Background Art

[0002] Satellite remote sensing technology plays an irreplaceable role in earth observation and environmental monitoring. However, single satellite data has limitations in spatial and temporal resolution, which makes it difficult to meet the needs of modern scientific research and practical applications.

[0003] Geostationary satellites maintain a fixed position relative to the Earth's surface and are able to continuously monitor the same area, providing continuous, uninterrupted data, which is essential for weather forecasting, meteorological monitoring, and disaster warning. The high temporal resolution of geostationary satellites allows for frequent updates of images and data, capturing rapidly changing phenomena in real time. Their wide coverage and wide field of view allow a single satellite to monitor large areas of the Earth's surface, especially those near the equator. Although geostationary satellites have the advantages of high temporal resolution and wide coverage, they also have significant disadvantages. First, due to their synchronous orbits located approximately 36,000 kilometers above the Earth, geostationary satellites generally have low spatial resolution and cannot provide high-detail images. In particular, geostationary satellites directly overlook the equatorial region and have low observation angles for high-latitude regions, resulting in poor image and data quality, and the observed images have problems of geometric distortion and severe resolution reduction. In contrast, polar-orbiting satellites operate along near-polar orbits, covering different areas of the Earth's surface, and are able to provide data covering the entire world. The advantage of polar-orbiting satellites is that they provide data with high spatial resolution and can capture surface details. However, their low temporal resolution and low observation frequency make them unsuitable for real-time monitoring. The data from geostationary satellites and polar-orbiting satellites are complementary. By integrating the data from the two, the temporal and spatial resolution of remote sensing data can be effectively improved, achieving more comprehensive and accurate earth observation.

[0004] Deep learning technology performs well in image processing and time series analysis. Using deep learning models, the spatiotemporal features of multi-source satellite data can be effectively extracted and fused. The existing deep learning multi-source satellite data fusion methods are only for a specific data product, such as sea fog products, temperature products, etc., but there is no spatiotemporal fusion method for satellite remote sensing primary data products, that is, multi-spectral channel data. Primary data products are the basis for the generation of subsequent advanced data products. Many secondary and tertiary data products, such as vegetation index, soil moisture, sea surface temperature, etc., are inverted from primary data.

[0005] Therefore, there is a need for a deep learning-based geostationary satellite and polar-orbiting satellite data fusion method that can perform earth observation more comprehensively and accurately. Summary of the invention

[0006] The main purpose of the present invention is to provide a method for fusing geostationary satellite and polar-orbiting satellite data based on deep learning to solve the problem that the prior art cannot conduct comprehensive and accurate earth observation.

[0007] To achieve the above object, the present invention provides a method for fusing geostationary satellite and polar-orbiting satellite data based on deep learning, which specifically includes the following steps:

[0008] S1, download geostationary satellite data, polar-orbiting satellite data and geographic information data. Both geostationary satellite data and polar-orbiting satellite data include: satellite sensor raw data and satellite geographic information data. Geographic information data includes: surface elevation data and surface type data;

[0009] S2, calibrates and corrects the original data of satellite sensors, and obtains the solar zenith angle, solar azimuth angle, satellite zenith angle and satellite azimuth angle from the satellite geographic information data.

[0010] S3, determine the polar-orbiting satellite sensor channel closest to each geostationary satellite sensor channel based on the central band of each sensor channel, and further determine the similarity between the geostationary satellite and the polar-orbiting satellite sensor channels based on the spectral response function using the spectral angle. If the spectra are similar, perform spatiotemporal matching according to S4 and proceed directly to step S6. If the spectra are not similar, execute step S4 and then step S5.

[0011] S4, finding the geostationary satellite and polar-orbiting satellite data scanned simultaneously within the target range according to the time information in the satellite data name, and making geostationary satellite and polar-orbiting satellite data pairs according to the corresponding channel sequence determined in step S3, and unifying the geostationary satellite and polar-orbiting satellite data into a high spatial resolution latitude and longitude grid.

[0012] S5 uses the method of minimizing the probability distribution distance and the Wasserstein distance as the metric for image registration.

[0013] S6, constructs a Swin Transformer model for each geostationary satellite channel, where the input is the geostationary satellite spectral channel data, satellite zenith angle, satellite azimuth angle, solar zenith angle, solar azimuth angle, longitude and latitude, elevation, and surface type, and the label is the corresponding polar-orbiting satellite spectral channel data.

[0014] S7, for each pre-trained model, a transfer learning model is constructed to enable the Swin Transformer model to generalize to all regions, and finally form an intelligent downscaling model.

[0015] S8, for each channel of the geostationary satellite, the high-temporal-resolution geostationary satellite spectral channel data, longitude and latitude data, elevation data and surface type data are used as input, and downscaling is performed through the Swin Transformer model trained in step S6 to obtain spectral channel data with high temporal and spatial resolution, and combined with the intelligent downscaling model trained in step S7 to form high-temporal and spatial resolution multispectral channel data covering the entire geostationary satellite scan area.

[0016] Furthermore, the geostationary satellite data in step S1 is a primary data product of a geostationary satellite imager, including spectral information data and geographic information data of each channel of the sensor; the polar-orbiting satellite data is a primary data product of a polar-orbiting satellite imager, including spectral information data and geographic information data of each channel of the sensor.

[0017] Furthermore, the data calibration and correction in step S2 are specifically as follows:

[0018] The first-level data products are radiometrically corrected and the original digital signals of the satellite sensors are converted into radiance. Based on the radiance and atmospheric conditions, the channel data of the visible to near-infrared band 0.2-4 μm are converted into apparent reflectance according to formula (1), and the channel data of the thermal infrared band 4-14 μm are converted into brightness temperature according to formula (2):

[0019] (1);

[0020] in, is the apparent reflectance, is the radiance observed by the satellite, is the distance between the Earth and the Sun, is the solar irradiance, is the solar zenith angle;

[0021] (2);

[0022] (3);

[0023] (4);

[0024] in, is the brightness temperature, is the radiance observed by the satellite, is the wavelength, is Planck's constant, which is 6.626×10 -34 Js, is the speed of light, take 3×10 8 m / s, is the Boltzmann constant, which is 1.38×10 -23 J / K, is Planck's first constant, is Planck's second constant.

[0025] Furthermore, in step S3, the similarity of the channel data is determined by using the spectral angle according to the spectral response function of the sensor, and the calculation method is as follows:

[0026] (5);

[0027] in, is the spectral angle in radians, is the spectral vector of the geostationary satellite, is the spectral vector of the polar-orbiting satellite, is the dot product of two vectors, is the magnitude of the vector, i.e. , is the magnitude of the vector, i.e. , set the spectral angle threshold is 0.2 radians, if , the two spectra are considered similar, and step S6 is performed after time-space matching according to step S4; if , the two spectra are considered to be dissimilar, and after performing the spatiotemporal matching of step S4, step S5 is executed to minimize the probability distribution of the two sensor spectral images.

[0028] Furthermore, step S4 specifically includes the following steps:

[0029] S4.1. The geostationary satellite uses the nominal projection of the geostationary orbit defined by the Integrated Global Observing System specification. The geographical location is represented by different row and column numbers. The method for converting row and column numbers to latitude and longitude is as follows:

[0030] (6);

[0031] (7);

[0032] (8);

[0033] (9);

[0034] (10);

[0035] (11);

[0036] (12);

[0037] (13);

[0038] (14);

[0039] (15);

[0040] in, is the nominal grid row number of the geostationary satellite data, is the nominal grid column number of the geostationary satellite data, and They are the results of transforming the image grid coordinates into the projection coordinates; is the distance from the satellite to a point on the Earth’s surface; is an intermediate variable used to calculate points on the Earth's surface; , and are the Cartesian coordinate components in the Earth-centered coordinate system, is the component along the equator, is the component along the longitude direction, is the component in the vertical direction; For the plane and The magnitude of the vector formed; is the geographic longitude, is the geographical latitude; is the semi-major axis of the Earth, which is 6378.137 km; is the semi-minor axis of the Earth, which is 6356.7523 km; is the distance from the center of the earth to the center of mass of the satellite, which is 42164 kilometers; The longitude of the satellite's subsatellite point, which is 104.7. and are column offset and row offset respectively; and are the column scale factor and row scale factor respectively.

[0041] S4.2, using bilinear interpolation method, the geostationary satellite multispectral channel data and the corresponding angle data, polar orbit satellite multispectral channel data, longitude and latitude data, surface elevation data, and surface type data are unified into a 1km×1km longitude and latitude grid. The calculation formula is as follows:

[0042] (16);

[0043] in, is the pixel coordinate The corresponding satellite observations are is the pixel coordinate The corresponding satellite observations are and As parameters, , , express Round down, express Round down, and are the neighboring integer pixel coordinates.

[0044] Furthermore, in step S5, the dissimilar geostationary satellite and polar-orbiting satellite spectral channel pair images are registered as follows:

[0045] S5.1, standardize the image to eliminate the influence of different data ranges. The calculation formula is as follows:

[0046] (17);

[0047] (18);

[0048] (19);

[0049] in, is the pixel value of the original image, is the normalized pixel value, is the total number of pixels in the image, It is The value of pixels.

[0050] S5.2, convert the image into a histogram representation and normalize it to a probability distribution:

[0051] (20);

[0052] in, Is the pixel value The probability distribution of is the total number of pixels, It is pixel values, is the Diracdelta function.

[0053] Furthermore, in step S5, the image is processed as follows for the spectral channels judged to be dissimilar:

[0054] S5.3, Calculate Wasserstein distance :

[0055] (twenty one);

[0056] in, and are two probability distributions, is the joint distribution of all possible A collection of is the infimum, indicating that among all possible joint distributions In the search, we can find the expected value The one that reaches the minimum The corresponding minimum expected value is yes and The distance between.

[0057] Calculate the discrete form of Wasserstein distance :

[0058] (twenty two);

[0059] in, is the joint distribution matrix, is the cost matrix.

[0060] S5.4, define the loss function, which represents the Wasserstein distance between two images:

[0061] (twenty three);

[0062] in, and is the probability distribution of the transformed image and the target image.

[0063] S5.5, use the optimization algorithm to minimize the loss function:

[0064] (twenty four);

[0065] in, is the optimization parameter, is the optimal parameter that minimizes the loss function, To take the minimum value.

[0066] S5.6, using the optimized parameters to transform the polar-orbiting satellite image to obtain an image that matches the stationary satellite image.

[0067] Furthermore, step S6 specifically includes the following steps:

[0068] S6.1, designing the Swin Transformer model includes: an input layer, a convolutional layer, a Swin Transformer module, an upsampling module and an output layer connected in sequence, wherein the input layer takes the geostationary satellite spectral channel data, satellite zenith angle, satellite azimuth angle, solar zenith angle, solar azimuth angle, longitude and latitude, elevation and surface type as input.

[0069] S6.2, combining multiple loss functions into a hybrid loss function , in order to simultaneously consider different aspects of optimization objectives:

[0070] (25);

[0071] in, , , is the weight of each loss term, is the mean square error loss, is the perceived loss, To combat the loss, the loss expressions are as follows:

[0072] (26);

[0073] (27);

[0074] (28);

[0075] in, is the pixel value predicted by the model, is the corresponding true pixel value, is the total number of pixels, represents the first Layer feature map, is the weight of the corresponding layer, is the discriminant probability of the generated image, is a logarithmic function.

[0076] S6.3, using the structural similarity index SSIM to evaluate model performance, for two images and , The calculation formula is:

[0077] (29).

[0078] in, and Respectively, images and The average brightness, and Respectively, images and The brightness variance, yes and The covariance between and is a small constant used for stability to avoid zero denominator, usually defined as:

[0079] (30);

[0080] (31);

[0081] in, is the dynamic range of pixel values. Since all the data involved have been normalized, Take 1, and is a constant, usually taken as and .

[0082] Furthermore, step S7 specifically includes the following steps:

[0083] S7.1, obtain labeled source domain data, i.e., matched geostationary satellite and polar-orbiting satellite data pairs and unlabeled target domain data, use convolutional neural network to extract features from source domain and target domain data, and normalize and standardize the features.

[0084] S7.2, use domain adversarial neural network to design loss function, by training a domain classifier to minimize the difference between source domain and target domain features, the loss function of the domain classifier for:

[0085] (32);

[0086] in, is the predicted probability of the domain classifier, is the real domain label, is the number of samples.

[0087] S7.3, use the domain adversarial neural network to train the model so that the feature extraction network can extract features that are consistent with both the source domain and the target domain. The label prediction error of the source domain sample is calculated using the cross entropy loss. The calculation formula is as follows:

[0088] (33);

[0089] in, is the label classifier for the category The predicted probability of is the true label.

[0090] Total loss function for:

[0091] (34);

[0092] in, is the weight hyperparameter of the domain adversarial loss.

[0093] The present invention has the following beneficial effects:

[0094] The present invention uses the primary data products of polar-orbiting satellites as labels to construct a downscaling algorithm for geostationary satellite multispectral channel data based on deep learning, and realizes the spatiotemporal fusion of geostationary-polar-orbiting satellite multispectral channel data. This method has the following obvious advantages:

[0095] (1) Automatic feature extraction: Deep learning models can automatically learn high-level feature representations from raw data. Compared with traditional feature extraction techniques that rely on manual design, they reduce human intervention and can better capture complex spatiotemporal patterns in the data.

[0096] (2) Multi-scale information processing: Deep learning models, especially those based on hierarchical structures such as Swin Transformer, are able to process and fuse information from different spatiotemporal scales, which is particularly important for multi-source satellite data with different spatiotemporal resolutions obtained by different sensors.

[0097] (3) Efficient processing capabilities for large-scale data: Satellite remote sensing data are usually large in scale. Deep learning algorithms are based on GPU-based distributed training and can efficiently process and train large-scale high-resolution data sets.

[0098] Flexibility and scalability: Deep learning can adapt to different data characteristics and task requirements by increasing the number of network layers, changing the network architecture, and combining different learning strategies (such as transfer learning, unsupervised learning, etc.). It has good flexibility and scalability. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] In order to more clearly illustrate the specific implementation of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the specific implementation or the prior art description. Obviously, the drawings described below are some implementations of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0100] Figure 1 A flow chart of a method for fusing geostationary satellite and polar-orbiting satellite data based on deep learning according to the present invention is shown.

[0101] Figure 2 Shown is the geostationary satellite spectral channel image before downscaling.

[0102] Figure 3The downscaled geostationary satellite spectral channel image is shown. DETAILED DESCRIPTION

[0103] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0104] A method for fusing geostationary satellite and polar-orbiting satellite data based on deep learning specifically comprises the following steps:

[0105] S1, download geostationary satellite data, polar-orbiting satellite data and geographic information data. Both geostationary satellite data and polar-orbiting satellite data include: satellite sensor raw data and satellite geographic information data. Geographic information data includes surface elevation data and surface type data.

[0106] S2, calibrates and corrects the original data of satellite sensors, and obtains the solar zenith angle, solar azimuth angle, satellite zenith angle and satellite azimuth angle from the satellite geographic information data.

[0107] S3, determining the polar-orbiting satellite sensor channel closest to each geostationary satellite sensor channel according to the central band of each sensor channel, and further determining the similarity between the geostationary satellite and the polar-orbiting satellite sensor channels according to the spectral response function and the spectral angle. If the spectra are similar, proceed directly to step S6 after performing spatiotemporal matching according to S4. If the spectra are not similar, proceed to step S5 after executing S4.

[0108] S4, finding the geostationary satellite and polar-orbiting satellite data scanned simultaneously within the target range according to the time information in the satellite data name, and making geostationary satellite and polar-orbiting satellite data pairs according to the corresponding channel sequence determined in step S3, and unifying the geostationary satellite and polar-orbiting satellite data into high spatial resolution latitude and longitude grids through resampling.

[0109] S5 uses the method of minimizing the probability distribution distance and the Wasserstein distance as the metric for image registration.

[0110] S6, constructs a Swin Transformer model for each geostationary satellite channel, where the input is the geostationary satellite spectral channel data, satellite zenith angle, satellite azimuth angle, solar zenith angle, solar azimuth angle, longitude and latitude, elevation, and surface type, and the label is the corresponding polar-orbiting satellite spectral channel data. For example, if the geostationary satellite sensor has a total of 14 channels (Chnl_01, Chnl_02, ..., Chnl_14), and S3 determines the corresponding 14 polar-orbiting satellite channels (Chnl_03p, Chnl_01p, ..., Chnl_33p), then the model for the first channel (Chnl_01) of the geostationary satellite uses the spectral data of the geostationary satellite Chnl_01, satellite zenith angle, satellite azimuth, solar zenith angle, solar azimuth, elevation, surface type, longitude and latitude as feature inputs, and the spectral data of the third channel Chnl_03p of the polar-orbiting satellite as labels to build a pre-trained model with SwinTransformer as the architecture. The same is true for other channels, and the number of pre-trained models is consistent with the number of geostationary satellite channels.

[0111] S7, for each pre-trained model, a transfer learning model is constructed to enable the Swin Transformer model to be generalized to all regions, and finally an intelligent downscaling model is formed. The number of models is consistent with the number of geostationary satellite channels.

[0112] S8, for each channel of the geostationary satellite, the high-temporal-resolution geostationary satellite spectral channel data, longitude and latitude data, elevation data and surface type data are used as input, and downscaling is performed through the Swin Transformer model trained in step S6 to obtain spectral channel data with high temporal and spatial resolution, and combined with the intelligent downscaling model trained in step S7 to form high-temporal and spatial resolution multispectral channel data covering the entire geostationary satellite scan area.

[0113] The present invention is a method for improving the degree of refinement of satellite primary data by using an artificial intelligence model. The present invention uses polar-orbiting satellite data with high spatial resolution but low temporal resolution as labels, and geostationary satellite data with high temporal resolution but low spatial resolution and corresponding satellite observation angles and geographic information data as features for model pre-training, establishes a mapping relationship between low spatial resolution geostationary satellite data and high spatial resolution polar-orbiting satellite primary data products in the same space and time, and obtains high spatial resolution primary data products of all geostationary satellite coverage areas through transfer learning.

[0114] Below in conjunction with the present invention Figure 1Briefly describe the principle of the present invention: download geostationary satellite level 1 data products, polar-orbiting satellite level 1 data products and geographic information data, wherein the geostationary satellite data and polar-orbiting satellite level 1 data products include spectral data of each channel of the satellite sensor and satellite geographic information, and the downloaded geographic information data include surface elevation data and surface type data. Since the geostationary satellite data uses a nominal grid, it must be converted into a longitude and latitude grid. According to the spectral response function, the geostationary satellite data and polar-orbiting satellite data converted to the longitude and latitude grid are channel matched, and the spectral similarity of each channel is judged. For channel images with dissimilar spectra, image registration must be performed. The matched geostationary satellite data and polar-orbiting satellite data are matched in time and space together with the geographic information data, and resampled to a high spatial resolution grid. A Swin Transformer model is constructed for each geostationary satellite channel. For example, the geostationary satellite sensor has a total of 14 channels (Chnl_01, Chnl_02, ..., Chnl_14), and the corresponding 14 polar-orbiting satellite channels (Chnl_03p, Chnl_01p, ..., Chnl_33p) are determined through the channel matching step. Then, for the geostationary satellite Chnl_01 (Chnl_02, ..., Chnl_14), the spectral data of Chnl_01 (Chnl_02, ..., Chnl_14), satellite zenith angle, satellite azimuth angle, solar zenith angle, solar azimuth angle, elevation, surface type, longitude and latitude are used as feature inputs, and the spectral data of the third channel Chnl_03p (Chnl_01p, ..., Chnl_33p) of the polar-orbiting satellite is used as a label to perform model pre-training 1 (model pre-training 2, ..., model pre-training 14) based on the Swin Transformer architecture. The number of model pre-training is consistent with the number of geostationary satellite channels. For each pre-trained model, feature migration is performed to enable the Swin Transformer model to generalize to all regions, and finally form intelligent downscaling model 1, intelligent downscaling model 2, ..., intelligent downscaling model 14, and the number of models is consistent with the number of geostationary satellite channels. After the intelligent downscaling model for each channel of the geostationary satellite is trained, the high-temporal-resolution geostationary satellite spectral channel data, longitude and latitude data, elevation data, and surface type data are input into the intelligent downscaling model of the corresponding channel, and high-temporal-spatial-resolution multispectral channel data covering the entire geostationary satellite scan area can be obtained.

[0115] Specifically, the geostationary satellite data in step S1 is a primary data product of a geostationary satellite imager, including spectral information data and geographic information data of each channel of the sensor; the polar-orbiting satellite data is a primary data product of a polar-orbiting satellite imager, including spectral information data and geographic information data of each channel of the sensor.

[0116] Specifically, the data calibration and correction in step S2 are as follows:

[0117] The first-level data products are radiometrically corrected and the original digital signals of the satellite sensors are converted into radiance. Based on the radiance and atmospheric conditions, the channel data of the visible to near-infrared band 0.2-4 μm are converted into apparent reflectance according to formula (1), and the channel data of the thermal infrared band 4-14 μm are converted into brightness temperature according to formula (2):

[0118] (1);

[0119] in, is the apparent reflectance, is the radiance observed by the satellite, is the distance between the Earth and the Sun, is the solar irradiance, is the solar zenith angle.

[0120] (2);

[0121] (3);

[0122] (4);

[0123] in, is the brightness temperature, is the radiance observed by the satellite, is the wavelength, is Planck's constant, which is 6.626×10 -34 Js, is the speed of light, take 3×10 8 m / s, is the Boltzmann constant, which is 1.38×10 -23 J / K, is Planck's first constant, is Planck's second constant.

[0124] Specifically, a geostationary satellite imager usually has multiple channels, and each channel obtains different spectral information. The present invention performs downscaling model training for each channel, and it is necessary to determine the polar-orbiting satellite channel that is closest to each channel of the geostationary satellite according to the central wavelength of the channel. Since the sensors carried by geostationary satellites and polar-orbiting satellites are different, even channels with similar central wavelengths may still have performance differences. Therefore, the similarity of channel data is first determined by using the spectral angle based on the spectral response function of the sensor. In step S3, the similarity of channel data is determined by using the spectral angle based on the spectral response function of the sensor, and the calculation method is as follows:

[0125] (5);

[0126] in, is the spectral angle in radians, is the spectral vector of the geostationary satellite, is the spectral vector of the polar-orbiting satellite, is the dot product of two vectors, is the magnitude of the vector, i.e. , is the magnitude of the vector, i.e. , set the spectral angle threshold is 0.2 radians, if , the two spectra are considered similar, and step S6 is performed after time-space matching according to step S4; if , the two spectra are considered to be dissimilar, and after performing the spatiotemporal matching of step S4, step S5 is executed to minimize the probability distribution of the two sensor spectral images.

[0127] Specifically, the satellite data name usually contains time information. According to the time information in the name, the geostationary satellite and polar-orbiting satellite data scanned simultaneously within the target range are found, and the geostationary satellite-polar-orbiting satellite data pair is produced according to the corresponding channel sequence determined in the previous step. Further, the geostationary satellite and polar-orbiting satellite data are unified into a high spatial resolution latitude and longitude grid. Step S4 specifically includes the following steps: S4.1, geostationary satellite grid conversion: The geostationary satellite uses the geostationary orbit nominal projection defined by the Integrated Global Observing System specification, and the geographical location is represented by different row and column numbers. The method for converting the row and column numbers into longitude and latitude is as follows:

[0128] (6);

[0129] (7);

[0130] (8);

[0131] (9);

[0132] (10);

[0133] (11);

[0134] (12);

[0135] (13);

[0136] (14);

[0137] (15);

[0138] in, is the nominal grid row number of the geostationary satellite data, is the nominal grid column number of the geostationary satellite data, and They are the results of transforming the image grid coordinates into the projection coordinates; is the distance from the satellite to a point on the Earth’s surface; is an intermediate variable used to calculate points on the Earth's surface; , and are the Cartesian coordinate components in the Earth-centered coordinate system, is the component along the equator, is the component along the longitude direction, is the component in the vertical direction; For the plane and The magnitude of the vector formed; is the geographic longitude, is the geographical latitude; is the semi-major axis of the Earth, which is 6378.137 km; is the semi-minor axis of the Earth, which is 6356.7523 km; is the distance from the center of the earth to the center of mass of the satellite, which is 42164 kilometers; The longitude of the satellite's subsatellite point, which is 104.7. and are column offset and row offset respectively; and are the column scale factor and row scale factor respectively.

[0139] S4.2, resampling: Use bilinear interpolation to unify the low-resolution geostationary satellite multispectral channel data and corresponding angle data, the high-resolution polar-orbiting satellite multispectral channel data, longitude and latitude data, surface elevation data, and surface type data into a 1km×1km longitude and latitude grid. The calculation formula is as follows:

[0140] (16);

[0141] in, is the pixel coordinate The corresponding satellite observations are is the pixel coordinate The corresponding satellite observations are and As parameters, , , and are the neighboring integer pixel coordinates.

[0142] Specifically, in step S5, the dissimilar spectral channel pairs of the geostationary satellite and the polar-orbiting satellite are processed as follows:

[0143] S5.1, standardize the image to eliminate the influence of different data ranges. The calculation formula is as follows:

[0144] (17);

[0145] (18);

[0146] (19);

[0147] in, is the pixel value of the original image, is the normalized pixel value, is the total number of pixels in the image, It is The value of pixels.

[0148] S5.2, convert the image into a histogram representation and normalize it to a probability distribution:

[0149] (20);

[0150] in, Is the pixel value The probability distribution of is the total number of pixels, It is pixel values, is the Diracdelta function.

[0151] Specifically, in step S5, the image is processed as follows for the spectral channels judged to be dissimilar:

[0152] S5.3, Wasserstein distance is also called earth transport distance. The core idea is to measure the "work" required to transform one probability distribution into another. Specifically, Wasserstein distance can be regarded as a metric in optimal transport theory. Calculate Wasserstein distance :

[0153] (twenty one);

[0154] in, and are two probability distributions, is the joint distribution of all possible A collection of is the infimum, indicating that among all possible joint distributions In the search, we can find the expected value The one that reaches the minimum The corresponding minimum expected value is yes and The distance between.

[0155] Calculate the discrete form of Wasserstein distance :

[0156] (twenty two);

[0157] in, is the joint distribution matrix, is the cost matrix.

[0158] S5.4, define the loss function, which represents the Wasserstein distance between two images:

[0159] (twenty three);

[0160] in, and is the probability distribution of the transformed image and the target image.

[0161] S5.5, use the optimization algorithm to minimize the loss function:

[0162] (twenty four);

[0163] in, is the optimization parameter, is the optimal parameter that minimizes the loss function, To take the minimum value.

[0164] S5.6, using the optimized parameters to transform the polar-orbiting satellite image to obtain an image that matches the stationary satellite image.

[0165] Specifically, the above steps can ensure that the geostationary satellite data, polar-orbiting satellite data and geographic information data are aligned, that is, all data cover the same geographical area, are unified to the same grid and standardized, and the corresponding spectral channels are matched and image registration is achieved. For each channel, all paired images are divided into a training set, a validation set and a test set with geostationary satellite spectral channel data, satellite zenith angle, satellite azimuth angle, solar zenith angle, solar azimuth angle, elevation, and surface type as input. Using polar-orbiting satellite spectral channel data as labels, an image super-resolution deep learning model is constructed. The sliding window transformer Swin Transformer introduces a window sliding mechanism based on the self-attention mechanism of the traditional Transformer, divides the image into small blocks through a hierarchical structure, and applies the self-attention mechanism within these small blocks. Swin Transformer has significant advantages in processing image super-resolution tasks with spatial information, and can improve super-resolution effects in feature extraction and feature fusion.

[0166] Step S6 specifically includes the following steps:

[0167] S6.1, the design of the Swin Transformer model includes: a sequentially connected input layer, a convolutional layer (feature extraction of the input), a Swin Transformer module (used to extract multi-scale features of the image, where each module consists of multiple layers, each layer processes different resolutions, and within each layer, the image is divided into windows of fixed size, and a self-attention mechanism is applied within each window to capture local features; the displacement mechanism between windows moves the position of the window between different layers to ensure feature interaction across windows, thereby capturing global information), an upsampling module (using a sub-pixel convolutional layer to upsample the low-resolution feature map to the target resolution) and an output layer (i.e., converting the feature map into a high-resolution image through the last convolutional layer), where the input layer takes the geostationary satellite spectral channel data, satellite zenith angle, satellite azimuth angle, solar zenith angle, solar azimuth angle, longitude and latitude, elevation, and surface type as input.

[0168] S6.2, combining multiple loss functions into a hybrid loss function , in order to simultaneously consider different aspects of optimization objectives:

[0169] (25);

[0170] in, , , is the weight of each loss term, is the mean square error loss, is the perceived loss, To combat the loss, the loss expressions are as follows:

[0171] (26);

[0172] (27);

[0173] (28);

[0174] in, is the pixel value predicted by the model, is the corresponding true pixel value, is the total number of pixels, represents the first Layer feature map, is the weight of the corresponding layer, is the discriminant probability of the generated image, is a logarithmic function.

[0175] S6.3, the structural similarity index SSIM is used to evaluate the model performance. SSIM measures the similarity between images by comparing brightness, contrast and structure. and , The calculation formula is:

[0176] (29).

[0177] in, and Respectively, images and The average brightness, and Respectively, images and The brightness variance, yes and The covariance between and is a small constant used for stability to avoid zero denominator, usually defined as:

[0178] (30);

[0179] (31);

[0180] in, is the dynamic range of pixel values. Since all the data involved have been normalized, Take 1, and is a constant, usually taken as and For the SSIM of the entire image, take the average of the local SSIMs; take the average of the SSIM of the entire test set to get the average SSIM value of all image pairs. The value range of SSIM is between 0 and 1. The closer the value is to 1, the more similar the structures of the two images are.

[0181] Specifically, since the polar-orbiting satellite has a small scanning range, the matched geostationary satellite and polar-orbiting satellite data pairs may not cover all areas covered by the geostationary satellite itself. Therefore, a transfer learning method is required to generalize the intelligent downscaling model to all areas. Step S7 specifically includes the following steps:

[0182] S7.1, obtain labeled source domain data, i.e., matched geostationary satellite and polar-orbiting satellite data pairs, and unlabeled target domain data, i.e., geostationary satellite spectral channel data corresponding to areas not covered by matched geostationary satellite and polar-orbiting satellite data, use convolutional neural network to extract features from source domain and target domain data, and normalize and standardize the features.

[0183] S7.2, use domain adversarial neural network to design loss function, by training a domain classifier to minimize the difference between source domain and target domain features, the loss function of the domain classifier for:

[0184] (32);

[0185] in, is the predicted probability of the domain classifier, is the real domain label, is the number of samples.

[0186] S7.3, use domain adversarial neural network to train the model, so that the feature extraction network can extract features that are consistent in both the source domain and the target domain, thereby reducing the distribution difference between domains. During the training process, the source domain loss and domain alignment loss are optimized at the same time to ensure that the model not only performs well in the source domain, but also has good generalization ability in the target domain. The label prediction error of the source domain sample uses the cross entropy loss The calculation formula is as follows:

[0187] (33);

[0188] in, is the label classifier for the category The predicted probability of is the true label.

[0189] Total loss function for:

[0190] (34);

[0191] in, It is the weight hyperparameter of the domain adversarial loss. It is used to balance the label bandit loss and the domain adversarial loss. According to the performance in the target domain, the model hyperparameters such as learning rate are adjusted to optimize the model to the ideal state.

[0192] Combination Figure 2 and Figure 3 It can be seen that the present invention significantly improves the spatial resolution of geostationary satellite spectral images and enhances the details of the images, thereby improving their analysis and application value.

[0193] Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the essential scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A method for fusion of geostationary satellite and polar-orbiting satellite data based on deep learning, characterized in that: The specific steps include: S1, download geostationary satellite data, polar-orbiting satellite data and geographic information data. Both geostationary satellite data and polar-orbiting satellite data include: satellite sensor raw data and satellite geographic information data. Geographic information data includes: surface elevation data and surface type data; S2, calibrates and corrects the original data of satellite sensors, and obtains the solar zenith angle, solar azimuth angle, satellite zenith angle and satellite azimuth angle from the satellite geographic information data; S3, determining the polar-orbiting satellite sensor channel closest to each geostationary satellite sensor channel according to the central band of each sensor channel, and further determining the similarity between the geostationary satellite and the polar-orbiting satellite sensor channels according to the spectral response function and the spectral angle. If the spectra are similar, proceed directly to step S6 after performing spatiotemporal matching according to S4. If the spectra are not similar, proceed to step S5 after executing S4. S4, finding the geostationary satellite and polar-orbiting satellite data scanned simultaneously within the target range according to the time information in the satellite data name, and making geostationary satellite and polar-orbiting satellite data pairs according to the corresponding channel sequence determined in step S3, and unifying the geostationary satellite and polar-orbiting satellite data into a high spatial resolution latitude and longitude grid; S5, uses the method of minimizing the probability distribution distance and the Wasserstein distance as the metric for image registration; S6, construct a Swin Transformer model for each geostationary satellite channel, where the input is geostationary satellite spectral channel data, satellite zenith angle, satellite azimuth angle, solar zenith angle, solar azimuth angle, longitude and latitude, elevation, and surface type, and the label is the corresponding polar-orbiting satellite spectral channel data; S7, for each pre-trained model, build a transfer learning model to enable the Swin Transformer model to generalize to all regions, and finally form an intelligent downscaling model; S8, for each channel of the geostationary satellite, the high-temporal-resolution geostationary satellite spectral channel data, longitude and latitude data, elevation data and surface type data are used as input, and downscaling is performed through the Swin Transformer model trained in step S6 to obtain spectral channel data with high temporal and spatial resolution, and combined with the intelligent downscaling model trained in step S7 to form high-temporal and spatial resolution multispectral channel data covering the entire geostationary satellite scan area.

2. The method for fusion of geostationary satellite and polar-orbiting satellite data based on deep learning according to claim 1, characterized in that: The geostationary satellite data in step S1 is the primary data product of the geostationary satellite imager, including spectral information data and geographic information data of each channel of the sensor; the polar-orbiting satellite data is the primary data product of the polar-orbiting satellite imager, including spectral information data and geographic information data of each channel of the sensor.

3. The method for fusion of geostationary satellite and polar-orbiting satellite data based on deep learning according to claim 1, characterized in that: The data calibration and correction in step S2 are specifically as follows: The first-level data products are radiometrically corrected and the original digital signals of the satellite sensors are converted into radiance. Based on the radiance and atmospheric conditions, the channel data of the visible to near-infrared band 0.2-4 μm are converted into apparent reflectance according to formula (1), and the channel data of the thermal infrared band 4-14 μm are converted into brightness temperature according to formula (2): （1）； in, is the apparent reflectance, is the radiance observed by the satellite, is the distance between the Earth and the Sun, is the solar irradiance, is the solar zenith angle; （2）； （3）； （4）； in, is the brightness temperature, is the radiance observed by the satellite, is the wavelength, is Planck's constant, which is 6.626×10 -34 Js, is the speed of light, take 3×10 8 m / s, is the Boltzmann constant, which is 1.38×10 -23 J / K, is Planck's first constant, is Planck's second constant.

4. The method for fusion of geostationary satellite and polar-orbiting satellite data based on deep learning according to claim 1, characterized in that: In step S3, the similarity of the channel data is determined by using the spectral angle according to the spectral response function of the sensor. The calculation method is as follows: （5）； in, is the spectral angle in radians, is the spectral vector of the geostationary satellite, is the spectral vector of the polar-orbiting satellite, is the dot product of two vectors, is the magnitude of the vector, i.e. , is the magnitude of the vector, that is , set the spectral angle threshold is 0.2 radians, if , the two spectra are considered similar, and step S6 is performed after time-space matching according to step S4; if , the two spectra are considered to be dissimilar, and after performing the spatiotemporal matching of step S4, step S5 is executed to minimize the probability distribution of the two sensor spectral images.

5. The method for fusion of geostationary satellite and polar-orbiting satellite data based on deep learning according to claim 1, characterized in that: Step S4 specifically includes the following steps: S4.

1. The geostationary satellite uses the nominal projection of the geostationary orbit defined by the Integrated Global Observing System specification. The geographical location is represented by different row and column numbers. The method for converting row and column numbers to latitude and longitude is as follows: （6）； （7）； （8）； （9）； （10）； （11）； （12）； （13）； （14）； （15）； in, is the nominal grid row number of the geostationary satellite data, is the nominal grid column number of the geostationary satellite data, and They are the results of transforming image grid coordinates into projection coordinates; is the distance from the satellite to a point on the Earth’s surface; is an intermediate variable used to calculate points on the Earth's surface; , and are the Cartesian coordinate components in the Earth-centered coordinate system, is the component along the equator, is the component along the longitude direction, is the component in the vertical direction; For the plane and The magnitude of the vector formed; is the geographic longitude, is the geographical latitude; is the semi-major axis of the Earth, which is 6378.137 km; is the semi-minor axis of the Earth, which is 6356.7523 km; is the distance from the center of the earth to the center of mass of the satellite, which is 42164 kilometers; The longitude of the satellite's subsatellite point, which is 104.

7. and are column offset and row offset respectively; and are the column scale factor and row scale factor respectively; S4.2, using bilinear interpolation method, the geostationary satellite multispectral channel data and the corresponding angle data, polar orbit satellite multispectral channel data, longitude and latitude data, surface elevation data, and surface type data are unified into a 1km×1km longitude and latitude grid. The calculation formula is as follows: （16）； in, is the pixel coordinate The corresponding satellite observations are is the pixel coordinate The corresponding satellite observations are and As parameters, , , express Round down, express Round down, and are the neighboring integer pixel coordinates.

6. The method for fusion of geostationary satellite and polar-orbiting satellite data based on deep learning according to claim 1, characterized in that: In step S5, the dissimilar geostationary satellite and polar-orbiting satellite spectral channel pair images are registered as follows: S5.1, standardize the image to eliminate the influence of different data ranges. The calculation formula is as follows: （17）； （18）； （19）； in, is the pixel value of the original image, is the normalized pixel value, is the total number of pixels in the image, It is The value of pixels; S5.2, convert the image into a histogram representation and normalize it to a probability distribution: （20）； in, Is the pixel value The probability distribution of is the total number of pixels, It is pixel values, is the Diracdelta function.

7. The method for fusion of geostationary satellite and polar-orbiting satellite data based on deep learning according to claim 1, characterized in that: In step S5, the image is processed as follows for the spectral channels judged to be dissimilar: S5.3, Calculate Wasserstein distance : （21）； in, and are two probability distributions, is the joint distribution of all possible A collection of is the infimum, indicating that among all possible joint distributions In the search, we can find the expected value The one that reaches the minimum The corresponding minimum expected value is yes and The distance between Calculate the discrete form of Wasserstein distance : （22）； in, is the joint distribution matrix, is the cost matrix; S5.4, define the loss function, which represents the Wasserstein distance between two images: （23）； in, and is the probability distribution of the transformed image and the target image; S5.5, use the optimization algorithm to minimize the loss function: （24）； in, is the optimization parameter, is the optimal parameter that minimizes the loss function, To take the minimum value; S5.6, using the optimized parameters to transform the polar-orbiting satellite image to obtain an image that matches the stationary satellite image.

8. The method for fusion of geostationary satellite and polar-orbiting satellite data based on deep learning according to claim 1, characterized in that: Step S6 specifically includes the following steps: S6.1, designing a Swin Transformer model includes: an input layer, a convolutional layer, a Swin Transformer module, an upsampling module and an output layer connected in sequence, wherein the input layer takes the geostationary satellite spectral channel data, satellite zenith angle, satellite azimuth angle, solar zenith angle, solar azimuth angle, longitude and latitude, elevation and surface type as input; S6.2, combining multiple loss functions into a hybrid loss function , in order to simultaneously consider different aspects of optimization objectives: （25）； in, , , is the weight of each loss term, is the mean square error loss, is the perceived loss, To combat the loss, the loss expressions are as follows: （26）； （27）； （28）； in, is the pixel value predicted by the model, is the corresponding true pixel value, is the total number of pixels, represents the first Layer feature map, is the weight of the corresponding layer, is the discriminant probability of the generated image, It is a logarithmic function; S6.3, using the structural similarity index SSIM to evaluate model performance, for two images and , The calculation formula is: （29）； in, and Respectively, images and The average brightness, and Respectively, images and The brightness variance, yes and The covariance between and is a small constant used for stability to avoid zero denominator and is usually defined as: （30）； （31）； in, is the dynamic range of pixel values. Since all the data involved have been normalized, Take 1, and is a constant, usually taken as and .

9. The method for fusion of geostationary satellite and polar-orbiting satellite data based on deep learning according to claim 1, characterized in that: Step S7 specifically includes the following steps: S7.1, obtain labeled source domain data, i.e., matched geostationary satellite and polar-orbiting satellite data pairs and unlabeled target domain data, use convolutional neural network to extract features from the source domain and target domain data, and normalize and standardize the features; S7.2, use domain adversarial neural network to design loss function, by training a domain classifier to minimize the difference between source domain and target domain features, the loss function of the domain classifier for: （32）； in, is the predicted probability of the domain classifier, is the real domain label, is the number of samples; S7.3, use the domain adversarial neural network to train the model so that the feature extraction network can extract features that are consistent with both the source domain and the target domain. The label prediction error of the source domain sample is calculated using the cross entropy loss. The calculation formula is as follows: （33）； in, is the label classifier for the category The predicted probability of is the true label; Total loss function for: （34）； in, is the weight hyperparameter of the domain adversarial loss.

Images