Multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning

Abstract

The present invention relates to a multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning. It includes: obtaining a training data set, which includes data pairs; constructing a time series super-resolution pre-training model; inputting the data pairs into the time series super-resolution pre-training model, and training based on a pre-constructed loss function to obtain a trained time series super-resolution model; during training, a latent space encoder processes the input data pairs, outputs a latent space representation tensor and a latent space residual tensor; inputs the latent space representation tensor into a sequence spatiotemporal motion flow estimation module to obtain a spatiotemporal motion flow representation tensor; inputs the spatiotemporal motion flow representation tensor into an interpolation frame prediction module for convolution operation to obtain a latent space interpolation frame; inputs the latent space interpolation frame and the latent space residual tensor into an interpolation frame decoding module for restoration to obtain a high-resolution time series super-resolution result. The present invention changes the paradigm of multi-channel geostationary satellite time series super-resolution and improves accuracy and reliability.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning. Background Art

[0002] High temporal resolution geostationary meteorological satellite data can provide a more detailed convection change process, and provide a good data basis for studying convective system tracking, convective initiation, severe convective weather prediction, etc. However, the temporal resolution of current geostationary meteorological satellites is not enough to describe the rapid change process of convection. Limited by factors such as data and technical means, there are currently few studies on geostationary meteorological satellite time series super-resolution. Most studies treat geostationary meteorological satellite time series super-resolution as a video interpolation problem in the field of computer vision, and use the video interpolation method to perform geostationary meteorological satellite time series super-resolution.

[0003] In traditional video interpolation methods, temporal super-resolution methods based on linear interpolation or optical flow interpolation use linear interpolation or optical flow interpolation to perform temporal super-resolution on a set of adjacent frames to be super-resolution, and use the pixel value relationship of the corresponding positions of the frames to be super-resolution to synthesize the intermediate frames. The parameters of the method are constant values ​​and cannot be optimized using data-driven methods. Temporal super-resolution methods based on the deep learning optical flow estimation framework use different neural networks such as U-Net and Transformer to estimate the optical flow of the super-resolution frames, and synthesize the intermediate frames by twisting the super-resolution frames forward and backward to achieve temporal super-resolution.

[0004] However, the time series super-resolution method based on traditional linear interpolation or optical flow interpolation uses constant parameter values ​​for time series super-resolution. The time series super-resolution result is only determined by the frame to be super-resolution. It is impossible to use the massive historical data in stock to drive the optimization of model algorithm parameters, and lacks consideration of the physical properties of geostationary satellite data. Therefore, it is difficult to obtain accurate time series super-resolution results.

[0005] The temporal super-resolution method based on the deep learning optical flow estimation framework is mainly used to deal with the video interpolation problem in the field of computer vision. However, the properties and characteristics of geostationary satellite data and video data are different in many aspects. For example, the RGB channels of video data have good consistency in temporal changes, while the multi-channel representation of geostationary satellite data has bands with different physical meanings, so there are differences in temporal changes; on the other hand, the interpolation of video data is usually used for high frame rate of video, that is, the number of video frames is interpolated from single frame to multiple frames within one second, while the original frame rate of geostationary satellite data is 10 to 15 minutes per frame, which has obvious length difference in temporal interval with video data, so the factors considered when interpolating are also different. Therefore, the temporal super-resolution method of video interpolation in the field of computer vision is directly used in geostationary satellite data, which is not completely consistent with the task in terms of modeling paradigm. Summary of the invention

[0006] Based on this, it is necessary to provide a multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning, which can achieve time series super-resolution of multi-channel geostationary satellite data by estimating and modeling the motion flow of the entire sequence, in order to address the above technical problems.

[0007] A multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning, the method comprising:

[0008] Acquire a training data set, wherein the training data set includes data pairs consisting of low-resolution data to be super-resolved and high-resolution true value labels;

[0009] Constructing a temporal super-resolution pre-training model, wherein the temporal super-resolution pre-training model includes a latent space encoder, a sequence spatiotemporal motion flow estimation module, an interpolated frame prediction module, and an interpolated frame decoding module;

[0010] The data pairs are input into the temporal super-resolution pre-trained model, and training is performed based on a pre-constructed loss function to obtain a trained temporal super-resolution model; during training, the latent space encoder processes the input data pairs and outputs a latent space representation tensor and a latent space residual tensor; the latent space representation tensor is input into the sequence spatiotemporal motion flow estimation module to perform spatiotemporal motion flow estimation of the entire sequence to obtain a spatiotemporal motion flow representation tensor; the spatiotemporal motion flow representation tensor is input into the interpolated frame prediction module to perform convolution operation to obtain a latent space interpolated frame; the latent space interpolated frame and the latent space residual tensor are input into the interpolated frame decoding module to perform decoding and restoration to obtain a high-resolution temporal super-resolution result.

[0011] Compared with the prior art, the multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning proposed in the present invention has the following beneficial technical effects:

[0012] 1. The method proposed in the present invention is more in line with the time series variation characteristics of multi-channel geostationary satellites in the time series super-resolution modeling paradigm. By estimating and modeling the motion flow of the entire sequence, the accuracy and reliability of super-resolution data are improved, and it can provide key data support for meteorological research such as the prediction of convective initiation and convective weather.

[0013] 2. The present invention adopts an artificial intelligence model, which can continuously iterate and optimize the model to obtain more accurate high-resolution time series super-resolution results.

[0014] 3. The method proposed in the present invention has strong scalability and can select any area for time series super-resolution. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0016] Figure 1 1 is a flow chart of a multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning in one embodiment;

[0017] Figure 2 A schematic diagram of the structure of a time series super-resolution pre-training model in one embodiment;

[0018] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0020] It can be understood that in response to the defects of the above-mentioned traditional time series super-resolution methods, due to the many differences between geostationary satellite data and video data, the present invention breaks through the limitations of the past geostationary meteorological satellite time series super-resolution model based on traditional optical flow and video interpolation methods, by changing the modeling paradigm of geostationary meteorological satellite time series super-resolution, from using adjacent frames to estimate optical flow to using the entire sequence to estimate motion flow, and using motion flow to guide the super-resolution frame to generate predicted frames to achieve super-resolution interpolation. The time series super-resolution modeling paradigm is more in line with the time series change characteristics of multi-channel geostationary satellites, improves the accuracy and reliability of the data, and provides key data support for research in the meteorological field.

[0021] It is worth noting that the method proposed in the present invention can perform time series super-resolution according to any region. For ease of understanding, the following embodiments use the FY-4A satellite product to illustrate the training process, but this is not intended to limit the present invention. For example, it can also be generalized and applied to satellite B with the same viewing angle as FY-4A.

[0022] The following will describe the implementation of the present invention in detail with reference to the accompanying drawings in the embodiment diagram of the present invention.

[0023] In one embodiment, Figure 1 As shown, a multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning is provided, comprising the following steps:

[0024] Step 1: Obtain a training data set, which includes data pairs consisting of low-resolution data to be super-resolved and high-resolution true value labels.

[0025] It is understood that when performing model training, the data set generally includes a training set, a validation set, and a test set. After obtaining the data set and preprocessing it, this embodiment separates part of the data that does not participate in the training process as a test set, and the remaining data is used as a training data set in the neural network training process.

[0026] In addition, during training, the model input is low-resolution data to be super-resolved, and the final output is high-resolution time series super-resolved results. The high-resolution true value label is used as the target data to provide a standard for the model during the training process, guide the model optimization through loss calculation, and ultimately improve the reconstruction effect of high-resolution time series super-resolved results.

[0027] Step 2: construct a temporal super-resolution pre-training model, which includes a latent space encoder, a sequence spatiotemporal motion flow estimation module, an interpolated frame prediction module, and an interpolated frame decoding module.

[0028] Step 3, input the data pair into the time series super-resolution pre-trained model, and train it based on the pre-constructed loss function to obtain a trained time series super-resolution model; during training, the latent space encoder processes the input data pair and outputs the latent space representation tensor and the latent space residual tensor; the latent space representation tensor is input into the sequence spatiotemporal motion flow estimation module to perform spatiotemporal motion flow estimation of the entire sequence to obtain the spatiotemporal motion flow representation tensor; the spatiotemporal motion flow representation tensor is input into the interpolated frame prediction module for convolution operation to obtain the latent space interpolated frame; the latent space interpolated frame and the latent space residual tensor are input into the interpolated frame decoding module for decoding and restoration to obtain a high-resolution time series super-resolution result.

[0029] It can be understood that the time series super-resolution pre-training model constructed by the present invention has changed the traditional modeling paradigm of geostationary meteorological satellite time series super-resolution. Figure 2 , is the structure of the temporal super-resolution pre-training model proposed in this embodiment, which realizes the encoding of the input data sequence from the geostationary satellite data space to the latent space, performs the sequence spatiotemporal motion flow estimation module, predicts the latent space interpolation frame, and decodes the interpolation frame from the latent space to the geostationary satellite data space through the continuous processing process of the latent space encoder, the sequence spatiotemporal motion flow estimation module, the interpolation frame prediction module, and the interpolation frame decoding module, so as to obtain the high-resolution temporal super-resolution result. This model structure is more in line with the temporal variation characteristics of multi-channel geostationary satellites, and can significantly improve the accuracy and reliability of high-resolution temporal super-resolution results.

[0030] In the above-mentioned multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning, an artificial intelligence model is used to continuously iterate and optimize the model, thereby obtaining a more accurate high-resolution time series super-resolution result. In addition, the present invention breaks through the limitations of the previous geostationary meteorological satellite time series super-resolution model based on traditional optical flow and video interpolation methods. By changing the modeling paradigm of geostationary meteorological satellite time series super-resolution, the optical flow estimation using adjacent frames is changed to the motion flow estimation using the entire sequence, and the motion flow is used to guide the generation of prediction frames for the super-resolution frames to achieve super-resolution interpolation. In terms of the time series super-resolution modeling paradigm, it is more in line with the time series change characteristics of multi-channel geostationary satellites, improves the accuracy and reliability of the data, and can provide key data support for meteorological research such as the prediction of convective initiation and convective weather.

[0031] In one embodiment, obtaining a training data set includes:

[0032] Obtain satellite multi-channel data, perform row and column number conversion on the satellite multi-channel data to select regional data to be super-resolved; perform normalization on the regional data to obtain a training data set for training.

[0033] Specifically, take the multi-channel L1 product generated by Fengyun-4A as an example. First, receive the multi-channel L1 product generated by Fengyun-4A from 2018 to 2023, select the regional data to be super-resolved by row and column number conversion, select satellite channels 7-14 as the required channels of the embodiment, and use 160 and 350 as abnormal threshold limits. The data is normalized to the range of 0-1 through the maximum-minimum normalization method, thereby forming a training data set that can be used for neural network training. In addition, the 2023 data does not participate in the training process and is used as a test set.

[0034] In one embodiment, the training data set includes data pairs consisting of low-resolution data to be super-resolved and high-resolution true value labels, including:

[0035] Low-resolution data from two adjacent time periods are randomly selected from the training data set at a set interval as the data to be super-resolved, and the data to be super-resolved is expanded into an input data sequence to form low-resolution data to be super-resolved; fast scanning data at the middle time of the data to be super-resolved is selected as the true value label to form a high-resolution true value label; data pairs are constructed with low-resolution data to be super-resolved and high-resolution true value labels.

[0036] It can be understood that expanding the low-resolution data to be super-resolved into an input data sequence and combining it with the subsequent full-sequence estimation motion flow process is more in line with the temporal variation characteristics of multi-channel geostationary satellites; in addition, merging multiple low-resolution image frames to provide more contextual information can help reconstruct more detailed high-resolution images. Moreover, by expanding and supplementing data, the model can learn more diverse image features and improve its generalization ability, thereby enabling the model to generate higher-quality high-resolution images when processing different types of images and optimize the training effect.

[0037] Specifically, in step 1, the training data set formed after preprocessing is consistent with the original L1 product of FY-4A in terms of time series. There are two time resolutions alternating in the data in continuous time series, that is, each natural day starts at 0 o'clock and is a cycle of three hours. In the first 30 minutes of the first hour, the first 15 minutes of the second hour, the first 15 minutes and the last 15 minutes of the third hour, the time resolution is 15 minutes (one set of full-disk mode observation results every 15 minutes), and the remaining time is 5 minutes as the time resolution (three sets of fast scan mode observation results every 15 minutes). When constructing a data pair consisting of low-resolution data to be super-resolved and high-resolution true value labels, it is necessary to ensure that the high-resolution true value labels exist. Therefore, firstly, the rapid scanning data of 30 minutes in a row is selected, and the second set of rapid scanning data within every 15 minutes is used as the low-resolution data to be super-resolved for that 15 minutes (according to the scanning characteristics of Fengyun-4A, the second set of rapid scanning data within every 15 minutes and the adjacent full disk data can be regarded as a 15-minute interval). At the same time, according to the model requirements, the low-resolution data to be super-resolved is expanded and supplemented to a duration of 60 minutes or more. A 75-minute input data sequence (when the model is used for super-resolution of historical stock data, the frame to be super-resolution is selected as the midpoint and extended forward and backward to a sequence of 75 minutes in length; when the model is used for super-resolution of real-time data, the frame to be super-resolution is selected as the end point and extended forward to a sequence of 60 minutes in length); then the fast scanning data in the middle time of the low-resolution data to be super-resolution (that is, the third group of data in the first 15 minutes and the first group of data in the second 15 minutes) are selected as the true value labels, and a data pair consisting of low-resolution data to be super-resolution and high-resolution true value labels can be constructed.

[0038] In one embodiment, the structure expression of the latent space encoder is:

[0039] ;

[0040] In the formula, represents the channel attention module, which first calculates the global weights of the input data sequence, and then weights the different channels of the data sequence through channel weighting, thereby forming an adaptive satellite channel selection; Represents a module that combines convolutional layers and group normalization layers of different scales, where , , The step sizes are 1, 2, and 2 respectively, and the convolution kernel size is 3 to extract spatial features of different scales from the data, thereby encoding the data from the original distribution space into the latent feature space; Indicates low-resolution data to be super-resolved; represents the latent space representation tensor, which is the representation tensor of the input data sequence encoded in the latent space by the latent space encoder; Represents the latent space residual tensor, which is the residual tensor to be connected generated by encoding the super-resolution frame through the latent space encoder.

[0041] It is worth mentioning that , , The step size and convolution kernel size can be set according to requirements. The data proposed in this embodiment is to show the specific setting method and is not intended to be a specific limitation on the latent space encoder structure proposed in this embodiment.

[0042] In one embodiment, the structural expression of the sequence spatiotemporal motion flow estimation module is:

[0043] ;

[0044] In the formula, Represents recurrent convolution modules of different scales; A tensor representing the flow of spatiotemporal motion at different scales.

[0045] Among them, the circular convolution modules of different scales It can simultaneously extract coupled spatiotemporal features, and through stacking multiple circular convolution modules, it can provide spatiotemporal features of different scales, thereby obtaining a full sequence of spatiotemporal motion flow representation.

[0046] This embodiment obtains spatiotemporal motion flow representation tensors of different scales and estimates the spatiotemporal motion flow of the entire sequence through this process to represent the spatiotemporal changes of the sequence, thereby comprehensively capturing the spatiotemporal change information of the sequence, improving the model performance, and improving the accuracy of generating high-resolution time series super-resolution results.

[0047] In one embodiment, the structure expression of the interpolation frame prediction module is:

[0048] ;

[0049] In the formula, Represents a stacked recurrent convolution module; is the representation tensor of the data to be super-resolved; Represents the simulated prediction tensor of the first set of super-resolved data; , Represents two interpolated frames predicted by the interpolated frame prediction module in the latent space.

[0050] Specifically, two interpolation frame prediction modules are provided, namely, an interpolation frame prediction module and interpolated frame prediction module ,Will Multi-scale spatiotemporal motion flow , , , Input interpolation frame prediction module After the simulation prediction, the simulation prediction tensor of the first set of super-resolution data is generated And the first interpolated frame of the corresponding latent space prediction , then Input interpolation frame prediction module Perform simulated prediction to obtain the second interpolated frame of latent space prediction .

[0051] It can be understood that the initial hidden state of the interpolation frame prediction module is the multi-scale spatiotemporal motion flow estimated by the sequence spatiotemporal motion flow estimation module, and the motion flow is used to guide the super-resolution frame to generate the prediction frame to achieve super-resolution interpolation, that is, through this process, the interpolation frame prediction under the action of the spatiotemporal motion flow is generated in the latent space. It is worth noting that the number of interpolation frame prediction modules can be set according to the number of interpolation frames to be generated, and is not limited to the setting of two provided in this embodiment.

[0052] In one of the embodiments, the structure expression of the interpolation frame decoding module is:

[0053] ;

[0054] In the formula, Modules representing combinations of deconvolutional layers and group normalization layers of different scales; It represents the high-resolution temporal super-resolution result restored by the interpolation frame decoding module from the latent space.

[0055] It can be seen that the interpolated frame decoding module and the latent space encoder are symmetrical and inverse structures, so as to restore the interpolated frame prediction from the latent space decoding to the original geostationary satellite data space.

[0056] In one embodiment, the pre-constructed loss function expression is:

[0057] ;

[0058] In the formula, represents the loss function; Indicates the total number of data points; Represents the weighted coefficient of the loss function under different thresholds; Represents the model's high-resolution time series super-resolution data points; represents a high-resolution ground-truth labeled data point.

[0059] It can be understood that the loss function is mainly used in the training stage. It calculates the Manhattan distance between the true value label and the decoded restoration result of the interpolated frame and multiplies it by the weight corresponding to the interval as the formal training reconstruction loss, guiding the model to iterate in the direction of reconstructing accurate interpolated frames. Among them, 273 and 241 are thresholds of different depths of convection areas, and the losses of the corresponding intervals are assigned to enhance the super-resolution performance of the model in the convection area. It is worth noting that 273 and 241 in the loss function expression are one of the threshold setting methods provided in this embodiment. The thresholds of different depths of convection areas can be set according to the situation. The setting of thresholds 273 and 241 does not serve as a limitation on the loss function expression.

[0060] In addition, during training, the data of the Fengyun-4A geostationary satellite from 2018 to 2022 was used as training data, and the data of 2023 was used as test data. The number of input channels of the time series super-resolution pre-training model was 8 (channels 7-14), and the number of output channels was 1 (channel 12). During the training phase, the batch size was set to 64, the maximum number of training rounds was set to 200, the Adam optimizer was used, and the initial learning rate was set to 3×10 -4 , the learning rate decay strategy is platform decay, with 0.9 as the decay scale.

[0061] In one of the embodiments, the method further includes: processing historical multi-channel geostationary satellite data through a trained time series super-resolution model to generate a high-resolution time series super-resolution result.

[0062] Specifically, pairs of data to be super-resolution with adjacent time intervals of 15 minutes are selected from historical multi-channel geostationary satellite data in turn, where the time without full disk observation is replaced by a second set of data with rapid scanning data within 15 minutes, and the data to be super-resolution is expanded forward and backward to supplement the input data sequence of 75 minutes, forming low-resolution data to be super-resolution, which is used as the input of the time series super-resolution model, and the model generates high-resolution time series super-resolution results corresponding to the low-resolution data to be super-resolution. By traversing the existing historical multi-channel geostationary satellite data with the above process, a multi-channel geostationary satellite data set with high time resolution (5 minutes) can be obtained. It can be understood that the data pair to be super-resolution is the low-resolution data of two adjacent time points to be super-resolution. After obtaining the data pair to be super-resolution, it is expanded into a sequence in time series to form low-resolution data to be super-resolution.

[0063] In one of the embodiments, it also includes: processing real-time multi-channel geostationary satellite data through a trained time series super-resolution model to generate a high-resolution time series super-resolution result.

[0064] Specifically, real-time multi-channel geostationary satellite data is obtained, and two sets of data of the current time and the previous time are used as the data pair to be super-resolved, wherein the time without full disk observation is replaced by the second set of data of fast scanning data within 15 minutes, and the data to be super-resolved is extended forward to supplement the input data sequence of one hour, forming low-resolution data to be super-resolved, which is used as the input of the time series super-resolution model, and the model generates high-resolution time series super-resolution results corresponding to the low-resolution data to be super-resolved. The above process is performed on the real-time multi-channel geostationary satellite data to super-resolve the time resolution of the real-time multi-channel geostationary satellite data from 15 minutes to 5 minutes.

[0065] In one of the embodiments, in order to verify the effectiveness of the model method in the present invention, five evaluation indicators are used to evaluate the model method in this embodiment, and the five evaluation indicators are: peak signal-to-noise ratio (PSNR), structural similarity (SSIM), critical success index (CSI), hit rate (POD), and false alarm rate (FAR).

[0066] Among them, PSNR and SSIM represent the similarity between the super-resolved data and the true value label; CSI, POD, and FAR are commonly used scoring functions in meteorology, which represent the accuracy performance of super-resolved data under different thresholds. In this embodiment, 273, 241, and 221 are used as thresholds respectively to evaluate the super-resolved accuracy performance of super-resolved data in convective areas and deep convective areas.

[0067] The comparison of the index evaluation results based on the above-mentioned time series super-resolution method is shown in Table 1:

[0068] Table 1 Comparison of evaluation results of different time series super-resolution methods

[0069]

[0070] Among them, linear interpolation is a traditional geostationary satellite time series super-resolution method, SSM is a representative geostationary satellite time series super-resolution method based on a deep learning optical flow framework, the real-time super-resolution of the present invention represents the performance of the model method of the present invention in real-time super-resolution, and the historical super-resolution of the present invention represents the performance of the model method of the present invention in super-resolution of historical data. The comparison of the above evaluation results shows that the multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning proposed in the present invention has higher accuracy than the existing time series super-resolution methods, and the super-resolution results in the convection / deep convection area are particularly outstanding. The method proposed in the present invention changes the paradigm of multi-channel geostationary satellite time series super-resolution based on the time series variation characteristics of multi-channel geostationary satellites. By estimating and modeling the motion flow of the entire sequence, the accuracy and reliability of the super-resolution data are improved, which can provide key data support for meteorological research such as the prediction of convection initiation and convective weather.

[0071] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0072] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. During training, any reference to memory, storage, database or other media used in the embodiments provided by the present invention may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0073] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0074] The above-mentioned embodiments only express several implementation modes of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning, characterized in that: The method comprises: Acquire a training data set, wherein the training data set includes data pairs consisting of low-resolution data to be super-resolved and high-resolution true value labels; Constructing a temporal super-resolution pre-training model, wherein the temporal super-resolution pre-training model includes a latent space encoder, a sequence spatiotemporal motion flow estimation module, an interpolated frame prediction module, and an interpolated frame decoding module; The data pair is input into the temporal super-resolution pre-training model, and training is performed based on a pre-constructed loss function to obtain a trained temporal super-resolution model; during training, the latent space encoder processes the input data pair and outputs a latent space representation tensor and a latent space residual tensor; the latent space representation tensor is input into the sequence spatiotemporal motion flow estimation module to perform spatiotemporal motion flow estimation of the entire sequence to obtain a spatiotemporal motion flow representation tensor; the spatiotemporal motion flow representation tensor is input into the interpolation frame prediction module to perform convolution operation to obtain a latent space interpolation frame; the latent space interpolation frame and the latent space residual tensor are input into the interpolation frame decoding module to perform decoding and restoration to obtain a high-resolution temporal super-resolution result; The structure expression of the latent space encoder is: ； In the formula, represents the channel attention module; Modules representing combinations of convolutional layers and group normalization layers of different scales; Indicates low-resolution data to be super-resolved; Denotes the latent space representation tensor; represents the latent space residual tensor; The structural expression of the sequence spatiotemporal motion flow estimation module is: ； In the formula, Represents recurrent convolution modules of different scales; A tensor representing the flow of space-time motion at different scales; The structural expression of the interpolation frame prediction module is: ； In the formula, Represents a stacked recurrent convolution module; is the representation tensor of the data to be super-resolved; Represents the simulated prediction tensor of the first set of super-resolved data; , represents two interpolated frames predicted by the interpolated frame prediction module in the latent space; The structural expression of the interpolation frame decoding module is: ； In the formula, Modules representing combinations of deconvolutional layers and group normalization layers of different scales; It represents the high-resolution temporal super-resolution result restored by the interpolation frame decoding module from the latent space.

2. The multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning according to claim 1 is characterized in that: Get the training dataset, including: Acquire satellite multi-channel data, perform row and column number conversion on the satellite multi-channel data to select regional data to be super-resolved; The regional data is normalized to obtain a training data set for training.

3. The multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning according to claim 2 is characterized in that: The training data set includes data pairs consisting of low-resolution data to be super-resolved and high-resolution true value labels, including: Randomly select two adjacent low-resolution data at set intervals from the training data set as data to be super-resolved, and expand the data to be super-resolved into an input data sequence to form low-resolution data to be super-resolved; Selecting the fast scan data at the middle time of the data to be super-resolved as the true value label to form a high-resolution true value label; A data pair is constructed using the low-resolution data to be super-resolved and the high-resolution true value label.

4. The multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning according to claim 1, characterized in that: The pre-built loss function expression is: ； In the formula, represents the loss function; Indicates the total number of data points; Represents the weighted coefficient of the loss function under different thresholds; Represents the model's high-resolution time series super-resolution data points; represents a high-resolution ground-truth labeled data point.

5. The multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning according to any one of claims 1 to 4, characterized in that: Also includes: The historical multi-channel geostationary satellite data are processed through the trained time series super-resolution model to generate high-resolution time series super-resolution results.

6. The multi-channel geostationary satellite time series super-resolution method based on spatiotemporal self-supervised learning according to any one of claims 1 to 4, characterized in that: Also includes: The real-time multi-channel geostationary satellite data is processed through the trained time series super-resolution model to generate high-resolution time series super-resolution results.

Images