A Deep Learning Prediction Method for Tropical Cyclone Satellite Cloud Images

Abstract

The present invention relates to a deep learning prediction method for tropical cyclone satellite cloud images. The method includes: obtaining a tropical cyclone satellite cloud image data set, and obtaining a prediction result of the tropical cyclone satellite cloud image through a prediction model obtained by training; the prediction model is constructed based on the TCSTque network, and the TCSTque network includes an encoder, and the encoder includes 3 Stacked-CNN layers and 3 ST-LSTM layers. Each encoding stage of the sub-encoder of the Stacked-CNN layer and the ST-LSTM layer has a corresponding RecConx mechanism to each decoding stage. Compared with the prior art, the present invention has the advantages of tracking the spatio-temporal sequence movement trajectory of tropical cyclones and realizing the simultaneous flow of spatio-temporal memory, etc.

Description

Technical Field

[0001] The present invention relates to the field of satellite cloud image prediction, and in particular to a deep learning prediction method for tropical cyclone satellite cloud images. Background Art

[0002] A tropical cyclone is a mesoscale or warm weather cyclone generated on the surface of tropical and subtropical oceans. A severe tropical cyclone, namely a typhoon (hurricane), is one of the most destructive and unpredictable natural disasters and a cause of life and property losses. Tropical cyclones bring many catastrophic impacts to coastal countries every year. In recent years, with the significant research progress of deep learning in computer vision, speech recognition, etc., video prediction technology has begun to be widely used in tasks such as autonomous driving, traffic flow prediction, and action recognition. This has played a huge enlightenment role in the research of the meteorological forecasting field and provided new ideas for its future development direction. Therefore, today with the increasing growth of meteorological data, video prediction can learn and model the complex evolution law of typhoons from past satellite cloud image data, and then predict the development details of typhoons at future moments, and serve as a visual supplement and reference for existing numerical prediction means. Therefore, in order for people and the government to better cope with extreme natural disasters, it is very important to accurately predict tropical cyclone satellite cloud images.

[0003] Currently, many scholars at home and abroad have studied satellite cloud image prediction, and most of them still use traditional non-deep learning methods. Based on the cloud image grayscale, methods of cross-matching and correlation matching are adopted, and combined with backward trajectory prediction means to explore the movement law of cloud clusters, and cloud image prediction is objectively carried out; the concept of motion vector is applied to predict the movement of cloud clusters. However, the modern social needs and scientific research work make the non-linear and non-stationary characteristics of cloud clusters impossible to ignore. For example, the empirical orthogonal decomposition algorithm and the genetic algorithm are combined to construct a non-linear prediction algorithm based on model inversion; the singular value decomposition algorithm is used and combined with a radial basis network to realize the prediction research work of cloud clusters. These traditional methods have had outstanding performances in such prediction work, but with the establishment of more and more meteorological satellites, ocean stations and ground stations, the amount of meteorological data is increasing. As an important type of meteorological data, satellite cloud images have a large observation range, high data dimension, and strong spatial complexity. The currently adopted methods do not fully consider the time correlation of tropical cyclone changes and cannot well represent the path trajectory of satellite cloud images, which easily leads to problems such as complex processing processes and low prediction efficiency. Summary of the Invention

[0004] The purpose of the present invention is to provide a deep learning prediction method for tropical cyclone satellite cloud images that can track the spatio-temporal sequence movement trajectory of tropical cyclones and realize the simultaneous flow of spatio-temporal memory to overcome the defects of the above-mentioned existing technologies.

[0005] The object of the present invention can be achieved by the following technical solutions:

[0006] A deep learning prediction method for tropical cyclone satellite cloud images, the method comprising:

[0007] Obtain a tropical cyclone satellite cloud image data set, and obtain a prediction result of the tropical cyclone satellite cloud image through a prediction model obtained by training;

[0008] The prediction model is constructed based on the TCSTque network. The TCSTque network includes an encoder, and the encoder includes 3 Stacked-CNN layers and 3 ST-LSTM layers. There is a corresponding RecConx mechanism from each encoding stage to each decoding stage of the sub-encoder of the Stacked-CNN layer and the ST-LSTM layer. The construction and training process of the prediction model includes the following steps:

[0009] Normalize the data for modeling, and divide the data for modeling into a training set, a validation set, and a test set. Convert the satellite cloud image data of the training set into a two-dimensional matrix of the training set through missing value processing and normalization;

[0010] Calculate the model loss function of the TCSTque network;

[0011] Construct a Stacked-CNN layer, input the two-dimensional matrix of the training set into the Stacked-CNN layer, and perform training based on two-dimensional convolution and two-dimensional deconvolution. Extract features through the Stacked-CNN layer and the Polling layer to obtain a multi-dimensional input tensor;

[0012] Construct an ST-LSTM layer, input the multi-dimensional input tensor into the ST-LSTM layer, update the gates and states in the ST-LSTM layer based on the hidden state of the previous layer at the current time step to obtain a time memory unit, and update the gates and states in the ST-LSTM layer based on the state of the previous layer to obtain a space memory unit;

[0013] Obtain a final hidden state based on the time memory unit and the space memory unit, transfer the final hidden state to the next layer or the next time step, and perform a convolution operation on the last layer of all time steps to obtain a sequence of prediction tensors;

[0014] Integrate the Stacked-CNN layer and the ST-LSTM layer to obtain the TCSTque network. Test the accuracy of the TCSTque based on the model loss function, and update all the weights and bias values of the TCSTque network by calculating the error function through backpropagation to obtain a prediction model;

[0015] Verify the generalization ability of the prediction model based on the validation set, and input the test set into the prediction model. Measure the error of the prediction model according to the mean square error, structural similarity, and peak signal-to-noise ratio error.

[0016] Furthermore, the calculation process of the temporal memory unit is as follows:

[0017]

[0018]

[0019]

[0020]

[0021] where g t is the first input modulation gate, X t is the input at time t, * represents the convolution operator, ⊙ represents the Hadamard product, W xg is the input weight parameter of the first input modulation gate, is the hidden state of the l-th layer in the spatial dimension at time t - 1, W hg is the hidden state weight parameter of the first input modulation gate, b g is the bias term of the first input modulation gate, i t is the first input gate, σ is the sigmoid activation function, W xi is the input weight parameter of the first input gate, W ni is the hidden state weight parameter of the first input gate, b i is the bias term of the first input gate, f t is the first forget gate, W xf is the input weight parameter of the first forget gate, W hf is the hidden state weight parameter of the first forget gate, b f is the bias term of the first forget gate, is the temporal memory unit of the l-th layer in the spatial dimension at time t, is the temporal memory unit of the l-th layer in the spatial dimension at time t - 1.

[0022] Furthermore, the calculation process of the spatial memory unit is as follows:

[0023]

[0024]

[0025]

[0026]

[0027] where g′t is the second input modulation gate, X t is the input at time t, * represents the convolution operator, and ⊙ represents the Hadamard product, W′ xg is the input weight parameter of the second input modulation gate, W mg is the spatial memory weight parameter of the second input modulation gate is the spatial memory unit of the (l - 1)-th layer at time t in the spatial dimension, b′ g is the bias term of the second input modulation gate, i′ t is the second input gate, σ is the sigmoid activation function, W′ xi is the input weight parameter of the second input gate, W mi is the spatial memory weight parameter of the second input gate, b′ i is the bias term of the second input gate, f′ t is the second forget gate, W′ xf is the input weight parameter of the second forget gate, W mf is the spatial memory weight parameter of the second forget gate, b′ f is the bias term of the second forget gate is the spatial memory unit of the l-th layer at time t in the spatial dimension

[0028] Furthermore, the calculation process of the final hidden state is as follows:

[0029]

[0030]

[0031] where, o t is the shared output gate, W xo is the input weight parameter of the shared output gate is the hidden state of the l-th layer at time t - 1 in the spatial dimension, W ho is the hidden state weight parameter of the shared output gate is the time memory unit at time t, W co is the time memory unit weight parameter of the shared output gate is the spatial memory unit at time t, W mo is the spatial memory unit weight parameter of the shared output gate, b o is the bias term of the shared output gate is the final hidden state, W 1×1 is a 1×1 convolutional layer

[0032] Furthermore, the calculation process of the model loss function of the TCSTque network is as follows:

[0033]

[0034] Among them, loss t is the model loss function at time t, M is the height of the cloud map, N is the width of the cloud map, i is the height position, and j is the width position. is the predicted cloud map, is the true value.

[0035] Furthermore, the method used for normalizing the data for modeling is the Min - Max normalization method, and the Min - Max normalization method is specifically:

[0036]

[0037] Among them, Norm k is the normalized data, R k is the original data, MinR k and MaxR k are respectively the maximum and minimum values of a specific attribute in the data for modeling.

[0038] Furthermore, the single - channel grayscale map data of the normalized data is between [0, 1].

[0039] Furthermore, the error threshold of the prediction model is 0.01 - 0.0001, the learning rate of the prediction model is 0.01 - 0.1, and the maximum number of iterations of the prediction model is 50.

[0040] Furthermore, the number of hidden layer neurons in the 3 Stacked - CNN layers and the number of hidden layer neurons in the 3 ST - LSTM layers are 32, 64, and 128 respectively.

[0041] Furthermore, the division ratios of the training set, validation set, and test set are 80%, 10%, and 10% respectively.

[0042] Compared with the prior art, the present invention has the following beneficial effects:

[0043] (1) By adopting the ST - LSTM layer, the input gate, output gate, forget gate, and input modulation gate of the time memory unit in the ST - LSTM layer are updated by the hidden state of the previous layer at the current time step. Compared with the traditional method that uses the hidden state of the previous time step, the gradient during model training will not quickly disappear because it is trapped in the memory unit. The final hidden state in the ST - LSTM layer depends on the fusion of time and space memories, and thus realizes the simultaneous flow of standard time memory and spatio - temporal memory by means of the space memory unit.

[0044] (2) The spatial memory unit of the ST-LSTM layer is strengthened by increasing the depth of the recursive transition. The input gate, output gate, forget gate, and input modulation gate are updated by the hidden state of the previous layer at the current time step, which can better model and predict the details of the spatial representation and effectively model the shape deformation and motion trajectory of the target in spatio-temporal sequence prediction.

[0045] (2) The ST-LSTM structure uses a shared output gate for all memory types, which can effectively model shape deformation and track the motion trajectory in spatio-temporal sequences. Brief Description of the Drawings

[0046] Figure 1 is a flowchart of the present invention;

[0047] Figure 2 is a schematic flowchart of the process of constructing the cloud map dataset of the present invention;

[0048] Figure 3 is a schematic structural diagram of the prediction model of the present invention. Detailed Embodiments

[0049] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented on the premise of the technical solution of the present invention, and the detailed implementation manner and specific operation process are given, but the protection scope of the present invention is not limited to the following embodiments.

[0050] Definition 1: Traditional cloud map prediction: The non-deep learning satellite cloud map prediction method is also called the traditional prediction method, including using the cross-correlation method, time mutation method, linear extrapolation based on the local features of the cloud map and the vector relationship of motion, inputting the meteorological parameters of the atmosphere and clouds into the established numerical model, statistical methods for discovering the characteristics of tropical cyclone satellite cloud maps from historical data, comprehensive models that combine multiple models, multiple physical parameters, and multiple initial conditions in a prediction model, and prediction models based on traditional machine learning all belong to the traditional cloud map prediction model.

[0051] Definition 2: Tropical cyclone cloud map prediction: Mainly through the historical tropical cyclone satellite infrared channel cloud map data, predict the cloud map of the target tropical cyclone within a certain period of time in the future, which is one of the key research topics in multiple disciplines such as marine science, meteorological science, and computer science, and thus has a certain degree of interdisciplinary nature.

[0052] The following gives the specific steps of a deep learning prediction method for tropical cyclone satellite cloud maps. The flowchart of the present invention is as Figure 1 shown.

[0053] Based on the principles of deep learning and the characteristics of ST-LSTM and RecConx mechanisms, according to the spatio-temporal distribution characteristics of the tropical cyclone satellite cloud image dataset made from the tropical cyclone best track data and the Japan Digital Typhoon website, with the change of satellite cloud images in a future period as the prediction target, a tropical cyclone satellite cloud image prediction model is constructed based on the TCSTque network. The TCSTque network includes an encoder, and the encoder includes 3 Stacked-CNN layers and 3 ST-LSTM layers; the CNN and Pooling layers are used as the model extraction layers to compress and extract the important features of the input data; ST-LSTM and RecConx are used to extract time series features and transfer feature residuals; the results are used as the input of the Decoder to generate the final prediction results.

[0054] S1. According to the characteristics of the constructed TCSTque network model, select appropriate training and test data from the tropical cyclone satellite cloud image dataset to complete the training and testing of the constructed prediction. The process of constructing the cloud image dataset is as Figure 2 shown.

[0055] ① Normalize the data used for modeling to improve the training speed and prediction accuracy of the tropical cyclone cloud image prediction model. The normalization method selected in this paper is the Min-Max normalization method:

[0056]

[0057] This method realizes the equal-proportion scaling of the original data. NormR k is the normalized data, R k is the original data, MaxR k and MinR k are the maximum and minimum values of this attribute in the original dataset respectively. The normalized single-channel grayscale image data are all between [0, 1].

[0058] ② Divide the dataset into a training set, a validation set, and a test set according to the ratio of 80%, 10%, and 10% to improve the generalization ability of the model.

[0059] Reasonably set the error threshold of the model, with a value range between 0.01 and 0.0001, the learning rate is taken between 0.01 and 0.1, and the maximum number of iterations is 50. The satellite cloud image data of the training set is processed for missing values and normalized into a two-dimensional matrix, which is input into the neural network model. For the TCSTque network, the encoder contains three Stacked-CNN layers and three ST-LSTM layers. The number of hidden layer neurons in the Stacked-CNN layer and the ST-LSTM layer are respectively set to 32, 64, and 128. For the three sub-encoders of different scales, there is a corresponding RecConx mechanism from each encoding stage to each decoding stage.

[0060] ③ For the TCSTque network, define the model loss function during its training stage as the cloud proximity prediction loss function (loss):

[0061]

[0062] For each output image sample at time t, predict the cloud image and the true value The prediction loss between them is as shown in the above formula. Among them, loss t is the model loss function at time t, M is the height of the cloud image, N is the width of the cloud image, i is the height position, j is the width position, is the predicted cloud image, is the true value. Different from MSE, this loss function assigns higher weights to pixels with lower values by using a non-linear function with multiple penalty rates.

[0063] S2. Train the prediction model. The construction and training of the entire prediction model can be specifically divided into the following two steps:

[0064] ① Construction of Stacked-CNN: Input the feature of the satellite cloud image converted into a two-dimensional matrix into Stacked-CNN. The training process of Stacked-CNN is divided into two parts, two-dimensional convolution and two-dimensional transposed convolution. Among them, the number of layers of convolution and transposed convolution is the same.

[0065] Two-dimensional convolution:

[0066]

[0067]

[0068] Two-dimensional transposed convolution:

[0069] H out =(H in(-1)×stride[0]-2×padding[0]+dilation[0]×(kernel_size[0]-1)+output_padding[0]+1

[0070] W out =(W in -1)×stride[1]-2×padding[1]+dilation[1]×(kernel_size[1]-1)+output_padding[1]+1

[0071] where H out and W out represent the height and width of the feature map after convolution or deconvolution respectively, and H in and W in represent the height and width of the feature map before convolution or deconvolution respectively. Stride, padding, dilation, and kernel_size represent the convolution kernel moving step, padding, dilation, and the size of the convolution kernel respectively. In this stage, the infrared channel corresponding to the initial satellite cloud image is mainly used as the input, and feature extraction is performed at multiple scales through the Stacked-CNN and Polling layers to generate a multi-dimensional input tensor.

[0072] ② Construction of the ST-LSTM layer: After the two-dimensional input matrix is extracted by the Stacked-CNN, it is transformed into a temporal multi-dimensional tensor as the input of the ST-LSTM. Compared with the traditional LSTM unit, the ST-LSTM unit can not only simulate temporal dynamics, but also realize the simultaneous flow of standard time memory and spatio-temporal memory with the help of the spatial memory unit. The values (X t-k+1 , X t-k+2 , …, X t ,) before time t for k hours are used as the input of the model, and the predicted target is the feature map of the tropical cyclone satellite cloud image at one or more time steps after time t. Each ST-LSTM unit contains a hidden state time memory unit and spatial memory unit At time t, the l-th layer consists of a hidden state and a storage unit. Specifically, the spatial memory unit Different from the LSTM, the input gate, output gate, forget gate, and input modulation gate are all updated by the hidden state of the previous layer at the current time step, rather than the hidden state of the previous time step. In other words, the purpose of this unit is to memorize and transmit the spatial information between layers. The detailed formula is shown below.

[0073]

[0074]

[0075]

[0076]

[0077] where g t is the first input modulation gate, X t is the input at time t, * represents the convolution operator, ⊙ represents the Hadamard product, and W xg is the input weight parameter of the first input modulation gate, is the hidden state of the l-th layer in the spatial dimension at time t - 1, and W hg is the hidden state weight parameter of the first input modulation gate, and b g is the bias term of the first input modulation gate, and i t is the first input gate, σ is the sigmoid activation function, and W xi is the input weight parameter of the first input gate, and W ni is the hidden state weight parameter of the first input gate, and b i is the bias term of the first input gate, and f t is the first forget gate, and W xf is the input weight parameter of the first forget gate, and W hf is the hidden state weight parameter of the first forget gate, and b f is the bias term of the first forget gate, is the time memory unit of the l-th layer in the spatial dimension at time t, is the time memory unit of the l-th layer in the spatial dimension at time t - 1.

[0078] C l is the classical time memory unit, which is transmitted from the previous node at t - 1 to the current time step within each LSTM unit. The first input gate i t represents a gate that can select information. The first forget gate f t deletes information from the memory. And the first input modulation gate g t represents the degree to which the previous result needs to be updated. That is to say, these three gates are used to control the flow of information through C l i.e., the time memory unit. In this way, the gradient during model training will not quickly disappear due to being trapped in the memory unit.

[0079] Similarly, at the same time step, the spatial state M lVertically transmit from layer l - 1 to this node. The short - term modeling ability of this mechanism is enhanced by increasing the depth of the recursive transition, and each gate is updated by the hidden state of the previous layer at the current time step. In other words, the purpose of this unit is to memorize the spatial information between layers. Therefore, it can better model and predict the details of the spatial representation, and effectively model the shape deformation and motion trajectory of the target in spatio - temporal sequence prediction. The detailed formula is as follows.

[0080]

[0081]

[0082]

[0083]

[0084] Among them, g′ t is the second input modulation gate, X t is the input at time t, * represents the convolution operator, ⊙ represents the Hadamard product, W′ xg is the input weight parameter of the second input modulation gate, W mg is the spatial memory weight parameter of the second input modulation gate, is the spatial memory unit of the (l - 1) - th layer in the spatial dimension at time t, b′ g is the bias term of the second input modulation gate, i′ t is the second input gate, σ is the sigmoid activation function, w′ xi is the input weight parameter of the second input gate, W mi is the spatial memory weight parameter of the second input gate, b′ i is the bias term of the second input gate, f′ t is the second forgetting gate, W′ xf is the input weight parameter of the second forgetting gate, W mf is the spatial memory weight parameter of the second forgetting gate, b′ f is the bias term of the second forgetting gate, is the spatial memory unit of the l - th layer in the spatial dimension at time t.

[0085] The corresponding gates and states on the spatial memory unit are updated between layers. Specifically, use the superscript l to represent the l - th layer in the spatial dimension, because the spatial memory unit M t will be transmitted along the spatial dimension. Similarly, the input gate i′ t is used to select information, the forgetting gate f′ t is used to delete information from the memory, then the input module gate g′ tDetermines the degree of update required. These gates are all determined by the input X of the current layer t and the spatial memory unit Finally, the final hidden state of the units in this layer depends on the fusion of temporal and spatial memories. These memories from different directions are concatenated together. In order to make the hidden state H l have the same dimension as the memory unit, a 1×1 convolutional layer is applied for dimensionality reduction. To achieve seamless fusion of memories, different from simple memory concatenation, the ST-LSTM structure adopts a shared output gate o t for all memory types, which can effectively model shape deformations and track motion trajectories in spatio-temporal sequences. The main formula for the final hidden state of ST-LSTM is given as follows.

[0086]

[0087]

[0088] where, o t is the shared output gate, W xo is the input weight parameter of the shared output gate, is the hidden state of the l-th layer in the spatial dimension at time t-1, W ho is the hidden state weight parameter of the shared output gate, is the temporal memory unit at time t, W co is the temporal memory unit weight parameter of the shared output gate, is the spatial memory unit at time t, W mo is the spatial memory unit weight parameter of the shared output gate, b o is the bias term of the shared output gate, is the final hidden state, W 1×1 is a 1×1 convolutional layer.

[0089] The final hidden state is passed as the state to the next layer or time step. To obtain the predicted frame at each step, a convolution operation is performed on Therefore, by performing convolution operations on the last layer at all time steps, a sequence of prediction tensors can be obtained, which has the same size and the same number of channels as the tensor sequence in the ST-LSTM module.

[0090] Finally, a new TCSTque model is generated by combining the above ST-LSTM layer and Stacked-CNN layer. The structure diagram of the model is as shown in Figure 3As shown in the figure. The model is based on an encoder-decoder architecture. The encoder network contains an ST-LSTM layer and stacked sequential convolutional layers, which can extract spatio-temporal features into a hidden feature vector. Subsequently, the decoder network, which contains an ST-LSTM layer and stacked sequential deconvolutional layers, can decode the feature vector into a series of future image sequences. The accuracy of the prediction is measured by the model loss function, and the error function is calculated by backpropagation to update all the weights and bias values of the network to achieve the optimal result. When the model training is completed, the values of each parameter are determined.

[0091] S3. Verify the generalization ability of the obtained prediction model using the validation set. For the test set, input it into the model to obtain the prediction result, and use the mean square error, structural similarity, and peak signal-to-noise ratio error to measure the error of the model. The calculation formulas are as follows:

[0092]

[0093]

[0094]

[0095] MSE calculates the mean squared error between the pixels of the predicted image and the true pixels of the target image, where g is a set of pixels of the target image within the size of M×N, is a set of pixels of the predicted image. SSIM can measure the similarity between the true image and the predicted image in terms of brightness, contrast, and structure. PSNR is used to measure the ratio between the maximum possible signal power and the power of the distorted noise that affects its representation quality. Here, peakval is the maximum value in the data. If the value of each pixel is of 8-bit unsigned integer data type, the peak value is 255. In short, the smaller the value of MSE and the larger the values of SSIM and PSNR, the more accurate the prediction result, indicating that the model meets the expectations and can be used to predict the cloud images of tropical satellites within a certain period of time in the future.

[0096] Evaluate the performance of the predicted model obtained from the training. Through experimental analysis and comparison, compared with other existing methods, the model of the present invention has good improvements in terms of time efficiency and prediction accuracy, and can process a large number of tropical cyclone satellite cloud image datasets.

[0097] S4. Obtain the tropical cyclone satellite cloud image dataset and obtain the tropical cyclone satellite cloud image prediction result through the predicted model obtained by training.

[0098] The preferred specific embodiments of the present invention have been described in detail above. It should be understood that those of ordinary skill in the art can make many modifications and variations based on the concept of the present invention without creative efforts. Therefore, all technical solutions that can be obtained by those skilled in the art in the technical field based on the concept of the present invention through logical analysis, reasoning, or limited experiments on the basis of the prior art should fall within the protection scope determined by the claims.

Claims

1. A deep learning prediction method for tropical cyclone satellite cloud images, characterized in that the method Including: Obtain the satellite cloud image dataset of tropical cyclones, and get the prediction results of tropical cyclone satellite cloud images through the prediction model obtained by training; The prediction model is constructed based on the TCSTque network. The TCSTque network includes an encoder, and the encoder includes 3 Stacked-CNN layers and 3 ST-LSTM layers. There is a corresponding RecConx mechanism from each encoding stage to each decoding stage of the sub-encoder of the Stacked-CNN layer and the ST-LSTM layer. The construction and training process of the prediction model includes the following steps: Normalize the data for modeling, and divide the data for modeling into a training set, a validation set, and a test set. Convert the satellite cloud image data of the training set into a two-dimensional matrix of the training set through missing value processing and normalization; Calculate the model loss function of the TCSTque network; Construct a Stacked-CNN layer, input the two-dimensional matrix of the training set into the Stacked-CNN layer, and perform training based on two-dimensional convolution and two-dimensional deconvolution. Extract features through the Stacked-CNN layer and the Polling layer to obtain a multi-dimensional input tensor; Construct an ST-LSTM layer, input the multi-dimensional input tensor into the ST-LSTM layer, update the gates and states in the ST-LSTM layer based on the hidden state of the previous layer at the current time step to obtain a time memory unit, and update the gates and states in the ST-LSTM layer based on the state of the previous layer to obtain a spatial memory unit; Obtain the final hidden state based on the time memory unit and the spatial memory unit, transfer the final hidden state to the next layer or the next time step, and perform a convolution operation on the last layer of all time steps to obtain a sequence of prediction tensors; Integrate the Stacked-CNN layer and the ST-LSTM layer to obtain the TCSTque network. Test the accuracy of the TCSTque based on the model loss function, and update all the weights and bias values of the TCSTque network by calculating the error function through backpropagation to obtain the prediction model; Verify the generalization ability of the prediction model based on the validation set, input the test set into the prediction model, and measure the error of the prediction model according to the mean square error, structural similarity, and peak signal-to-noise ratio error.

2. The deep learning prediction method for tropical cyclone satellite cloud images according to claim 1, characterized in that The calculation process of the time memory unit is: Among them, g t is the first input modulation gate, X t is the input at time t, * represents the convolution operator, ⊙ represents the Hadamard product, W xg is the input weight parameter of the first input modulation gate, is the hidden state of the l-th layer in the spatial dimension at time t - 1, W hg is the hidden state weight parameter of the first input modulation gate, b g is the bias term of the first input modulation gate, i t is the first input gate, σ is the sigmoid activation function, W xi is the input weight parameter of the first input gate, W hi is the hidden state weight parameter of the first input gate, b i is the bias term of the first input gate, f t is the first forget gate, W xf is the input weight parameter of the first forget gate, W hf is the hidden state weight parameter of the first forget gate, b f is the bias term of the first forget gate, is the time memory unit of the l-th layer in the spatial dimension at time t, is the time memory unit of the l-th layer in the spatial dimension at time t - 1.

3. A deep learning prediction method for tropical cyclone satellite cloud images according to claim 1, characterized in that The calculation process of the spatial memory unit is: Among them, g′ t is the second input modulation gate, X t is the input at time t, * represents the convolution operator, ⊙ represents the Hadamard product, W′ xg is the input weight parameter of the second input modulation gate, W mg is the spatial memory weight parameter of the second input modulation gate, is the spatial memory unit of the (l - 1)-th layer spatial dimension at time t, b′ g is the bias term of the second input modulation gate, i′ t is the second input gate, σ is the sigmoid activation function, W′ xi is the input weight parameter of the second input gate, W mi is the spatial memory weight parameter of the second input gate, b′ i is the bias term of the second input gate, f t ′ is the second forget gate, W′ xf is the input weight parameter of the second forget gate, W mf is the spatial memory weight parameter of the second forget gate, b′ f is the bias term of the second forget gate, is the spatial memory unit of the l-th layer spatial dimension at time t.

4. A deep learning prediction method for tropical cyclone satellite cloud images according to claim 1, characterized in that, The calculation process of the final hidden state is: Among them, o t is the shared output gate, W xo is the input weight parameter of the shared output gate, is the hidden state of the l-th layer in the spatial dimension at time t-1, W ho is the hidden state weight parameter of the shared output gate, is the time memory cell at time t, W co is the time memory cell weight parameter of the shared output gate, is the spatial memory cell at time t, W mo is the spatial memory cell weight parameter of the shared output gate, b o is the bias term of the shared output gate, is the final hidden state, W 1×1 is a 1×1 convolutional layer.

5. A deep learning prediction method for tropical cyclone satellite cloud images according to claim 1, characterized in that, The calculation process of the model loss function of the TCSTque network is: Among them, loss t is the model loss function at time t, M is the height of the cloud map, N is the width of the cloud map, i is the height position, and j is the width position. is the predicted cloud map, is the true value.

6. The deep learning prediction method for tropical cyclone satellite cloud images according to claim 1, characterized in that The method used for normalizing the data for modeling is the Min-Max normalization method. The Min-Max normalization method is specifically: Among them, Norm k is the normalized data, and R k is the original data. MinR k and MaxR k are the maximum and minimum values of a specific attribute in the data for modeling, respectively.

7. A deep learning prediction method for tropical cyclone satellite cloud images according to claim 6, characterized in that, The single-channel grayscale image data of the normalized data is between [0, 1].

8. A deep learning prediction method for tropical cyclone satellite cloud images according to claim 1, characterized in that The error threshold of the prediction model is 0.01 - 0.0001, the learning rate of the prediction model is 0.01 - 0.1, and the maximum number of iterations of the prediction model is 50.

9. A deep learning prediction method for tropical cyclone satellite cloud images according to claim 1, characterized in that The number of hidden layer neurons in the 3 Stacked-CNN layers and the number of hidden layer neurons in the 3 ST-LSTM layers are 32, 64, and 128 respectively.

10. A deep learning prediction method for tropical cyclone satellite cloud images according to claim 1, characterized in that, The division ratios of the training set, validation set, and test set are 80%, 10%, and 10% respectively.

Images