Radar echo image prediction method based on global enhanced PredRNN

Abstract

This application relates to a radar echo image prediction method based on global enhanced PredRNN. The method comprises: obtaining the first t‑m Time step t The radar echo images of each time step are preprocessed and combined into a radar echo image sequence of fixed length in chronological order; the radar echo images of each time step in the radar echo image sequence are divided into blocks to obtain the radar echo sub-images of each time step; the radar echo sub-images of each time step are input into the trained radar echo image prediction model based on global reinforcement PredRNN in chronological order to obtain the future t +1 time step to t+m The radar echo prediction image with +1 time step significantly improves the ability to capture long-term dependencies, thereby improving the extrapolation prediction accuracy of the radar echo image.

Description

Technical Field

[0001] The present application relates to the technical field of meteorological forecasting based on deep learning, and in particular to a radar echo image prediction method based on a global enhanced PredRNN. Background Art

[0002] In recent years, the demand for real-time analysis and prediction of radar echo images in weather forecasting has been increasing. As crucial data reflecting the spatial distribution of precipitation and other weather phenomena, radar echo images possess complex spatiotemporal dynamics and are influenced by multiple nonlinear factors. Traditional statistical and physical models often struggle to accurately capture the inherent connections between local details and global structure when processing these highly nonlinear, time-varying image data, thus limiting forecast accuracy and timeliness.

[0003] With the development of deep learning technology, models based on recurrent neural networks (RNNs), long short-term memory networks (LSTMs), and their improved models (such as PredRNN) have achieved remarkable results in spatiotemporal sequence prediction tasks. These methods have improved their ability to capture dynamic information to a certain extent, but they still suffer from the problem of vanishing or exploding gradients in long-sequence predictions, making it difficult to effectively transmit long-range information, resulting in low accuracy in the extrapolation prediction of radar echo images. Summary of the Invention

[0004] Based on this, it is necessary to provide a radar echo image prediction method based on global enhanced PredRNN that can improve the extrapolation prediction accuracy of radar echo images in response to the above technical problems.

[0005] A radar echo image prediction method based on a global enhanced PredRNN, the radar echo image prediction method based on a global enhanced PredRNN comprising:

[0006] Get the radar echo image of the target area from the tmth time step to the tth time step, where t is the current time step and m is a positive integer;

[0007] Preprocessing the radar echo images and combining the radar echo images into a radar echo image sequence of fixed length in chronological order;

[0008] performing block processing on the radar echo image of each time step of the radar echo image sequence to obtain a block-based radar echo image sequence, wherein the block-based radar echo image sequence includes a radar echo sub-image of each time step;

[0009] Inputting the radar echo sub-images of each time step into the trained radar echo image prediction model based on global enhancement PredRNN in chronological order for prediction, and obtaining radar echo prediction images from the future t+1 time step to t+m+1 time step;

[0010] The radar echo image prediction model includes: a first globally enhanced ST-LSTM module, a second globally enhanced ST-LSTM module, a gradient highway module, a third globally enhanced ST-LSTM module, and a fourth globally enhanced ST-LSTM module;

[0011] The first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the gradient highway module, the third globally enhanced ST-LSTM module and the fourth globally enhanced ST-LSTM module are connected in sequence.

[0012] The radar echo image prediction method based on global enhanced PredRNN obtains radar echo images from the tmth time step to the tth time step of the target area; preprocesses each of the radar echo images and combines each of the radar echo images into a radar echo image sequence of fixed length in chronological order; performs block processing on the radar echo image of each time step of the radar echo image sequence to obtain a radar echo sub-image of each time step; and sequentially inputs the radar echo sub-image of each time step into the trained radar echo image prediction model based on global enhanced PredRNN in chronological order for prediction to obtain The radar echo prediction image for the future time step t+1 to t+m+1 is obtained, wherein the radar echo image prediction model includes: a first globally enhanced ST-LSTM module, a second globally enhanced ST-LSTM module, a gradient highway module, a third globally enhanced ST-LSTM module, and a fourth globally enhanced ST-LSTM module; the first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the gradient highway module, the third globally enhanced ST-LSTM module, and the fourth globally enhanced ST-LSTM module are connected in sequence. Thus, the gradient highway unit is introduced into the radar echo image prediction model, which effectively alleviates the problem of gradient vanishing or exploding in deep recurrent networks, significantly improves the ability to capture long-term dependencies, and thus improves the extrapolation prediction accuracy of the radar echo image. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 1 is a flow chart of a radar echo image prediction method based on a global enhanced PredRNN in one embodiment;

[0014] Figure 2Schematic diagram of the structure of a gradient highway unit (GHU) module in one embodiment;

[0015] Figure 3 Schematic diagram of the structure of a globally enhanced ST-LSTM network in one embodiment;

[0016] Figure 4 A schematic diagram of a structure of a multi-source attention unit in one embodiment;

[0017] Figure 5 1. A schematic diagram of the operation flow of a radar echo image prediction model at different time steps during the training process in one embodiment;

[0018] Figure 6 FIG1 is a schematic diagram of a curve showing changes in MSE over time in one embodiment;

[0019] Figure 7 FIG1 is a schematic diagram of a curve showing SSIM changing over time in one embodiment;

[0020] Figure 8 A schematic diagram for visual comparison of prediction results in one embodiment. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0022] In one embodiment, Figure 1 As shown, a radar echo image prediction method based on a global enhanced PredRNN is provided, comprising the following steps:

[0023] Step S220 , obtaining radar echo images of the target area from the tmth time step to the tth time step, where t is the current time step and m is a positive integer.

[0024] Step S240 : pre-processing the radar echo images and combining the radar echo images into a radar echo image sequence of fixed length in chronological order.

[0025] Among them, preprocessing includes noise removal, normalization and other operations.

[0026] It should be understood that radar echo image sequences are packaged into a data structure of the shape [batch,m+1,1,img_height, img_width] and input into the block processing module for block processing. Specifically, batch represents the batch size, which indicates the number of radar echo image sequences input into the radar echo image prediction model at a time; the sequence length of the radar echo image sequence is m+1, meaning that each radar echo image sequence contains m+1 radar echo image frames; the number of radar echo image channels is 1; and img_height and img_width represent the height and width of each radar echo image, respectively.

[0027] Among them, using a radar echo image sequence of a fixed length can ensure that each sequence can fully capture the temporal and spatial change information of the target area over a period of time.

[0028] Among them, the block processing module is a module that implements block processing of the radar echo image of each time step in the radar echo image sequence to obtain a block-based radar echo image sequence, which includes the radar echo sub-image of each time step.

[0029] Step S260 , performing block processing on the radar echo image of each time step of the radar echo image sequence to obtain a block-based radar echo image sequence, wherein the block-based radar echo image sequence includes a radar echo sub-image of each time step.

[0030] Among them, the block processing is to evenly divide the radar echo image of each time step into several radar echo sub-images of equal size. Assuming that the height and width of the radar echo image are evenly divided into n parts respectively, the radar echo image of each time step will be divided into n 2 radar echo sub-images, n 2 A radar echo sub-image is a radar echo sub-image at each time step.

[0031] It should be understood that the radar echo image sequence input to the block processing module is a data structure of the shape [batch,m+1,1,img_height,img_width]. After the block processing, the data structure of the formed radar echo image sequence becomes [batch,m+1,n²,img_height / n,img_width / n]. Among them, the number of channels after the radar echo image is expanded by block processing is n. 2 The number of channels is equal to the total number of radar echo sub-images in each radar echo image multiplied by the number of channels of the radar echo image; img_height / n and img_width / n: the height and width of each radar echo sub-image.

[0032] In step S280, the radar echo sub-images of each time step are sequentially input into the trained radar echo image prediction model based on the global enhancement PredRNN in chronological order for prediction, and the radar echo prediction images for the future time steps t+1 to t+m+1 are obtained.

[0033] Among them, the radar echo image prediction model includes: a first globally enhanced ST-LSTM module, a second globally enhanced ST-LSTM module, a gradient highway module, a third globally enhanced ST-LSTM module and a fourth globally enhanced ST-LSTM module; the first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the gradient highway module, the third globally enhanced ST-LSTM module and the fourth globally enhanced ST-LSTM module are connected in sequence.

[0034] Among them, the input radar echo sub-image is processed by block processing, and all prediction results also exist in the form of blocks. Therefore, the various radar echo prediction sub-images can be reassembled into a complete radar echo image through the block recovery algorithm.

[0035] Among them, the radar echo image prediction model based on the global enhanced PredRNN changes the computing module ST-LSTM of the existing PredRNN model, performs global context enhancement on the spatiotemporal memory in the computing module, and integrates multiple source attention units inside the computing module to form a globally enhanced ST-LSTM (Global EnhancedSpatioTemporalLSTM, GEn-STLSTM) module. The original model architecture of the existing PredRNN model is changed, and the Gradient Highway Unit (GHU) module and residual connection are added to construct a radar echo image prediction model based on the global enhanced PredRNN, namely the GEn-PredRNN model.

[0036] Among them, at each time step, the residual addition strategy is implemented to superimpose the network output with the current input, which significantly alleviates the current situation where the prediction performance decreases sharply with the increase of time steps, and provides robust and consistent technical support for the extrapolation prediction of radar echo images, which is especially suitable for real-time forecasting in complex meteorological scenarios.

[0037] The radar echo image prediction method based on global enhanced PredRNN obtains the radar echo images of the target area from the tmth time step to the tth time step; preprocesses each radar echo image and combines each radar echo image into a radar echo image sequence of fixed length in chronological order; divides the radar echo image of each time step of the radar echo image sequence into blocks to obtain a radar echo sub-image of each time step; and inputs the radar echo sub-image of each time step into the trained radar echo image prediction model based on global enhanced PredRNN in chronological order for prediction. , obtaining the radar echo prediction image from the future time step t+1 to the time step t+m+1. The radar echo image prediction model includes: a first globally enhanced ST-LSTM module, a second globally enhanced ST-LSTM module, a gradient highway module, a third globally enhanced ST-LSTM module, and a fourth globally enhanced ST-LSTM module; the first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the gradient highway module, the third globally enhanced ST-LSTM module, and the fourth globally enhanced ST-LSTM module are connected in sequence. Thus, the gradient highway unit is introduced into the radar echo image prediction model, effectively alleviating the problem of vanishing or exploding gradients in deep recurrent networks, significantly improving the ability to capture long-term dependencies, and thus improving the extrapolated prediction accuracy of radar echo images.

[0038] In one embodiment, the radar echo sub-images at each time step are sequentially input into a trained radar echo image prediction model based on a global enhanced PredRNN in chronological order for prediction, and radar echo prediction images for the future time steps t+1 to t+m+1 are obtained, including:

[0039] The radar echo sub-image of the tm-th time step is input into the radar echo image prediction model based on the global reinforcement PredRNN that has been trained, and the radar echo prediction sub-image of the t-m+1-th time step is output; the radar echo sub-image of the t-m+1-th time step is input into the radar echo image prediction model based on the global reinforcement PredRNN that has been trained, and the radar echo prediction sub-image of the t-m+2-th time step is output; and so on, until the radar echo sub-image of the t-m+m-th time step is input into the radar echo image prediction model based on the global reinforcement PredRNN that has been trained, and the radar echo prediction sub-image of the future t+ The radar echo prediction sub-image of the t+1th time step is input into the trained radar echo image prediction model based on global reinforcement PredRNN, and the radar echo prediction sub-image of the future t+2th time step is output; and so on, the radar echo prediction sub-image of the t+mth time step is input into the trained radar echo image prediction model based on global reinforcement PredRNN, and the radar echo prediction sub-image of the future t+m+1th time step is output; the radar echo prediction sub-images from the t+1th time step to the t+m+1th time step are spliced ​​to obtain the radar echo prediction images from the t+1th time step to the t+m+1th time step in the future.

[0040] Among them, the radar echo sub-image of each input time step is obtained through block processing, and all prediction results also exist in the form of blocks. Therefore, it is necessary to use a block recovery algorithm to re-stitch the various radar echo prediction sub-images into a complete radar echo prediction image.

[0041] It should be understood that although the radar echo sub-image based on the tm time step can predict the radar echo prediction sub-image of the t-m+1 time step, the radar echo sub-image of the t-m+1 time step can predict the radar echo prediction sub-image of the t-m+2 time step, and so on, until the radar echo sub-image of the t-1 time step predicts the radar echo prediction sub-image of the t time step, what needs to be predicted is the radar echo prediction image from the future t+1 time step to t+m+1 time step. Therefore, the predicted radar echo sub-image from the tm time step to the t time step is useless data and can be not output as the final prediction result. The predicted radar echo sub-image from the tm time step to the t time step is to enable the radar echo image prediction model to capture key features, thereby improving the extrapolation prediction accuracy of the radar echo image.

[0042] In one example, the radar echo prediction sub-images from the t+1th time step to the t+m+1th time step are spliced ​​to obtain the radar echo prediction image from the future t+1th time step to the t+m+1th time step. Specifically, the radar echo prediction sub-images of the future t+1th time step are spliced ​​using a block recovery algorithm to obtain the radar echo prediction image of the future t+1th time step. The radar echo prediction sub-images of the future t+2th time step are spliced ​​using a block recovery algorithm to obtain the radar echo prediction image of the future t+2th time step. Similarly, the radar echo prediction sub-images of the future t+m+1th time step are spliced ​​using a block recovery algorithm to obtain the radar echo prediction image of the future t+m+1th time step.

[0043] In one embodiment, the radar echo image prediction model predicts the radar echo prediction sub-image at the i-th time step in the following manner:

[0044] The sub-image of the i-1th time step input to the radar echo image prediction model and the fourth global enhanced spatiotemporal memory generated during the i-1th time step prediction are input to the first globally enhanced ST-LSTM module. The first globally enhanced ST-LSTM module generates the first hidden state, the first cell state, the fourth global enhanced spatiotemporal memory generated during the i-1th time step prediction, and the image of the i-1th time step, wherein, , where i is to At any time step in the i-1 time step, the sub-image of the i-1 time step is the radar echo sub-image of the i-1 time step. to At any time step in , the sub-image at the i-th time step is the radar echo prediction sub-image at the i-1-th time step.

[0045] The first hidden state and the first global enhanced spatiotemporal memory at the time of prediction of the i-th time step are input into the second globally enhanced ST-LSTM module. The second globally enhanced ST-LSTM module generates the second hidden state, the second cell state and the second global enhanced spatiotemporal memory at the time of prediction of the i-1-th time step according to the second hidden state, the second cell state and the first hidden state at the time of prediction of the i-th time step and the first global enhanced spatiotemporal memory.

[0046] The second hidden state generated when predicting the i-th time step and the third hidden state generated when predicting the i-1-th time step are input into the gradient highway module. The gradient highway module generates the highway state when predicting the i-th time step based on the second hidden state generated when predicting the i-th time step and the third hidden state generated when predicting the i-1-th time step.

[0047] The highway state and the second global enhanced spatiotemporal memory at the time of prediction of the i-th time step are input into the third globally enhanced ST-LSTM module. The third globally enhanced ST-LSTM module generates the third hidden state, the third cell state and the third global enhanced spatiotemporal memory at the time of prediction of the i-1-th time step according to the third hidden state, the third cell state, the highway state at the time of prediction of the i-th time step and the second global enhanced spatiotemporal memory.

[0048] The third hidden state and the third global enhanced spatiotemporal memory at the time of prediction of the i-th time step are input into the fourth globally enhanced ST-LSTM module. The fourth globally enhanced ST-LSTM module generates the fourth hidden state, the fourth cell state and the fourth global enhanced spatiotemporal memory at the time of prediction of the i-1-th time step according to the fourth hidden state, the fourth cell state and the third global enhanced spatiotemporal memory at the time of prediction of the i-th time step.

[0049] The fourth hidden state at the time of prediction of the i-th time step is added element by element to the sub-image of the i-1-th time step to obtain the radar echo prediction sub-image of the i-th time step.

[0050] It should be understood that the prediction of the i-1th time step refers to the prediction of the radar echo prediction sub-image of the i-1th time step, and the prediction of the i-th time step refers to the prediction of the radar echo prediction sub-image of the i-th time step.

[0051] Among them, the fourth global enhanced spatiotemporal memory generated during the prediction of the i-1th time step is the fourth global enhanced spatiotemporal memory generated when the fourth globally enhanced ST-LSTM module predicts the radar echo prediction sub-image of the i-1th time step.

[0052] Among them, the first hidden state and the first cell state generated during the prediction of the i-1th time step are the first hidden state and the first cell state generated when the first globally reinforced ST-LSTM module predicts the radar echo prediction sub-image of the i-1th time step.

[0053] Among them, the second hidden state and the second cell state generated during the prediction of the i-1th time step are the second hidden state and the second cell state generated when the second globally reinforced ST-LSTM module predicts the radar echo prediction sub-image of the i-1th time step.

[0054] Among them, the third hidden state and the third cell state generated during the prediction of the i-1th time step are the third hidden state and the third cell state generated when the third globally reinforced ST-LSTM module predicts the radar echo prediction sub-image of the i-1th time step.

[0055] Among them, the fourth hidden state and the fourth cell state generated during the prediction of the i-1th time step are the fourth hidden state and the fourth cell state generated when the fourth globally reinforced ST-LSTM module predicts the radar echo prediction sub-image of the i-1th time step.

[0056] Among them, by realizing residual connection by element-by-element addition of the fourth hidden state at the prediction of the i-th time step and the sub-image of the i-1-th time step, the input data can be directly combined with the prediction results, which can not only improve the detail recovery of the output of the radar echo image prediction model, but also effectively alleviate the problem of information attenuation in the deep network, ensuring that the output results are more accurate and stable.

[0057] Among them, if for When the radar echo image prediction model is The radar echo prediction sub-image of the time step is predicted in the following way:

[0058] Input the radar echo image prediction model The radar echo sub-image of the time step is input to the first globally enhanced ST-LSTM module, and the first globally enhanced ST-LSTM module is based on the The radar echo sub-image of the time step is generated The first hidden state, first cell state, and first global augmented spatiotemporal memory at the time step prediction.

[0059] The first The first hidden state and the first global enhanced spatiotemporal memory at the time step prediction are input to the second globally enhanced ST-LSTM module, and the second globally enhanced ST-LSTM module generates the first hidden state and the first global enhanced spatiotemporal memory at the time step prediction. The second hidden state, second cell state, and second global enhanced spatiotemporal memory at time step prediction.

[0060] The first The second hidden state generated during the time step prediction is input into the gradient highway module, which The second hidden state generated when predicting the time step is generated Highway status at the predicted time step.

[0061] The first The highway state at the time step prediction and the second global enhanced spatiotemporal memory are input to the third globally enhanced ST-LSTM module, which is then The highway state at the time step prediction and the second global enhanced spatiotemporal memory generate the The third hidden state, third cell state, and third global enhanced spatiotemporal memory at time step prediction.

[0062] The first The third hidden state and the third global enhanced spatiotemporal memory at the time step prediction are input to the fourth globally enhanced ST-LSTM module, and the fourth globally enhanced ST-LSTM module is based on the The third hidden state and the third global enhanced spatiotemporal memory at the time step prediction generate the The fourth hidden state, fourth cell state, and fourth global enhanced spatiotemporal memory at time step prediction.

[0063] The first The fourth hidden state at the time step prediction is the same as the first The radar echo sub-images of the time step are added element by element to obtain the Radar echo prediction sub-image for the time step.

[0064] It should be understood that when for When the radar echo image prediction model inputs the first radar echo sub-image of the radar echo image sequence after segmentation (i.e. The radar echo sub-image of the time step is predicted. The radar echo prediction sub-image of the time step is the first prediction using the blocked radar echo image sequence. Therefore, the hidden states, cell states, and global enhanced spatiotemporal memories corresponding to the first global enhanced ST-LSTM module, the second global enhanced ST-LSTM module, the third global enhanced ST-LSTM module, and the fourth global enhanced ST-LSTM module in the radar echo image prediction model are empty.

[0065] In one embodiment, the second hidden state generated during the prediction of the i-th time step and the third hidden state generated during the prediction of the i-1-th time step are input into the gradient highway module. The gradient highway module generates the highway state during the prediction of the i-th time step based on the second hidden state generated during the prediction of the i-th time step and the third hidden state generated during the prediction of the i-1-th time step, including:

[0066] The second hidden state generated during the prediction of the i-th time step is processed by the tanh activation function and added to the third hidden state generated during the prediction of the i-1-th time step to obtain a candidate tensor; the second hidden state generated during the prediction of the i-th time step is processed by the σ activation function and added to the third hidden state generated during the prediction of the i-1-th time step to obtain a first gating weight and a second gating weight; the candidate tensor is weighted according to the first gating weight to obtain a weighted candidate tensor; the third hidden state generated during the prediction of the i-1-th time step is weighted according to the second gating weight to obtain a weighted third hidden state; the weighted candidate tensor and the weighted third hidden state are feature fused to generate the highway state during the prediction of the i-th time step.

[0067] Among them, the gradient highway module plays a role in the multi-layer temporal convolutional network as a high-speed channel that specifically retains gradients and key information. Figure 2 As shown in the figure, the internal mechanism of the Gradient-Highway Unit (GHU) module is shown, which receives two inputs: the second hidden state generated by the previous layer (i.e., the second globally reinforced ST-LSTM module) at the time step i And the third hidden state generated by the third globally reinforced ST-LSTM module at the i-1 time step prediction , where l is the number of the globally reinforced ST-LSTM module. The second hidden state and the third hidden state First, convolution is used to map the network to a uniform channel dimension; one branch is activated by the tanh function to obtain the candidate tensor P, and the other branch is activated by the The activation function generates the first gating weight and the second gating weight, where the first gating weight is S, and the second gating weight is 1-S, S∈[0,1]. The gradient highway module then linearly interpolates the first and second gating weights element by element to obtain the highway state at the predicted time step i. , the expression is:

[0068] ;

[0069] in, is element-wise multiplication (Hadamard multiplication).

[0070] On the one hand, this expression injects the latest spatial-temporal features (i.e., candidate tensor P) into the memory of this layer, and on the other hand Retained in Historical information of proportions Therefore, the gradient can be along The identity branch of is back-propagated almost without attenuation, thus effectively alleviating the common gradient vanishing or exploding problems in deep temporal networks.

[0071] In the overall network topology, the gradient highway module is inserted between the second layer (i.e., the second globally reinforced ST-LSTM module) and the third layer (i.e., the third globally reinforced ST-LSTM module): vertically, it passes the features calculated by the second layer at the i-th time step upward to the third layer; horizontally, it maintains The recursive chain provides a stable information and gradient channel for the radar echo image prediction model. Compared to existing LSTM units and their variants, the Gradient Highway module relies on only two sets of lightweight convolutions and a single gate to achieve an adaptive trade-off between writing new information and retaining old memories. This significantly reduces the number of parameters and memory usage, while improving the network's ability to model long-range spatiotemporal dependencies and training stability.

[0072] In the middle layer of the network, the gradient highway module is used to fuse and adjust the features of the hidden state of the previous layer. The gradient highway module ensures that the gradient can be transmitted more smoothly in the deep network, alleviating the problems of gradient vanishing and explosion, so that the radar echo image prediction model can maintain better training stability and effect when capturing complex spatiotemporal dependencies.

[0073] Among them, the gradient highway module ensures the high-speed and stable transmission of gradients between network layers and in time, effectively alleviating the problem of gradient disappearance or explosion in deep recurrent networks, and significantly improving the ability to capture long-term dependencies.

[0074] In one embodiment, the first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the third globally enhanced ST-LSTM module and the fourth globally enhanced ST-LSTM module are globally enhanced ST-LSTM networks with the same structure.

[0075] Among them, as shown in Figure 3, the structural diagram of the globally enhanced ST-LSTM network, based on the original mechanism of the existing ST-LSTM network (including input gate, forget gate, output gate, cell state and hidden state, etc.), the globally enhanced ST-LSTM network additionally introduces multiple sources from the attention unit and the global enhanced memory path (i.e., the average pooling branch composed of the adaptive average pooling layer and the fully connected layer) (with the hidden state is represented by, where l is the number of the globally enhanced ST-LSTM module, ), and obtain global information through operations such as the adaptive average pooling layer (AdaptiveAvgPool2d) to strengthen spatiotemporal memory , update the cell state and hidden state. In this process, the adaptive average pooling layer (AdaptiveAvgPool2d) is used in conjunction with the fully connected layer (FC) to globally pool the input features and generate global contextual enhancement information. After multiplying it with the adaptive weight parameter α, it is added to the updated candidate spatiotemporal memory (i.e., the spatiotemporal memory that has not been added to the global enhancement information) to obtain the global enhanced spatiotemporal memory. , thus integrating local spatiotemporal information while dynamically adjusting with the help of global semantics, so that the radar echo image prediction model can better capture the long-term temporal dependencies in complex scenes. Overall, this "global-local" dual enhancement design can simultaneously integrate the hidden state of the previous time step of the current hidden layer at each time step. , cell status , space-time memory With the current input The multi-source information contained in it improves prediction accuracy while maintaining high stability.

[0076] Among them, the expression of the processing process of the globally enhanced ST-LSTM network is:

[0077] ,

[0078] ,

[0079] ,

[0080] ,

[0081] ,

[0082] ,

[0083] ,

[0084] ,

[0085] ,

[0086] ,

[0087] ,

[0088] ,

[0089] ,

[0090] ,

[0091] ,

[0092] ;

[0093] in, To map the input into a convolution of 7 gate vectors, To map the input into a convolution of 4 gate vectors, To map the input into a convolution of 3 gate vectors, The hidden state, highway state, or sub-image of the lth globally reinforced ST-LSTM module is input for the prediction of the i-th time step, is the hidden state generated by the lth globally reinforced ST-LSTM module when predicting the i-1th time step, The global enhanced spatiotemporal memory generated by the l-1th globally enhanced ST-LSTM module when predicting the i-th time step, For those from The input gate is pre-activated, For those from The forget gate pre-activation, For those from Candidate memory, is the global memory branch version of the input gate, is the global memory branch version of the forget gate, is the candidate memory of the global memory branch, is element-wise multiplication (Hadamard multiplication), for Contribution to the output gate, For those from The pre-activation of the input gate, For those from The pre-activation of the forget gate, For those from candidate memory, for Contribution to the output gate, For those from The pre-activation of the input gate, For those from The pre-activation of the forget gate, For those from The candidate memory of the input gate, is the output of the input gate, is the Sigmoid activation function, is the output of the forget gate, is the forget gate bias, is the output of the input modulation gate, is the hyperbolic tangent activation function, is the cell state generated by the lth globally reinforced ST-LSTM module when predicting at the i-th time step, is the cell state generated by the lth globally reinforced ST-LSTM module when predicting the i-1th time step, is the output of the input gate of the global enhanced memory branch, is the output of the forget gate of the global enhanced memory branch, is the output of the input modulation gate of the global enhanced memory branch, To enhance global information, is the adaptive average pooling layer, is the fully connected layer, To broadcast the 1*1 feature back to The same space size, is the globally enhanced spatiotemporal memory generated by the lth globally enhanced ST-LSTM module when predicting at the i-th time step, is the adaptive weight parameter, is the comprehensive memory tensor, To perform channel dimension stitching, is the output of the output gate, Indicates that Compression fusion obtains the features required to drive the output gate, Indicates that Compression fusion obtains the features required to generate hidden states, Most of them come from attention units. is the initial hidden state generated by the lth globally reinforced ST-LSTM module when predicting at the i-th time step, is the hidden state generated by the lth globally reinforced ST-LSTM module when predicting at the i-th time step.

[0094] in, 、 、 Used for gate vector splitting, the convolution kernel sizes of these three convolutions are uniformly controlled by the hyperparameters initialized by the radar echo image prediction model. The convolution kernel sizes of these three convolutions can be 5*5.

[0095] Among them, the adaptive average pooling layer is used to extract global context.

[0096] Among them, the fully connected layer is used to recalibrate the channels of the global context.

[0097] in, Used to generate output gate pre-activation, the convolution kernel size of this convolution can be 5*5.

[0098] in, Used to generate candidate hidden states, the convolution kernel size of this convolution can be 1*1.

[0099] in, Control the cell state generated by the lth globally reinforced ST-LSTM module when predicting at the i-th time step How much "new information" is received; Control the current cell-to-cell state How much to keep; After tanh, it participates in generating cell state ; and Jointly determine the global enhancement of spatiotemporal memory The amount of writes; and Decide together how much to keep .

[0100] Among them, the processing expression of the global enhanced memory branch is:

[0101] ,

[0102] ,

[0103] ,

[0104] ,

[0105] .

[0106] It should be understood that in the processing of the global enhanced ST-LSTM network of the first global enhanced ST-LSTM module, there is no global enhanced ST-LSTM module before the first global enhanced ST-LSTM module. Therefore, the input of the first global enhanced ST-LSTM module is a sub-image processed by the ST-LSTM module without global enhancement. If i is to At any time step in , the sub-image of the i-1th time step is the radar echo sub-image of the i-1th time step. If i is to At any time step in , the sub-image at the i-th time step is the radar echo prediction sub-image at the i-1-th time step.

[0107] It should be understood that in the processing of the global enhanced ST-LSTM network of the third global enhanced ST-LSTM module, the third global enhanced ST-LSTM module is preceded by the gradient highway module. Therefore, the input of the third global enhanced ST-LSTM module is the highway state generated by the gradient highway module.

[0108] It should be understood that in the processing process of the global enhanced ST-LSTM network of the second global enhanced ST-LSTM module and the fourth global enhanced ST-LSTM module, the second global enhanced ST-LSTM module is preceded by the first global enhanced ST-LSTM module, and the fourth global enhanced ST-LSTM module is preceded by the third global enhanced ST-LSTM module. Therefore, the input of the second global enhanced ST-LSTM module and the fourth global enhanced ST-LSTM module It is a hidden state.

[0109] The radar echo image prediction model embeds a multi-source attention mechanism within the globally enhanced ST-LSTM module. By fusing the hidden state of the current layer from the previous time step with the initial hidden state of the current time step, the multi-source attention unit can simultaneously focus on local details and global information, significantly enhancing feature expression capabilities. This provides the radar echo image prediction model with richer and more accurate feature information in spatiotemporal prediction tasks and improves overall prediction performance. If processing more input sources is required, the radar echo image prediction model can be expanded to a multi-source input mode to achieve more generalized feature fusion.

[0110] Among them, an adaptive average pooling layer is used to perform adaptive global average pooling on the global enhanced spatiotemporal memory of the previous layer of the i-th time step, and then a global semantic vector is obtained through mapping through the fully connected layer. In the spatial dimension, it is expanded back to the same width and height as the global enhanced spatiotemporal memory of the previous layer of the i-th time step to obtain a global enhancement information (i.e., the output of the average pooling branch). Finally, the global enhancement information is multiplied by the adaptive weight parameter α and added element-by-element to the spatiotemporal memory obtained by the gated update to obtain the final global enhanced spatiotemporal memory.

[0111] Among them, by introducing the average pooling branch for global context enhancement in the spatiotemporal memory update process, the limitation of relying solely on the local convolution receptive field is overcome, and the radar echo image prediction model based on the globally enhanced PredRNN achieves better global consistency in complex scenes and long sequence predictions.

[0112] Among them, after the input data (sub-image or hidden state) undergoes multi-layer spatiotemporal feature extraction and information fusion, the final high-dimensional features are converted back to the same dimension as the original data (i.e., the input data) through a layer of convolutional mapping, and the residual connection technology is used to directly combine the input data with the prediction results. This can not only improve the detail recovery of the radar echo image prediction model output, but also effectively alleviate the problem of information attenuation in deep networks, ensuring that the output results are more accurate and stable.

[0113] In one embodiment, the processing of multiple sources from the attention unit is as follows:

[0114] The initial hidden state generated by the lth globally reinforced ST-LSTM module when predicting the i-th time step and the hidden state generated by the lth globally reinforced ST-LSTM module when predicting the i-1th time step are added together and then shape adjusted to obtain the adjusted hidden state; the adjusted hidden state is processed by a multi-head self-attention sub-unit to obtain the self-attention enhanced feature; the self-attention enhanced feature is added to the adjusted hidden state and then processed by a normalization layer to obtain the normalized hidden state; the normalized hidden state is processed by a feedforward neural network and added to the normalized hidden state, and then processed by a normalization layer and shape adjustment in sequence to obtain the hidden state generated by the lth globally reinforced ST-LSTM module when predicting the i-th time step.

[0115] Among them, Figure 4 The schematic diagram of the structure of the multi-source attention unit shown in the figure shows the initial hidden state generated by the l-th global reinforced ST-LSTM module when predicting at the i-th time step. and the hidden state generated by the lth globally reinforced ST-LSTM module at the i-1th time step prediction Based on the 4D tensor (e.g., [1]), the model first converts the 4D tensor into a 3D form through Shape Transform. Multi-Head Attention subunit calculations and Feedforward Neural Network (FFN) processing are then performed. Layer Normalization (LayerNorm) and residual connections are added at key steps, and finally, Shape Transform is used to restore the input shape. This design enables flexible information exchange between spatial dimensions and channel / batch dimensions, resulting in a richer and more comprehensive representation of spatiotemporal features, improving prediction accuracy and model stability.

[0116] Among them, the multi-source attention unit fully integrates the short-term memory information carried by the hidden states of adjacent time steps, strengthens the explicit correlation modeling between different moments, and enhances the network's ability to capture spatial structures and key features.

[0117] The multi-head self-attention subunit is a multi-head self-attention network. The multi-head self-attention network applies three independent linear transformations to the adjusted hidden state (i.e., the input of the multi-head self-attention subunit) to generate a query feature sequence (Query), a key feature sequence (Key), and a value feature sequence (Value). To implement the multi-head attention mechanism, the query feature sequence, key feature sequence, and value feature sequence are reshaped and transposed along the embedding dimension, dividing them into a subspaces. Each subspace corresponds to an attention head, and each subspace corresponds to a query feature subsequence, a key feature subsequence, and a value feature subsequence. Each attention head processes its corresponding query feature subsequence, key feature subsequence, and value feature subsequence respectively. For each attention head, the similarity between its corresponding query feature subsequence and the key feature subsequence is calculated, and an attention weight is obtained through softmax. The weight is then applied to the corresponding value feature subsequence to obtain the attention output of that attention head. Then, the outputs of the n attention heads are concatenated in the embedding dimension and integrated through a linear layer to obtain a fused context-enhanced feature sequence, which is used as the output of the multi-head self-attention sub-unit, namely the self-attention enhanced feature. This output is used to update the hidden state of the current layer.

[0118] In one embodiment, the radar echo image prediction model based on the global enhanced PredRNN is trained as follows:

[0119] Step 1: Obtain a sample data set containing multiple radar echo image samples, each radar echo image sample includes radar echo images of q time steps.

[0120] During the data preparation stage, the VIL (vertically integrated liquid) subset data of the target area is extracted from the SEVIR dataset. This dataset contains radar echo images of multiple time steps to ensure that a sufficiently long time series is covered. At the same time, some auxiliary information (such as timestamps and metadata) is also collected to provide complete and accurate basic data for subsequent data preprocessing and model training.

[0121] The team used the VIL (Vertically Integrated Liquid) subset of the 2017-2019 SEVIR dataset to train a radar echo image prediction model based on a global enhancement PredRNN. VIL (Vertically Integrated Liquid) is a physical quantity calculated based on radar reflectivity that represents the total amount of liquid water in all cloud layers in the vertical direction of the atmosphere. The trained radar echo image prediction model based on a global enhancement PredRNN was used to predict radar echo image trends over the next one, two, or even longer periods.

[0122] Among them, the VIL subset of 2017-2019 in the SEVIR dataset was downloaded, and multiple radar echo image samples were obtained to construct a sample dataset.

[0123] Step 2: preprocess each radar echo image sample in the sample data set, and combine the radar echo images in each radar echo image sample into a radar echo image sample sequence of fixed length in chronological order.

[0124] During the data sequence construction phase, radar echo image samples are initially processed, including noise removal and normalization. Successive radar echo images are then combined in chronological order into a fixed-length radar echo image sample sequence (for example, 20 consecutive time-step radar echo images). These radar echo image sample sequences are packaged into a data structure of shape [batch, 20, 1, img_height, img_width] and input into the block processing module for block processing. Specifically, batch represents the batch size, which indicates the number of data sequences fed into the model at a time; the sequence length is 20, indicating that each radar echo image sample sequence contains 20 frames of radar echo images; the number of channels is 1. Since the dataset used in this example is grayscale data, the number of channels for radar echo images is 1; and img_height and img_width represent the height and width of each radar echo image, respectively. This ensures that each sequence fully captures the spatiotemporal changes of the target area over time.

[0125] It should be understood that the radar echo image sample sequence is a radar echo image of 20 time steps. During the training process, radar echo images of 19 time steps are actually used. In the verification and testing stages, radar echo sub-images of 20 time steps are required for prediction result comparison. Therefore, radar echo image sample sequence requires radar echo images of 20 time steps.

[0126] Step 3: Block the radar echo image of each time step of each radar echo image sample sequence to obtain each blocked sample sequence. All blocked sample sequences constitute a processed sample data set. Each blocked sample sequence includes radar echo sub-images corresponding to the radar echo images of q time steps.

[0127] Among them, the radar echo image sample sequence is preprocessed using methods such as blocking strategy. Each radar echo image sample consists of 20 frames of radar echo images, and each frame of radar echo image is processed by blocking. For example, if the height and width of a frame of radar echo image are evenly divided into n parts, then each frame of radar echo image will be divided into n 2 radar echo sub-images, that is, each frame of radar echo image corresponds to n2 A radar echo sub-image is generated. From this, a new processed sample dataset is constructed.

[0128] In the block processing stage, each frame of radar echo image is further refined. The specific steps are as follows:

[0129] Image segmentation: The original radar echo image is evenly divided into several radar echo sub-images of equal size. Assuming that the height and width of the radar echo image are evenly divided into n parts, each image will be divided into n 2 A radar echo sub-image.

[0130] Adjust the number of channels: By expanding the channel dimension, the features of each radar echo sub-image can independently express its local spatial information. Specifically, the original single channel (grayscale image) is expanded to n 2 channels, each channel corresponds to a radar echo sub-image.

[0131] Data reshape: After the above block processing, the radar echo image sample sequence with the data structure of [batch, 20, 1, img_height, img_width] is transformed into a block sample sequence with the data structure of [batch, 20, n², img_height / n, img_width / n]. Where: batch is the batch size, which represents the number of data sequences input to the model at one time; the sequence length is 20, which means that each radar echo image sample sequence contains 20 frames of radar echo images; the number of channels after the radar echo image is expanded after block processing is n. 2 The number of channels is equal to the total number of radar echo sub-images in each radar echo image multiplied by the number of channels of the radar echo image; img_height / n and img_width / n are the height and width of each radar echo sub-image.

[0132] Step 4: Divide the processed sample data set into training set, validation set and test set.

[0133] Step 5: Use the blocked sample sequence in the training set to train the radar echo image prediction model based on the global enhancement PredRNN to obtain the trained radar echo image prediction model.

[0134] Among them, the radar echo image prediction model is trained using the divided training set, and the validation set is used for testing at the end of each training round. The training parameters of the radar echo image prediction model are adjusted according to the test results to better perform subsequent training.

[0135] In step 6, the trained radar echo image prediction model is verified using the blocked sample sequence in the validation set. The network parameters of the trained radar echo image prediction model are adjusted according to the validation results, and the training is continued until the validation results reach the convergence condition, thereby obtaining the trained radar echo image prediction model based on the globally enhanced PredRNN.

[0136] Among them, the training set is used to train the radar echo image prediction model based on the globally enhanced PredRNN, the validation set is used to verify the training results during the training process, and the network parameters are adaptively adjusted for subsequent training.

[0137] Step 7: Use the blocked sample sequence in the test set to test the trained radar echo image prediction model based on the global enhancement PredRNN to evaluate the performance of the trained radar echo image prediction model based on the global enhancement PredRNN.

[0138] Among them, after the radar echo image prediction model is trained, the divided test set is used to test the prediction performance of the radar echo image prediction model on new data.

[0139] Among them, the trained radar echo image prediction model based on global enhancement PredRNN is tested using unseen test set data. The test results are evaluated by various indicators and compared with the prediction results of the existing PredRNN model and ConvLSTM model.

[0140] Among them, the two evaluation indicators, mean square error (MSE) and structural similarity index (SSIM), are used to quantitatively analyze the complete prediction results, and the differences between the prediction results and actual observations are intuitively displayed through visual charts.

[0141] Among them, the radar echo image prediction model includes: a first globally enhanced ST-LSTM module, a second globally enhanced ST-LSTM module, a gradient highway module, a third globally enhanced ST-LSTM module and a fourth globally enhanced ST-LSTM module; the first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the gradient highway module, the third globally enhanced ST-LSTM module and the fourth globally enhanced ST-LSTM module are connected in sequence.

[0142] Among them, at each time step, the residual addition strategy is implemented to superimpose the network output with the current input, so that the model only needs to learn increments, which further stabilizes the gradient and simplifies the training process.

[0143] In one embodiment, during the training of the radar echo image prediction model based on the global enhanced PredRNN, a planned sampling unit is added before the radar echo image prediction model based on the global enhanced PredRNN.

[0144] in, Figure 5 This paper demonstrates the structural composition of a radar echo image prediction model based on a globally enhanced PredRNN for time series prediction tasks, as well as its operational workflow at different time steps during training. The model consists of four stacked layers of GEn-STLSTM (Global Enhanced Spatiotemporal LSTM) modules, with a gradient highway module inserted after the second layer to optimize gradient propagation in long sequences.

[0145] It should be understood that the basic structure of the radar echo image prediction model includes a sampling layer, a hidden layer, and an output layer. The sampling layer is used to receive input data, and the hidden layer is responsible for extracting the features of the data, that is, it is composed of the first global enhanced ST-LSTM module, the second global enhanced ST-LSTM module, the gradient highway module, the third global enhanced ST-LSTM module, and the fourth global enhanced ST-LSTM module. The output layer generates the final prediction result.

[0146] During the training process, the radar echo sub-images corresponding to the radar echo image at each time step in the blocked sample sequence (i.e., all radar echo sub-images of the radar echo image) are input into the radar echo image prediction model based on the global enhanced PredRNN to be trained in chronological order. Each training process (i.e., completing the learning and inference of the radar echo images of all time steps of a blocked sample sequence) can be divided into a learning stage and an inference stage. Taking a blocked sample sequence including radar echo sub-images of 20 time steps (i.e., real radar echo sub-image data) as an example, the radar echo sub-images of the first 10 time steps are sequentially input into the radar echo image prediction model based on the global enhanced PredRNN, so that the radar echo image prediction model based on the global enhanced PredRNN learns the features in the samples. The radar echo sub-images of the last 9 time steps are used to assist the radar echo image prediction model based on the global enhanced PredRNN in inferring and predicting the radar echo predicted sub-images (radar echo predicted sub-image data) of the last 10 time steps.

[0147] Among them, in order to clearly illustrate the information interaction between the previous and next time steps, Figure 5The figure shows the typical cases of the learning stage (learning using radar echo images from the 1st time step to the 10th time step) and the inference stage (inference using radar echo images from the 11th time step to the 19th time step). The process of learning using the radar echo image from the 10th time step and inference using the radar echo image from the 11th time step is taken as an example:

[0148] In the learning phase, the radar echo image prediction model receives the real input frame X 10 (i.e., all radar echo sub-images of the radar echo image at the 10th time step) are used as the input of the current time step. The hidden state, cell state, and spatiotemporal memory are updated in each layer of GEn-STLSTM in turn after being generated in the previous time step, and are used to generate the hidden state, cell state, and spatiotemporal memory of the current time step. Among them, the hidden state and spatiotemporal memory not only realize cross-step transmission in the time dimension (i.e., transmission from the previous time step to the current time step), but also carry out longitudinal information flow between different network layers within the same time step (i.e., cross-layer flow, such as transmission from the first layer to the second layer), thereby enhancing the radar echo image prediction model's ability to model multi-level spatial-temporal dependencies. After completing the state update of the four-layer globally enhanced ST-LSTM module, in order to improve the stability of the radar echo image prediction model and enhance the ability to restore image details, the radar echo image prediction model performs a residual connection between the output of the last layer of globally enhanced ST-LSTM module GEn-STLSTM and the original input frame (i.e., all radar echo sub-images of the real radar echo image), thereby generating the final prediction frame O. 11 (i.e., all radar echo prediction sub-images of the radar echo prediction image at the 11th time step). This residual structure helps strengthen the modeling and restoration of local features while maintaining the consistency of the overall spatial structure. This stage uses a teacher forcing strategy, meaning that the input for each time step is provided by a real radar echo image, ensuring that the model effectively learns both short-term and long-term temporal dependencies.

[0149] In the inference phase, the radar echo image prediction model enters the autoregressive prediction mode and no longer relies entirely on real data as input. At this time, the radar echo image prediction model inputs the prediction frame O generated by the previous time step. 11 The predicted frame O is sent to the scheduled sampling unit together with the real frame of the current time step (i.e., all radar echo sub-images of the radar echo image at the 11th time step). The scheduled sampling unit decides whether to use the predicted frame O according to the set scheduling probability η. 11Or the real frame is used as the input of the first layer of GEn-STLSTM network, thereby alleviating the distribution offset problem in the learning and inference stages. Afterwards, the sampled input enters the same four-layer GEn-STLSTM network structure again for state update and prediction result generation. The hidden state output of the last layer of GEn-STLSTM network is residually connected with the input of the first layer of GEn-STLSTM network to output the final predicted frame O 12 (i.e., all radar echo prediction sub-images of the radar echo prediction image at the 12th time step).

[0150] Among them, in order to understand the cross-step transmission in the time dimension and the vertical information flow between different network layers in the same time step, Figure 5 Figure 3 shows an expanded view of the information transfer over multiple time steps (i.e., the radar echo images at the 10th and 11th time steps). The structure of the radar echo image prediction model (GEn-PredRNN model) (i.e., the first globally reinforced ST-LSTM module GEn-STLSTM1, the second globally reinforced ST-LSTM module GEn-STLSTM2, the gradient highway module GHU, the third globally reinforced ST-LSTM module GEn-STLSTM3, and the fourth globally reinforced ST-LSTM module GEn-STLSTM4, which are connected in sequence) remains consistent throughout the entire sequence and shares weights (SharedParameters) across all time steps.

[0151] Among them, the fourth global enhanced spatiotemporal memory generated by GEn-STLSTM4 when learning or reasoning the radar echo image of the previous time step will be transferred to the current time step learning or reasoning and passed layer by layer, such as: when learning with the radar echo image of the 10th time step, the fourth global enhanced spatiotemporal memory M generated by GEn-STLSTM4 l=4 10 The radar echo image at the 11th time step is passed to the inference as the input of GEn-STLSTM1, and the first global enhanced spatiotemporal memory M output by GEn-STLSTM1 is l=1 11 Passed to the next layer GEn-STLSTM2, the second global enhanced spatiotemporal memory M output by GEn-STLSTM2 l=2 11 Passed to the next layer GEn-STLSTM3, the third global enhanced spatiotemporal memory M output by GEn-STLSTM3 l=3 11 Passed to the next layer GEn-STLSTM4, and finally output by GEn-STLSTM4 the fourth global enhanced spatiotemporal memory M l=4 11Passed to inference at the next time step.

[0152] Among them, the new cell state generated by each layer of GEn-STLSTM when learning or reasoning the radar echo image of the previous time step updates the cell state of each layer. For example, when reasoning with the radar echo image of the 11th time step, GEn-STLSTM1 updates the cell state of each layer based on the current cell state (that is, the first cell state C generated when learning the radar echo image of the 10th time step). l=1 10 ) Generate the first cell state C l=1 11 , update the current cell state to the first cell state C l=1 11 GEn-STLSTM1 generates a new first cell state in the next time step; similarly, GEn-STLSTM2 generates a second cell state C based on the current cell state (i.e., the radar echo image learning at the 10th time step). l=2 10 ) Generate the second cell state C l=2 11 , update the current cell state to the second cell state C l=2 11 GEn-STLSTM2 generates a new second cell state in the next time step; GEn-STLSTM3 generates a third cell state C based on the current cell state (i.e., the radar echo image learning at the 10th time step). l=3 10 ) Generate the third cell state C l=3 11 , update the current cell state to the third cell state C l=3 11 GEn-STLSTM3 generates a new third cell state in the next time step; GEn-STLSTM4 generates the fourth cell state C based on the current cell state (i.e., the radar echo image learning at the 10th time step). l=4 10 ) Generate the fourth cell state C l =4 11 , update the current cell state to the fourth cell state C l=4 11 Used by GEn-STLSTM4 to generate a new fourth cell state at the next time step.

[0153] Among them, when learning or reasoning about the radar echo image of the previous time step, the hidden state generated by each layer of GEn-STLSTM updates the hidden state of each layer and transmits it layer by layer. For example, when reasoning with the radar echo image of the 11th time step, GEn-STLSTM1 updates the hidden state of each layer based on the current hidden state (that is, the first hidden state H generated when learning the radar echo image of the 10th time step). l=1 10 ) and the fourth hidden state H generated when learning the radar echo image at the 10th time step l=4 10 Generate the first hidden state H l=1 11 , update the current hidden state to the first hidden state H l=1 11 Use GEn-STLSTM1 to generate a new first hidden state in the next time step and convert the first hidden state H l=1 11 Input GEn-STLSTM2; GEn-STLSTM2 is based on the first hidden state H l=1 11 and the current hidden state (i.e., the second hidden state H generated when learning the radar echo image at the 10th time step l=2 10 ) Generate the second hidden state H l=2 11 , update the current hidden state to the second hidden state H l=2 11 Use GEn-STLSTM2 to generate a new second hidden state in the next time step and convert the second hidden state H l=2 11 Input GHU; GHU is based on the second hidden state H l=2 11 The third hidden state H generated by GEn-STLSTM3 when learning with the radar echo image at the 10th time step l=3 10 Generate highway state R 11 , change the highway state R 11 Enter GEn-STLSTM3; GEn-STLSTM3 is based on highway status R 11 and the current hidden state (i.e., the third hidden state H generated when learning the radar echo image at the 10th time step) l=3 10 ) Generate the third hidden state H l=3 11 , update the current hidden state to the third hidden state H l=3 11Use GEn-STLSTM3 to generate a new third hidden state in the next time step and convert the third hidden state H l=3 11 Input GEn-STLSTM4; GEn-STLSTM4 is based on the third hidden state H l=3 11 and the current hidden state (i.e., the fourth hidden state H generated when learning the radar echo image at the 10th time step) l=4 10 ) Generate the fourth hidden state H l=4 11 , update the current hidden state to the fourth hidden state H l=4 11 Use GEn-STLSTM4 to generate a new fourth hidden state in the next time step and convert the fourth hidden state H l =4 11 Output.

[0154] As a result, the hidden state, cell state, global enhanced spatiotemporal memory, and highway state are transmitted layer by layer between time steps to form a complete spatiotemporal memory chain, thereby supporting the radar echo image prediction model to model and predict the complex dynamic changes in the radar echo image sequence.

[0155] In one embodiment, a radar echo image prediction model based on a global enhanced PredRNN is trained using a block-based sample sequence in a training set to obtain a trained radar echo image prediction model, including:

[0156] The radar echo sub-image of the first time step of the blocked sample sequence is input into the radar echo image prediction model based on the global enhanced PredRNN to be trained, and the radar echo prediction sub-image of the second time step is output; the radar echo sub-image of the second time step of the blocked sample sequence is input into the radar echo image prediction model based on the global enhanced PredRNN to be trained, and the radar echo prediction sub-image of the third time step is output; and so on, until the radar echo sub-image of the first time step of the blocked sample sequence is input into the radar echo image prediction model based on the global enhanced PredRNN to be trained, and the radar echo prediction sub-image of the third time step is output. The radar echo sub-image of the time step is input into the radar echo image prediction model based on the global reinforcement PredRNN to be trained, and the output is the + 1 time step radar echo prediction sub-image; then the first + 1 time step radar echo sub-image and the The radar echo prediction sub-image of the + 1 time step is input into the planned sampling unit, so that the planned sampling unit determines the first + 1 time step radar echo sub-image or radar echo prediction sub-image input to be trained based on global reinforcement PredRNN radar echo image prediction model, output the future +2 time steps; and so on, the radar echo sub-image of the q-1th time step and the radar echo prediction sub-image of the q-1th time step of the blocked sample sequence are input into the planned sampling unit, so that the planned sampling unit determines, according to the preset scheduling probability η, to input the radar echo sub-image of the q-1th time step or the radar echo prediction sub-image into the radar echo image prediction model based on the global reinforcement PredRNN to be trained, and outputs the radar echo prediction sub-image of the qth time step.

[0157] Among them, during the training process of the radar echo image prediction model, a planned sampling strategy is first applied. By dynamically adjusting the scheduling probability η, it is decided whether to use real data (i.e., radar echo sub-image) input at each time step or rely on the radar echo image prediction model's own predicted output (i.e., radar echo predicted sub-image). This strategy mainly relies on real data in the early stage, and as the number of iterations increases, η gradually decreases, so that the training data gradually changes from real input to radar echo image prediction model prediction data, thereby effectively alleviating the distribution inconsistency problem between learning and inference, and laying the foundation for the subsequent autoregressive prediction of the radar echo image prediction model.

[0158] The initial value of the preset scheduling probability η is 1, and a hyperparameter μ is set for adjusting the scheduling probability η. After each iterative training is completed, μ is reduced based on the current scheduling probability η.

[0159] The value of the hyperparameter μ can be set according to actual conditions, such as 0.001, 0.002, 0.003, etc.

[0160] In an example, the value of the hyperparameter μ is 0.002. After each iterative training is completed, the scheduling probability η is reduced by 0.002 based on the current scheduling probability η. Taking the reduction of the scheduling probability η after the first iterative training as an example, the scheduling probability η before adjustment is 1, and the scheduling probability η after adjustment is 0.998.

[0161] This radar echo image prediction method based on a globally reinforced PredRNN is designed for spatiotemporal radar echo image prediction tasks. Using the VIL (vertically integrated liquid) data of the target area in the SEVIR dataset as the research object, it constructs an efficient and stable deep learning framework centered around key steps, including data preparation, data preprocessing, model learning and inference, and result post-processing and evaluation. The network design incorporates key technologies such as a planned sampling strategy, a globally reinforced ST-LSTM module, a gradient highway module, and a dual-source attention mechanism, effectively improving the model's accuracy and stability in long-time series prediction. Furthermore, the combination of patching and residual connection techniques further enhances the ability to capture and recover detailed features. Ultimately, this application enables comprehensive analysis of prediction results using multiple evaluation metrics and visualization methods, providing strong support for applications in weather forecasting, river basin flood control, disaster warning, and other fields.

[0162] This radar echo image prediction method based on a globally enhanced PredRNN introduces a gradient highway module, effectively alleviating the vanishing or exploding gradient problem in deep recurrent networks, thereby improving its ability to capture long-term dependencies. It also employs multi-source attention to enhance the model's comprehensive understanding of spatial structure and temporal dynamics. Furthermore, it utilizes a global context enhancement mechanism to enrich spatiotemporal memory and improve the model's overall expressive power. Compared to traditional prediction models, this method significantly improves the rapid degradation of predicted image quality over time, providing more reliable technical support for accurate radar echo image prediction and weather forecasting.

[0163] In one embodiment, the performance comparison of three models, ConvLSTM, PredRNN, and GEn-PredRNN (radar echo image prediction model of this application) is as follows:

[0164] like Figure 6 The figure shows a schematic diagram of the MSE curve changing over time. The figure uses time steps as the horizontal axis and mean square error (MSE) as the vertical axis, showing the changes in the prediction errors of the three models as the prediction time passes. The results show that GEn-PredRNN always has the lowest MSE value at most time steps, especially in the later moments, where it can maintain a lower error, indicating that it has higher accuracy in long-term predictions. PredRNN is second, but the error rises significantly in the middle and late stages, and then falls slightly. The curve of ConvLSTM is the highest, which shows its inadequacy in capturing long-term dependencies.

[0165] like Figure 7The diagram of the SSIM curve over time shown in the figure intuitively shows the comparison of the prediction quality of the three models, ConvLSTM, PredRNN, and GEn-PredRNN, at multiple future moments by marking the prediction time step on the horizontal axis and the structural similarity (SSIM) value on the vertical axis. From the overall trend of the curve, it can be seen that the SSIM value of GEn-PredRNN always remains at a relatively high level, indicating that it can better preserve the structural details of the image at different prediction moments, followed by PredRNN. The curve of ConvLSTM shows a significant decline in the later moments, reflecting that the model's ability to preserve the structure for long-term prediction is relatively insufficient.

[0166] like Figure 8 The diagram showing the visual comparison of prediction results shows the visual difference between the prediction results of different models and the ground truth at the same time. Each column represents a prediction time step or a real frame, and each row is the output of the real observation sequence, ConvLSTM, PredRNN, and GEn-PredRNN. It can be clearly seen that GEn-PredRNN can still generate images that are closer to the real distribution at later times, and the detailed areas are less blurred or distorted. In contrast, the results of ConvLSTM have a certain degree of distortion in subsequent time steps. Although the prediction of PredRNN is better than that of ConvLSTM, it is still inferior to GEn-PredRNN in detail restoration in key areas.

[0167] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0168] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0169] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A radar echo image prediction method based on global enhanced PredRNN, characterized in that: The radar echo image prediction method based on the global enhanced PredRNN includes: Get the radar echo image of the target area from the tmth time step to the tth time step, where t is the current time step and m is a positive integer; Preprocessing the radar echo images and combining the radar echo images into a radar echo image sequence of fixed length in chronological order; Blocking the radar echo image of each time step of the radar echo image sequence to obtain a block-based radar echo image sequence, wherein the block-based radar echo image sequence includes a radar echo sub-image of each time step; Inputting the radar echo sub-images of each time step into the trained radar echo image prediction model based on global enhancement PredRNN in chronological order for prediction, and obtaining radar echo prediction images from the future t+1 time step to t+m+1 time step; The radar echo image prediction model includes: a first globally enhanced ST-LSTM module, a second globally enhanced ST-LSTM module, a gradient highway module, a third globally enhanced ST-LSTM module, and a fourth globally enhanced ST-LSTM module; The first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the gradient highway module, the third globally enhanced ST-LSTM module and the fourth globally enhanced ST-LSTM module are connected in sequence.

2. The radar echo image prediction method based on global enhanced PredRNN according to claim 1 is characterized in that: The radar echo sub-images of each time step are sequentially input into the trained radar echo image prediction model based on the global enhancement PredRNN in chronological order for prediction, to obtain the radar echo prediction images from the future t+1 time step to t+m+1 time step, including: Input the radar echo sub-image at the tmth time step into the trained radar echo image prediction model based on the global reinforcement PredRNN, and output the radar echo prediction sub-image at the t-m+1th time step; Input the radar echo sub-image at the t-m+1th time step into the trained radar echo image prediction model based on the global reinforcement PredRNN, and output the radar echo prediction sub-image at the t-m+2th time step; This process is repeated until the radar echo sub-image at the t-m+mth time step is input into the trained radar echo image prediction model based on the global reinforcement PredRNN, and the radar echo prediction sub-image at the future t+1th time step is output. Then, the radar echo prediction sub-image at the t+1 time step is input into the trained radar echo image prediction model based on the global reinforcement PredRNN, and the radar echo prediction sub-image at the future t+2 time step is output; Similarly, the radar echo prediction sub-image at the t+mth time step is input into the trained radar echo image prediction model based on the global reinforcement PredRNN, and the radar echo prediction sub-image at the future t+m+1th time step is output; The radar echo prediction sub-images from the t+1th time step to the t+m+1th time step are spliced ​​to obtain the radar echo prediction image from the t+1th time step to the t+m+1th time step in the future.

3. The radar echo image prediction method based on global enhanced PredRNN according to claim 2 is characterized in that: The radar echo image prediction model predicts the radar echo prediction sub-image at the i-th time step in the following manner: The sub-image of the i-1th time step input to the radar echo image prediction model and the fourth global enhanced spatiotemporal memory generated during the prediction of the i-1th time step are input to the first globally enhanced ST-LSTM module. The first globally enhanced ST-LSTM module generates the first hidden state, the first cell state, the fourth global enhanced spatiotemporal memory, and the sub-image of the i-1th time step during the prediction of the i-1th time step, wherein: , where i is to At any time step in , the sub-image of the i-1th time step is the radar echo sub-image of the i-1th time step. to At any time step in , the sub-image of the i-th time step is the radar echo prediction sub-image of the i-1-th time step; Inputting the first hidden state at the time of the i-th time step prediction and the first global enhanced spatiotemporal memory into the second globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module generates the second hidden state, the second cell state and the second global enhanced spatiotemporal memory at the time of the i-th time step prediction according to the second hidden state, the second cell state generated at the time of the i-1-th time step prediction, the first hidden state at the time of the i-th time step prediction and the first global enhanced spatiotemporal memory; Inputting the second hidden state generated during the prediction of the i-th time step and the third hidden state generated during the prediction of the i-1-th time step into the gradient highway module, the gradient highway module generating a highway state during the prediction of the i-th time step based on the second hidden state generated during the prediction of the i-th time step and the third hidden state generated during the prediction of the i-1-th time step; Inputting the highway state and the second global enhanced spatiotemporal memory at the time of the i-th time step prediction into the third globally enhanced ST-LSTM module, the third globally enhanced ST-LSTM module generating a third hidden state, a third cell state and a third global enhanced spatiotemporal memory at the time of the i-th time step prediction based on the third hidden state, the third cell state, the highway state at the time of the i-th time step prediction and the second global enhanced spatiotemporal memory; Inputting the third hidden state and the third global enhanced spatiotemporal memory at the time of the i-th time step prediction into the fourth globally enhanced ST-LSTM module, the fourth globally enhanced ST-LSTM module generates a fourth hidden state, a fourth cell state and a fourth global enhanced spatiotemporal memory at the time of the i-th time step prediction according to the fourth hidden state, the fourth cell state generated at the time of the i-1-th time step prediction, the third hidden state and the third global enhanced spatiotemporal memory at the time of the i-th time step prediction; The fourth hidden state at the time of prediction of the i-th time step is added element by element to the sub-image of the i-1-th time step to obtain the radar echo prediction sub-image of the i-th time step.

4. The radar echo image prediction method based on global enhanced PredRNN according to claim 3 is characterized in that: The step of inputting the second hidden state generated during the prediction of the i-th time step and the third hidden state generated during the prediction of the i-1-th time step into the gradient highway module, wherein the gradient highway module generates a highway state during the prediction of the i-th time step according to the second hidden state generated during the prediction of the i-th time step and the third hidden state generated during the prediction of the i-1-th time step, comprises: The second hidden state generated during the prediction of the i-th time step is processed by a tanh activation function and then added to the third hidden state generated during the prediction of the i-1-th time step to obtain a candidate tensor; The second hidden state generated during the prediction of the i-th time step is processed by a σ activation function and then added to the third hidden state generated during the prediction of the i-1-th time step to obtain a first gating weight and a second gating weight; weighting the candidate tensor according to the first gating weight to obtain a weighted candidate tensor; weighting the third hidden state generated during the prediction of the i-1th time step according to the second gating weight to obtain a weighted third hidden state; The weighted candidate tensor and the weighted third hidden state are subjected to feature fusion to generate a highway state at the time of prediction of the i-th time step.

5. The radar echo image prediction method based on global enhanced PredRNN according to claim 3 is characterized in that: The first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the third globally enhanced ST-LSTM module, and the fourth globally enhanced ST-LSTM module are globally enhanced ST-LSTM networks with the same structure. The processing process of the globally enhanced ST-LSTM network is expressed as follows: ， ， ， ， ， ， ， ， ， ， ， ， ， ， ， ； in, To map the input into a convolution of 7 gate vectors, To map the input into a convolution of 4 gate vectors, To map the input into a convolution of 3 gate vectors, l is the number of the globally enhanced ST-LSTM module, , The hidden state, highway state, or sub-image of the lth globally reinforced ST-LSTM module is input for the prediction of the i-th time step, is the hidden state generated by the lth globally reinforced ST-LSTM module when predicting the i-1th time step, The global enhanced spatiotemporal memory generated by the l-1th globally enhanced ST-LSTM module when predicting the i-th time step, For those from The input gate is pre-activated, For those from The forget gate pre-activation, For those from Candidate memory, is the global memory branch version of the input gate, is the global memory branch version of the forget gate, is the candidate memory of the global memory branch, is element-wise multiplication, for Contribution to the output gate, For those from The pre-activation of the input gate, For those from The pre-activation of the forget gate, For those from Candidate memory, for Contribution to the output gate, For those from The pre-activation of the input gate, For those from The pre-activation of the forget gate, For those from The candidate memory of the input gate, is the output of the input gate, is the Sigmoid activation function, is the output of the forget gate, is the forget gate bias, is the output of the input modulation gate, is the hyperbolic tangent activation function, is the cell state generated by the lth globally reinforced ST-LSTM module when predicting at the i-th time step, is the cell state generated by the lth globally reinforced ST-LSTM module when predicting the i-1th time step, is the output of the input gate of the global enhanced memory branch, is the output of the forget gate of the global enhanced memory branch, is the output of the input modulation gate of the global enhanced memory branch, To enhance global information, is the adaptive average pooling layer, is the fully connected layer, To broadcast the 1*1 feature back to The same space size, is the globally enhanced spatiotemporal memory generated by the lth globally enhanced ST-LSTM module when predicting at the i-th time step, is the adaptive weight parameter, is the comprehensive memory tensor, To perform channel dimension stitching, is the output of the output gate, Indicates that Compression fusion obtains the features required to drive the output gate, Indicates that Compression fusion obtains the features required to generate hidden states, Most of them come from attention units. is the initial hidden state generated by the lth globally reinforced ST-LSTM module when predicting at the i-th time step, is the hidden state generated by the lth globally reinforced ST-LSTM module when predicting at the i-th time step.

6. The radar echo image prediction method based on global enhanced PredRNN according to claim 5 is characterized in that: The processing process of the multi-source attention unit is as follows: The initial hidden state generated by the lth globally reinforced ST-LSTM module at the i-th time step prediction and the hidden state generated by the lth globally reinforced ST-LSTM module at the i-1th time step prediction are added together and then shape-adjusted to obtain the adjusted hidden state; Processing the adjusted hidden state through a multi-head self-attention subunit to obtain a self-attention enhanced feature; Adding the self-attention enhancement feature to the adjusted hidden state and then processing it through a normalization layer to obtain a normalized hidden state; The normalized hidden state is processed by a feedforward neural network and added to the normalized hidden state, and then processed by a normalization layer and a shape adjustment layer in sequence to obtain the hidden state generated by the lth globally enhanced ST-LSTM module when predicting the i-th time step.

7. The radar echo image prediction method based on global enhanced PredRNN according to claim 1, characterized in that: The training method of the radar echo image prediction model based on the global enhanced PredRNN is: Acquire a sample data set comprising a plurality of radar echo image samples, each of the radar echo image samples comprising radar echo images of q time steps; Preprocessing each of the radar echo image samples in the sample data set, and combining the radar echo images in each of the radar echo image samples into a radar echo image sample sequence of fixed length in chronological order; Block-processing the radar echo image of each time step of each radar echo image sample sequence to obtain each blocked sample sequence, wherein all blocked sample sequences constitute a processed sample data set, and each blocked sample sequence includes radar echo sub-images corresponding to the radar echo images of q time steps; Dividing the processed sample data set into a training set, a validation set, and a test set; Using the blocked sample sequence in the training set to train the radar echo image prediction model based on the global enhancement PredRNN to obtain a trained radar echo image prediction model; Verifying the trained radar echo image prediction model using the block-based sample sequence in the validation set, adjusting the network parameters of the trained radar echo image prediction model according to the validation result, and continuing training until the validation result reaches a convergence condition, thereby obtaining a trained radar echo image prediction model based on the global enhanced PredRNN; The trained radar echo image prediction model based on the global enhanced PredRNN is tested using the blocked sample sequence in the test set to evaluate the performance of the trained radar echo image prediction model based on the global enhanced PredRNN.

8. The radar echo image prediction method based on global enhanced PredRNN according to claim 7 is characterized in that: The method of using the blocked sample sequence in the training set to train the radar echo image prediction model based on the global enhanced PredRNN to obtain the trained radar echo image prediction model includes: Inputting the radar echo sub-image of the first time step of the block sample sequence into the radar echo image prediction model based on the global enhanced PredRNN to be trained for training, and outputting the radar echo prediction sub-image of the second time step; Inputting the radar echo sub-image of the second time step of the block sample sequence into the radar echo image prediction model based on the global enhanced PredRNN to be trained, and outputting the radar echo prediction sub-image of the third time step; And so on, until the first The radar echo sub-image of the time step is input into the radar echo image prediction model based on the global reinforcement PredRNN to be trained, and the output is the + 1 time step radar echo prediction sub-image; Then the first + 1 time step radar echo sub-image and the The radar echo prediction sub-image of the + 1 time step is input into the planned sampling unit, so that the planned sampling unit determines the first + 1 time step radar echo sub-image or radar echo prediction sub-image input to be trained based on global reinforcement PredRNN radar echo image prediction model, output the future +2 time step radar echo prediction sub-image; Similarly, the radar echo sub-image of the q-1th time step and the radar echo prediction sub-image of the q-1th time step of the blocked sample sequence are input into the planned sampling unit, so that the planned sampling unit determines, according to the preset scheduling probability η, to input the radar echo sub-image of the q-1th time step or the radar echo prediction sub-image into the radar echo image prediction model based on the global enhancement PredRNN to be trained, and outputs the radar echo prediction sub-image of the qth time step.

Images