Radar echo image prediction method based on global enhanced PredRNN

Abstract

The invention relates to a radar echo image prediction method based on a global enhanced PredRNN. The method comprises the following steps: acquiring a radar echo image from a (t-m) th time step to a t time step of a target area; preprocessing each radar echo image, and combining each radar echo image into a radar echo image sequence with a fixed length according to a time sequence; performing block processing on the radar echo image of each time step of the radar echo image sequence to obtain a radar echo sub-image of each time step; the radar echo sub-images of each time step are sequentially input into the trained radar echo image prediction model based on the global enhanced PredRNN according to the time sequence for prediction, and radar echo prediction images from the (t + 1) th time step to the (t + m + 1) th time step in the future are obtained, so that the capturing capability for long-time-interval dependence is remarkably improved; therefore, the extrapolation prediction precision of the radar echo image is improved.

Description

Technical Field

[0001] This application relates to the technical field of meteorological prediction based on deep learning, and particularly to a method for predicting radar echo images based on global reinforcement PredRNN. Background Art

[0002] In recent years, the demand for real-time analysis and prediction of radar echo images in the field of meteorological forecasting has been increasing continuously. As important data reflecting the spatial distribution of atmospheric precipitation and other weather phenomena, the spatio-temporal dynamic characteristics of radar echo images are complex and affected by various non-linear factors. Traditional statistical models and physical models often have difficulty in accurately capturing the internal relationship between local details and global structure when dealing with these highly non-linear and time-varying image data, thus limiting the forecasting accuracy and timeliness.

[0003] With the development of deep learning technology, significant achievements have been made in spatio-temporal sequence prediction tasks based on recurrent neural network (RNN), long short-term memory network (LSTM) and their improved models (such as PredRNN). These methods have improved the ability to capture dynamic information to a certain extent, but there are still problems such as gradient disappearance or explosion in long sequence prediction, and it is difficult to effectively transmit long-range information, resulting in low extrapolation prediction accuracy of current radar echo images. Summary of the Invention

[0004] Based on this, it is necessary to provide a method for predicting radar echo images based on global reinforcement PredRNN that can improve the extrapolation prediction accuracy of radar echo images for the above technical problems.

[0005] A method for predicting radar echo images based on global reinforcement PredRNN, the method for predicting radar echo images based on global reinforcement PredRNN includes: Obtain radar echo images of the target area from the (t - m)-th time step to the t-th time step, where t is the current time step and m is a positive integer; Preprocess each of the radar echo images, and combine each of the radar echo images into a radar echo image sequence with a fixed length in chronological order; Perform block processing on the radar echo images at each time step of the radar echo image sequence to obtain a blocked radar echo image sequence, and the blocked radar echo image sequence includes radar echo sub-images at each time step; Input the radar echo sub-images at each time step into a trained radar echo image prediction model based on global reinforcement PredRNN in chronological order for prediction, and obtain radar echo prediction images for the (t + 1)-th time step to the (t + m + 1)-th time step in the future; Among them, the radar echo image prediction model includes: a first globally enhanced ST-LSTM module, a second globally enhanced ST-LSTM module, a gradient highway module, a third globally enhanced ST-LSTM module, and a fourth globally enhanced ST-LSTM module; The first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the gradient highway module, the third globally enhanced ST-LSTM module, and the fourth globally enhanced ST-LSTM module are connected in sequence.

[0006] The above radar echo image prediction method based on globally enhanced PredRNN obtains the radar echo images at the (t - m)-th time step to the t-th time step of the target area; preprocesses each of the radar echo images, and combines each of the radar echo images into a radar echo image sequence with a fixed length in chronological order; performs block processing on the radar echo image at each time step of the radar echo image sequence to obtain the radar echo sub-images at each time step; inputs the radar echo sub-images at each time step into the trained radar echo image prediction model based on globally enhanced PredRNN in chronological order for prediction, to obtain the radar echo prediction images at the (t + 1)-th time step to the (t + m + 1)-th time step in the future. Among them, the radar echo image prediction model includes: a first globally enhanced ST-LSTM module, a second globally enhanced ST-LSTM module, a gradient highway module, a third globally enhanced ST-LSTM module, and a fourth globally enhanced ST-LSTM module; the first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the gradient highway module, the third globally enhanced ST-LSTM module, and the fourth globally enhanced ST-LSTM module are connected in sequence. Thus, a gradient highway unit is introduced into the radar echo image prediction model, effectively alleviating the problem of gradient vanishing or explosion in deep recurrent networks, significantly improving the ability to capture long-term dependencies, and thus improving the extrapolation prediction accuracy of radar echo images. Description of the Drawings

[0007] Figure 1 It is a schematic flowchart of the radar echo image prediction method based on globally enhanced PredRNN in an embodiment; Figure 2 It is a schematic structural diagram of the gradient highway (Gradient-Highway Unit, GHU) module in an embodiment; Figure 3 It is a schematic structural diagram of the globally enhanced ST-LSTM network in an embodiment; Figure 4Schematic diagram of the multi-source self-attention unit in one embodiment; Figure 5 Schematic diagram of the operation process of the radar echo image prediction model at different time steps during training in one embodiment; Figure 6 Schematic diagram of the MSE curve changing with time in one embodiment; Figure 7 Schematic diagram of the SSIM curve changing with time in one embodiment; Figure 8 Schematic diagram of the visual comparison of the prediction results in one embodiment. Detailed implementation manners

[0008] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0009] In one embodiment, as Figure 1 shown, a radar echo image prediction method based on global reinforcement PredRNN is provided, including the following steps: Step S220: Obtain the radar echo images of the target area from the (t - m)-th time step to the t-th time step, where t is the current time step and m is a positive integer.

[0010] Step S240: Preprocess each radar echo image, and combine the radar echo images in chronological order into a radar echo image sequence with a fixed length.

[0011] Among them, the preprocessing includes operations such as noise removal and normalization.

[0012] It should be understood that the radar echo image sequence will be packed into a data structure with a shape of [batch, m + 1, 1, img_height, img_width] and input into the block processing module for block processing. Specifically: Batch is the batch size, indicating the number of radar echo image sequences input into the radar echo image prediction model at one time; the sequence length of the radar echo image sequence is m + 1, that is, each radar echo image sequence contains m + 1 frames of radar echo images; the number of channels of the radar echo image is 1; img_height and img_width respectively represent the height and width dimensions of each radar echo image.

[0013] Among them, using a radar echo image sequence with a fixed length can ensure that each sequence can fully capture the spatio-temporal change information of the target area within a period of time.

[0014] Among them, the block processing module is a module that implements block processing on the radar echo images at each time step of the radar echo image sequence to obtain a sequence of radar echo images after block processing. The sequence of radar echo images after block processing includes radar echo sub-images at each time step.

[0015] Step S260: Perform block processing on the radar echo images at each time step of the radar echo image sequence to obtain a sequence of radar echo images after block processing. The sequence of radar echo images after block processing includes radar echo sub-images at each time step.

[0016] Among them, block processing is to evenly divide the radar echo image at each time step into several radar echo sub-images of equal size. Assume that the height and width of the radar echo image are respectively evenly divided into n parts. Then, the radar echo image at each time step will be divided into n 2 radar echo sub-images, and n 2 radar echo sub-images are the radar echo sub-images at each time step.

[0017] It should be understood that the sequence of radar echo images input to the block processing module is a data structure with a shape of [batch, m + 1, 1, img_height, img_width]. After block processing, the data structure of the sequence of radar echo images after block processing becomes [batch, m + 1, n², img_height / n, img_width / n]. Among them, the number of channels after the radar echo image is unfolded by block processing is n 2 , and this number of channels is equal to the total number of radar echo sub-images of each radar echo image multiplied by the number of channels of the radar echo image; img_height / n and img_width / n: the height and width of each radar echo sub-image.

[0018] Step S280: Input the radar echo sub-images at each time step into the trained radar echo image prediction model of PredRNN based on global reinforcement in chronological order to obtain radar echo prediction images for the future time steps from t + 1 to t + m + 1.

[0019] Among them, the radar echo image prediction model includes: the first globally reinforced ST-LSTM module, the second globally reinforced ST-LSTM module, the gradient highway module, the third globally reinforced ST-LSTM module, and the fourth globally reinforced ST-LSTM module; the first globally reinforced ST-LSTM module, the second globally reinforced ST-LSTM module, the gradient highway module, the third globally reinforced ST-LSTM module, and the fourth globally reinforced ST-LSTM module are connected in sequence.

[0020] Among them, the input radar echo sub-images are processed by block division, and all predicted results also exist in the form of blocks. Therefore, the predicted sub-images of each radar echo can be reassembled into a complete radar echo image through the block restoration algorithm.

[0021] Among them, the radar echo image prediction model based on globally enhanced PredRNN changes the computational module ST-LSTM of the existing PredRNN model, enhances the global context of spatio-temporal memory in the computational module, and integrates multi-source self-attention units inside the computational module to form a globally enhanced ST-LSTM (Global Enhanced SpatioTemporal LSTM, GEn-STLSTM) module, changing the original model architecture of the existing PredRNN model, adding a Gradient Highway Unit (GHU) module and residual connections to construct a radar echo image prediction model based on globally enhanced PredRNN, that is, the GEn-PredRNN model.

[0022] Among them, at each time step, the residual addition strategy is implemented to superimpose the network output and the current input, significantly alleviating the current situation where the prediction performance drops sharply as the number of time steps increases, providing a robust and consistent technical support for the extrapolation prediction of radar echo images, especially suitable for real-time forecasting in complex meteorological scenarios.

[0023] The above radar echo image prediction method based on global reinforcement PredRNN includes: obtaining radar echo images from the (t - m)-th time step to the t-th time step of the target area; preprocessing each radar echo image and combining them into a radar echo image sequence with a fixed length in chronological order; dividing the radar echo image at each time step of the radar echo image sequence into blocks to obtain radar echo sub-images at each time step; sequentially inputting the radar echo sub-images at each time step into the trained radar echo image prediction model based on global reinforcement PredRNN for prediction to obtain radar echo prediction images from the (t + 1)-th time step to the (t + m + 1)-th time step in the future. The radar echo image prediction model includes: a first globally reinforced ST-LSTM module, a second globally reinforced ST-LSTM module, a gradient highway module, a third globally reinforced ST-LSTM module, and a fourth globally reinforced ST-LSTM module; the first globally reinforced ST-LSTM module, the second globally reinforced ST-LSTM module, the gradient highway module, the third globally reinforced ST-LSTM module, and the fourth globally reinforced ST-LSTM module are connected in sequence. Thus, a gradient highway unit is introduced into the radar echo image prediction model, effectively alleviating the problem of gradient vanishing or explosion in deep recurrent networks, significantly improving the ability to capture long-term dependencies, and thereby improving the extrapolation prediction accuracy of radar echo images.

[0024] In one embodiment, sequentially inputting the radar echo sub-images at each time step into the trained radar echo image prediction model based on global reinforcement PredRNN for prediction to obtain radar echo prediction images from the (t + 1)-th time step to the (t + m + 1)-th time step in the future includes: Input the radar echo sub - image at the \(t - m\) time step into the trained radar echo image prediction model of PredRNN based on global reinforcement to output the radar echo prediction sub - image at the \(t - m+1\) time step; input the radar echo sub - image at the \(t - m + 1\) time step into the trained radar echo image prediction model of PredRNN based on global reinforcement to output the radar echo prediction sub - image at the \(t - m+2\) time step; and so on, until the radar echo sub - image at the \(t - m + m\) time step is input into the trained radar echo image prediction model of PredRNN based on global reinforcement to output the radar echo prediction sub - image at the future \(t + 1\) time step; then input the radar echo prediction sub - image at the \(t + 1\) time step into the trained radar echo image prediction model of PredRNN based on global reinforcement to output the radar echo prediction sub - image at the future \(t + 2\) time step; and so on, then input the radar echo prediction sub - image at the \(t + m\) time step into the trained radar echo image prediction model of PredRNN based on global reinforcement to output the radar echo prediction sub - image at the future \(t + m+1\) time step; splice the radar echo prediction sub - images from the \(t + 1\) time step to the \(t + m+1\) time step to obtain the radar echo prediction image from the future \(t + 1\) time step to the \(t + m+1\) time step.

[0025] Among them, the radar echo sub - image at each input time step is obtained through block processing, and all prediction results also exist in block form. Therefore, it is necessary to use a block restoration algorithm to splice each radar echo prediction sub - image into a complete radar echo prediction image.

[0026] It should be understood that although the radar echo sub - image at the \(t - m\) time step can predict the radar echo prediction sub - image at the \(t - m+1\) time step, the radar echo sub - image at the \(t - m + 1\) time step can predict the radar echo prediction sub - image at the \(t - m+2\) time step, and so on, until the radar echo sub - image at the \(t - 1\) time step predicts the radar echo prediction sub - image at the \(t\) time step, but what needs to be predicted is the radar echo prediction image from the future \(t + 1\) time step to the \(t + m+1\) time step. Therefore, the radar echo sub - images from the \(t - m\) time step to the \(t\) time step in the prediction are useless data and can be not output as the final prediction result. The radar echo sub - images from the \(t - m\) time step to the \(t\) time step in the prediction are used to enable the radar echo image prediction model to capture key features, thereby improving the extrapolation prediction accuracy of the radar echo image.

[0027] In one example, the radar echo prediction sub-images from the (t + 1)-th time step to the (t + m + 1)-th time step are stitched together to obtain the radar echo prediction image for the future from the (t + 1)-th time step to the (t + m + 1)-th time step. Specifically: After the radar echo prediction sub-image of the (t + 1)-th future time step is stitched through the block restoration algorithm, the radar echo prediction image of the (t + 1)-th future time step is obtained. After the radar echo prediction sub-image of the (t + 2)-th future time step is stitched through the block restoration algorithm, the radar echo prediction image of the (t + 2)-th future time step is obtained, and so on. After the radar echo prediction sub-image of the (t + m + 1)-th future time step is stitched through the block restoration algorithm, the radar echo prediction image of the (t + m + 1)-th future time step is obtained.

[0028] In one embodiment, the way for the radar echo image prediction model to predict the radar echo prediction sub-image at the i-th time step is as follows: The sub-image at the (i - 1)-th time step input to the radar echo image prediction model and the fourth global enhanced spatio-temporal memory generated during the prediction at the (i - 1)-th time step are input into the first globally enhanced ST-LSTM module. The first globally enhanced ST-LSTM module generates the first hidden state, the first cell state, and the first global enhanced spatio-temporal memory during the prediction at the i-th time step based on the first hidden state, the first cell state, the fourth global enhanced spatio-temporal memory, and the image at the (i - 1)-th time step generated during the prediction at the (i - 1)-th time step. Among them, , when i is to any one of the time steps, the sub-image at the (i - 1)-th time step is the radar echo sub-image at the (i - 1)-th time step. When i is to any one of the time steps, the sub-image at the i-th time step is the radar echo prediction sub-image at the (i - 1)-th time step.

[0029] The first hidden state and the first global enhanced spatio-temporal memory during the prediction at the i-th time step are input into the second globally enhanced ST-LSTM module. The second globally enhanced ST-LSTM module generates the second hidden state, the second cell state, and the second global enhanced spatio-temporal memory during the prediction at the i-th time step based on the second hidden state, the second cell state, the first hidden state, and the first global enhanced spatio-temporal memory during the prediction at the (i - 1)-th time step.

[0030] The second hidden state generated during the prediction at the i-th time step and the third hidden state generated during the prediction at the (i - 1)-th time step are input into the gradient highway module. The gradient highway module generates the highway state during the prediction at the i-th time step based on the second hidden state generated during the prediction at the i-th time step and the third hidden state generated during the prediction at the (i - 1)-th time step.

[0031] The highway state and the second globally enhanced spatio-temporal memory at the i-th time step prediction are input into the third globally reinforced ST-LSTM module. The third globally reinforced ST-LSTM module generates the third hidden state, the third cell state, and the third globally enhanced spatio-temporal memory at the i-th time step prediction based on the third hidden state, the third cell state generated at the (i - 1)-th time step prediction, the highway state at the i-th time step prediction, and the second globally enhanced spatio-temporal memory.

[0032] The third hidden state and the third globally enhanced spatio-temporal memory at the i-th time step prediction are input into the fourth globally reinforced ST-LSTM module. The fourth globally reinforced ST-LSTM module generates the fourth hidden state, the fourth cell state, and the fourth globally enhanced spatio-temporal memory at the i-th time step prediction based on the fourth hidden state, the fourth cell state generated at the (i - 1)-th time step prediction, the third hidden state at the i-th time step prediction, and the third globally enhanced spatio-temporal memory.

[0033] The fourth hidden state at the i-th time step prediction is element-wise added to the sub-image at the (i - 1)-th time step to obtain the radar echo prediction sub-image at the i-th time step.

[0034] It should be understood that the (i - 1)-th time step prediction refers to the prediction of the radar echo prediction sub-image at the (i - 1)-th time step, and the i-th time step prediction refers to the prediction of the radar echo prediction sub-image at the i-th time step.

[0035] Among them, the fourth globally enhanced spatio-temporal memory generated at the (i - 1)-th time step prediction is the fourth globally enhanced spatio-temporal memory generated by the fourth globally reinforced ST-LSTM module when predicting the radar echo prediction sub-image at the (i - 1)-th time step.

[0036] Among them, the first hidden state and the first cell state generated at the (i - 1)-th time step prediction are the first hidden state and the first cell state generated by the first globally reinforced ST-LSTM module when predicting the radar echo prediction sub-image at the (i - 1)-th time step.

[0037] Among them, the second hidden state and the second cell state generated at the (i - 1)-th time step prediction are the second hidden state and the second cell state generated by the second globally reinforced ST-LSTM module when predicting the radar echo prediction sub-image at the (i - 1)-th time step.

[0038] Among them, the third hidden state and the third cell state generated at the (i - 1)-th time step prediction are the third hidden state and the third cell state generated by the third globally reinforced ST-LSTM module when predicting the radar echo prediction sub-image at the (i - 1)-th time step.

[0039] Among them, the fourth hidden state and the fourth cell state generated during the prediction at the (i - 1)-th time step are the fourth hidden state and the fourth cell state generated when the fourth globally enhanced ST-LSTM module predicts the radar echo prediction sub-image at the (i - 1)-th time step.

[0040] Among them, by implementing a residual connection by element-wise adding the fourth hidden state at the prediction of the i-th time step to the sub-image at the (i - 1)-th time step, the input data can be directly combined with the prediction result. This can not only improve the detail restoration of the output of the radar echo image prediction model but also effectively alleviate the attenuation problem of information in the deep network, ensuring that the output result is more accurate and stable.

[0041] Among them, if is at this time, that is, the way the radar echo image prediction model predicts the radar echo prediction sub-image at the time step is as follows: Input the radar echo sub-image at the time step input to the radar echo image prediction model into the first globally enhanced ST-LSTM module. The first globally enhanced ST-LSTM module generates the first hidden state, the first cell state, and the first globally enhanced spatio-temporal memory at the prediction of the time step according to the radar echo sub-image at the time step.

[0042] Input the first hidden state and the first globally enhanced spatio-temporal memory at the prediction of the time step into the second globally enhanced ST-LSTM module. The second globally enhanced ST-LSTM module generates the second hidden state, the second cell state, and the second globally enhanced spatio-temporal memory at the prediction of the time step according to the first hidden state and the first globally enhanced spatio-temporal memory at the prediction of the i-th time step.

[0043] Input the second hidden state generated at the prediction of the time step into the gradient highway module. The gradient highway module generates the highway state at the prediction of the time step according to the second hidden state generated at the prediction of the time step.

[0044] Input the highway state and the second globally enhanced spatio-temporal memory at the prediction of the time step into the third globally enhanced ST-LSTM module. The third globally enhanced ST-LSTM module generates the time step according to the highway state and the second globally enhanced spatio-temporal memory at the prediction of the The third hidden state, the third cell state, and the third globally enhanced spatio-temporal memory during time step prediction.

[0045] The third hidden state and the third globally enhanced spatio-temporal memory during time step prediction are input into the fourth globally enhanced ST-LSTM module. The fourth globally enhanced ST-LSTM module generates the fourth hidden state, the fourth cell state, and the fourth globally enhanced spatio-temporal memory during time step prediction based on the third hidden state and the third globally enhanced spatio-temporal memory during time step prediction.

[0046] The fourth hidden state during time step prediction is element-wise added to the radar echo sub-image at the time step to obtain the

[0047] It should be understood that when is , that is, the input of the radar echo image prediction model is the first radar echo sub-image of the segmented radar echo image sequence (i.e., the radar echo sub-image at the time step), and the predicted is the radar echo prediction sub-image at the time step. This is the first prediction using the segmented radar echo image sequence. Therefore, the hidden states, cell states, and globally enhanced spatio-temporal memories corresponding to the first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the third globally enhanced ST-LSTM module, and the fourth globally enhanced ST-LSTM module in the radar echo image prediction model are empty.

[0048] In one embodiment, the second hidden state generated during the prediction at the i-th time step and the third hidden state generated during the prediction at the (i - 1)-th time step are input into the gradient highway module. The gradient highway module generates the highway state during the prediction at the i-th time step based on the second hidden state generated during the prediction at the i-th time step and the third hidden state generated during the prediction at the (i - 1)-th time step, including: The second hidden state generated during the prediction at the i-th time step is processed through the tanh activation function and added to the third hidden state generated during the prediction at the (i - 1)-th time step to obtain a candidate tensor; the second hidden state generated during the prediction at the i-th time step is processed through the σ activation function and added to the third hidden state generated during the prediction at the (i - 1)-th time step to obtain the first gating weight and the second gating weight; the candidate tensor is weighted according to the first gating weight to obtain a weighted candidate tensor; the third hidden state generated during the prediction at the (i - 1)-th time step is weighted according to the second gating weight to obtain a weighted third hidden state; the weighted candidate tensor and the weighted third hidden state are feature fused to generate the highway state at the i-th time step prediction.

[0049] Among them, the gradient highway module acts as a high-speed channel that specifically retains gradients and key information in the multi-layer temporal convolutional network. As Figure 2 shown, it shows the internal mechanism of the Gradient-Highway Unit (GHU) module, which receives two inputs: the second hidden state generated during the prediction at the i-th time step from the previous layer (i.e., the second globally enhanced ST-LSTM module) and the third hidden state generated during the prediction at the (i - 1)-th time step from the third globally enhanced ST-LSTM module , where l is the number of the globally enhanced ST-LSTM module. The second hidden state and the third hidden state are first mapped to a unified channel dimension through convolution; one branch passes through the tanh activation function to obtain the candidate tensor P, and the other branch passes through the activation function to generate the first gating weight and the second gating weight. Among them, the first gating weight is S, and the second gating weight is 1 - S, S ∈ [0, 1]. Subsequently, the gradient highway module linearly interpolates element-wise according to the first gating weight and the second gating weight to obtain the highway state at the i-th time step prediction , and the expression is:

[0050] ; where, is the element-wise multiplication (Hadamard multiplication).

[0051] On the one hand, this expression injects the latest spatio-temporal features (i.e., the candidate tensor P) into the memory of this layer, and on the other hand, it retains in the proportion of historical information . Therefore, the gradient can be along The identity branch propagates backward with almost no attenuation, thus effectively alleviating the vanishing or exploding gradients commonly found in deep temporal networks.

[0052] In the overall network topology, the gradient highway module is inserted between the second layer (i.e., the second globally enhanced ST-LSTM module) and the third layer (i.e., the third globally enhanced ST-LSTM module): longitudinally, it passes the features calculated by the second layer at the i-th time step upward to the third layer; transversely, it maintains the recursive link, providing a stable information and gradient channel for the radar echo image prediction model. Compared with the existing LSTM units and variants of LSTM units, the gradient highway module can achieve an adaptive trade-off between writing new information and retaining old memories only relying on two groups of lightweight convolutions and a single gate, significantly reducing the number of parameters and video memory occupancy, while enhancing the network's ability to model long-range spatio-temporal dependencies and training stability.

[0053] Among them, in the middle layer of the network, the gradient highway module is used to perform feature fusion and adjustment on the hidden state of the previous layer. This gradient highway module ensures that gradients can be transmitted more smoothly in the deep network, alleviating the problems of vanishing and exploding gradients, so that the radar echo image prediction model can maintain better training stability and effect when capturing complex spatio-temporal dependency relationships.

[0054] Among them, the gradient highway module ensures the high-speed and stable transmission of gradients between network layers and in time series, effectively alleviating the problem of vanishing or exploding gradients in deep recurrent networks, and significantly enhancing the ability to capture long-term dependencies.

[0055] In one embodiment, the first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the third globally enhanced ST-LSTM module, and the fourth globally enhanced ST-LSTM module are globally enhanced ST-LSTM networks with the same structure.

[0056] Among them, as shown in Figure 3, it is a schematic diagram of the structure of the globally enhanced ST-LSTM network. Based on the original mechanism of the existing ST-LSTM network (including input gate, forget gate, output gate, cell state, and hidden state, etc.), this globally enhanced ST-LSTM network additionally introduces a multi-source self-attention unit and a global enhanced memory path (i.e., an average pooling branch composed of an adaptive average pooling layer and a fully connected layer) (taking the hidden state as an example, where l is the number of the globally enhanced ST-LSTM module, ) to obtain global information through operations such as the adaptive average pooling layer (AdaptiveAvgPool2d) to strengthen spatio-temporal memory , update the cell state and hidden state. During this process, the AdaptiveAvgPool2d layer is used in conjunction with the fully connected layer (FC) to perform global pooling on the input features, generate global context reinforcement information, multiply it by the adaptive weight parameter α, and then add it to the updated candidate spatio-temporal memory (i.e., the spatio-temporal memory without adding the global enhancement information) to obtain the globally enhanced spatio-temporal memory , thereby while fusing local spatio-temporal information, dynamically adjusting with the help of global semantics, enabling the radar echo image prediction model to better capture long-term temporal dependencies in complex scenarios. Generally speaking, this "global-local" dual enhancement design can integrate the hidden state of the previous time step of the current hidden layer at each time step , cell state , spatio-temporal memory and the current input The multi-source information contained therein, while improving the prediction accuracy, maintains high stability.

[0057] Among them, the expression of the processing process of the globally enhanced ST-LSTM network is: , , , , , , , , , , , , , , , ; Among them, is the convolution that maps the input to 7 gate vectors, is the convolution that maps the input to 4 gate vectors, is the convolution that maps the input to 3 gate vectors, The hidden state, highway state, or sub-image input to the l-th globally enhanced ST-LSTM module during the i-th time step prediction, The hidden state generated by the l-th globally enhanced ST-LSTM module during the (i-1)-th time step prediction, The globally enhanced spatio-temporal memory generated by the (l-1)-th globally enhanced ST-LSTM module during the i-th time step prediction, is from The input gate pre-activation, is from The forget gate pre-activation, is from The candidate memory, The globally enhanced memory branch version of the input gate, The globally enhanced memory branch version of the forget gate, The candidate memory of the globally enhanced memory branch, Element-wise multiplication (Hadamard multiplication), is The contribution to the output gate, is from The pre-activation of the input gate, is from The pre-activation of the forget gate, is from The candidate memory, is The contribution to the output gate, is from The pre-activation of the input gate, is from The pre-activation of the forget gate, is from The candidate memory of the input gate, The output of the input gate, The Sigmoid activation function, The output of the forget gate, The forget gate bias, The output of the input modulation gate, The hyperbolic tangent activation function, The cell state generated by the l-th globally enhanced ST-LSTM module during the i-th time step prediction, The cell state generated by the l-th globally enhanced ST-LSTM module during the (i-1)-th time step prediction, The output of the input gate of the globally enhanced memory branch, The output of the forget gate of the globally enhanced memory branch, The output of the input modulation gate of the globally enhanced memory branch, is the global enhancement information, is the adaptive average pooling layer, is the fully connected layer, is to broadcast the 1*1 feature back to the same spatial size, is the global enhanced spatio-temporal memory generated at the i-th time step prediction of the l-th globally enhanced ST-LSTM module, is the adaptive weight parameter, is the comprehensive memory tensor, is to perform channel dimension concatenation, is the output of the output gate, denotes compressed and fused to obtain the features required to drive the output gate, denotes compressed and fused to obtain the features required to generate the hidden state, is the multi-source self-attention unit, is the preliminary hidden state generated at the i-th time step prediction of the l-th globally enhanced ST-LSTM module, is the hidden state generated at the i-th time step prediction of the l-th globally enhanced ST-LSTM module.

[0058] Among them, , , are used for gate vector splitting. The convolutional kernel sizes of these three convolutions are uniformly controlled by the hyperparameters initialized by the radar echo image prediction model, and the convolutional kernel sizes of these three convolutions can be 5*5.

[0059] Among them, the adaptive average pooling layer is used to extract the global context.

[0060] Among them, the fully connected layer is used to perform channel recalibration on the global context.

[0061] Among them, is used to generate the output gate pre-activation, and the convolutional kernel size of this convolution can be 5*5.

[0062] Among them, is used to generate the candidate hidden state, and the convolutional kernel size of this convolution can be 1*1.

[0063] Among them, controls how much "new information" the cell state generated at the i-th time step prediction of the l-th globally enhanced ST-LSTM module receives; controls how much of the current cell's cell state is retained; participates in generating the cell state after passing through tanh ; jointly determine the write volume of the global enhanced spatio-temporal memory ; and jointly determine how much to retain .

[0064] Among them, the processing process expression of the global enhanced memory branch is: , , , , .

[0065] It should be understood that in the processing process of the globally enhanced ST-LSTM network of the first globally enhanced ST-LSTM module, there is no globally enhanced ST-LSTM module before the first globally enhanced ST-LSTM module. Therefore, the input to the first globally enhanced ST-LSTM module is a sub-image that has not been processed by the globally enhanced ST-LSTM module. If i is any time step from to , the sub-image at the (i - 1)-th time step is the radar echo sub-image at the (i - 1)-th time step. If i is any time step from to , the sub-image at the i-th time step is the radar echo prediction sub-image at the (i - 1)-th time step.

[0066] It should be understood that in the processing process of the globally enhanced ST-LSTM network of the third globally enhanced ST-LSTM module, the gradient highway module is before the third globally enhanced ST-LSTM module. Therefore, the input to the third globally enhanced ST-LSTM module is the highway state generated by the gradient highway module.

[0067] It should be understood that in the processing process of the globally enhanced ST-LSTM network of the second globally enhanced ST-LSTM module and the fourth globally enhanced ST-LSTM module, the first globally enhanced ST-LSTM module is before the second globally enhanced ST-LSTM module, and the third globally enhanced ST-LSTM module is before the fourth globally enhanced ST-LSTM module. Therefore, the input to the second globally enhanced ST-LSTM module and the fourth globally enhanced ST-LSTM module is the hidden state.

[0068] Among them, the radar echo image prediction model embeds a multi-source self-attention mechanism in the globally enhanced ST-LSTM module. By fusing the hidden state of the previous time step of the current layer with the preliminary hidden state of the current time step, the multi-source self-attention unit can simultaneously focus on local details and global information, thereby significantly enhancing the feature expression ability, providing richer and more accurate feature information for the radar echo image prediction model in spatio-temporal prediction tasks, and improving the overall prediction performance. If more input sources need to be processed, the radar echo image prediction model can also be extended to a multi-source input mode to achieve more generalized feature fusion.

[0069] Among them, an adaptive average pooling layer is used to perform adaptive global average pooling on the globally enhanced spatio-temporal memory of the previous layer at the i-th time step, and then a global semantic vector is obtained through mapping by a fully connected layer. The width and height are expanded back to the same as those of the globally enhanced spatio-temporal memory of the previous layer at the i-th time step in the spatial dimension to obtain a globally enhanced information (i.e., the output of the average pooling branch). Finally, the globally enhanced information is multiplied by the adaptive weight parameter α and added element-wise to the spatio-temporal memory obtained by gated update to obtain the final globally enhanced spatio-temporal memory.

[0070] Among them, by introducing an average pooling branch in the spatio-temporal memory update process for global context enhancement, the limitation of relying only on local convolutional receptive fields is overcome, enabling the radar echo image prediction model based on globally enhanced PredRNN to achieve better global consistency in complex scenarios and long sequence predictions.

[0071] Among them, after the input data (sub-image or hidden state) undergoes multi-layer spatio-temporal feature extraction and information fusion, the final high-dimensional features are converted back to the same dimension as the original data (i.e., the input data) through a layer of convolutional mapping, and the residual connection technology is used to directly combine the input data with the prediction result, which can not only improve the detail restoration of the output of the radar echo image prediction model, but also effectively alleviate the attenuation problem of information in the deep network, ensuring that the output result is more accurate and stable.

[0072] In one embodiment, the processing process of the multi-source self-attention unit is as follows: Add the preliminary hidden state generated by the $l$-th globally enhanced ST-LSTM module at the $i$-th time step prediction and the hidden state generated by the $l$-th globally enhanced ST-LSTM module at the $(i - 1)$-th time step prediction, and then perform shape adjustment to obtain the adjusted hidden state; process the adjusted hidden state through the multi-head self-attention sub-unit to obtain the self-attention enhanced feature; add the self-attention enhanced feature and the adjusted hidden state and then process through the normalization layer to obtain the normalized hidden state; process the normalized hidden state through the feed-forward neural network, add it to the normalized hidden state, and then successively perform normalization layer and shape adjustment processing to obtain the hidden state generated by the $l$-th globally enhanced ST-LSTM module at the $i$-th time step prediction.

[0073] Among them, as Figure 4 shown in the structural schematic diagram of the multi-source self-attention unit, on the basis of introducing multiple inputs (such as the preliminary hidden state generated by the $l$-th globally enhanced ST-LSTM module at the $i$-th time step prediction and the hidden state generated by the $l$-th globally enhanced ST-LSTM module at the $(i - 1)$-th time step prediction etc.), first convert the four-dimensional tensor to a three-dimensional form through shape adjustment (Shape Transform), then perform multi-head self-attention (Multi-HeadAttention) sub-unit calculation and feed-forward neural network (FFN) processing, and add layer normalization (LayerNorm) and residual connection at key steps. Finally, restore to the same shape as the input through shape adjustment (Shape Transform). This design can flexibly interact information between the spatial dimension and the channel / batch dimension, thereby obtaining a richer and more comprehensive spatio-temporal feature representation, improving the prediction accuracy and model stability.

[0074] Among them, the multi-source self-attention unit fully integrates the short-term memory information carried by the hidden states of adjacent time steps, strengthens the explicit correlation modeling between different moments, and enhances the network's ability to capture spatial structures and key features.

[0075] Among them, the multi-head self-attention sub-unit is a multi-head self-attention network. The multi-head self-attention network respectively performs three independent linear transformations on the adjusted hidden state (i.e., the input of the multi-head self-attention sub-unit) to generate a query feature sequence (Query), a key feature sequence (Key), and a value feature sequence (Value). Subsequently, to implement the multi-head attention mechanism, it is necessary to reshape and transpose the above query feature sequence, key feature sequence, and value feature sequence in the embedding dimension, divide them into a sub-spaces, each sub-space corresponding to an attention head, and each sub-space corresponding to a query feature sub-sequence, a key feature sub-sequence, and a value feature sub-sequence. Each attention head will separately process its corresponding query feature sub-sequence, key feature sub-sequence, and value feature sub-sequence. For each attention head, calculate the similarity between its corresponding query feature sub-sequence and key feature sub-sequence, and obtain the attention weight through softmax; then apply the weight to the corresponding value feature sub-sequence to obtain the attention output of this attention head. Then, splice the outputs of the above n attention heads in the embedding dimension and integrate them through a linear layer to obtain a fused context-enhanced feature sequence, which is used as the output of this multi-head self-attention sub-unit, that is, the self-attention enhanced feature. This output is used to update the hidden state of the current layer.

[0076] In one embodiment, the training method of the radar echo image prediction model based on global reinforcement PredRNN is as follows: Step 1, obtain a sample data set containing multiple radar echo image samples, and each radar echo image sample includes radar echo images at q time steps.

[0077] Among them, in the data preparation stage, extract the VIL (Vertically Integrated Liquid) subset data of the target area from the SEVIR data set. This data set contains radar echo images at multiple time steps, ensuring that it covers a long enough time series. At the same time, some auxiliary information (such as time stamps and metadata) will also be collected to provide complete and accurate basic data for subsequent data preprocessing and model training.

[0078] Among them, use the VIL (Vertically Integrated Liquid) subset data in the SEVIR data set from 2017 to 2019 to train the radar echo image prediction model based on global reinforcement PredRNN. VIL (Vertically Integrated Liquid) is a physical quantity calculated based on radar reflectivity, representing the total amount of liquid water in all clouds in the vertical direction of the atmosphere. Predict the change trend of radar echo images in the next one hour, two hours, or longer using the trained radar echo image prediction model based on global reinforcement PredRNN.

[0079] Among them, download the VIL subset of the SEVIR dataset from 2017 to 2019, obtain multiple radar echo image samples, and construct a sample dataset.

[0080] Step 2, preprocess each radar echo image sample in the sample dataset, and combine the radar echo images in each radar echo image sample into a radar echo image sample sequence with a fixed length in chronological order.

[0081] Among them, in the data sequence construction stage, first perform preliminary processing on the radar echo image samples, including operations such as noise removal and normalization, and then combine consecutive radar echo images into a radar echo image sample sequence with a fixed length in chronological order (for example, 20 consecutive time-step radar echo images). The radar echo image sample sequence will be packed into a data structure with the shape of [batch, 20, 1, img_height, img_width] and input into the block processing module for block processing. Specifically: Batch is the batch size, indicating the number of data sequences input to the model at one time; the sequence length is 20, indicating that each radar echo image sample sequence contains 20 frames of radar echo images; the number of channels is 1. The dataset used in this embodiment is grayscale image data, so the number of channels of the radar echo image is 1; img_height and img_width respectively represent the height and width dimensions of each radar echo image. In this way, it is ensured that each sequence can fully capture the spatio-temporal change information of the target area over a period of time.

[0082] It should be understood that the radar echo image sample sequence is 20 time-step radar echo images. During the training process, 19 time-step radar echo images are actually used. In the validation and test phases, 20 time-step radar echo sub-images are required for comparing the prediction results. Therefore, the radar echo image sample sequence requires 20 time-step radar echo images.

[0083] Step 3, perform block processing on the radar echo images at each time step of each radar echo image sample sequence to obtain each block-processed sample sequence. All the block-processed sample sequences constitute the processed sample dataset. Each block-processed sample sequence includes radar echo sub-images corresponding to the radar echo images at q time steps.

[0084] Among them, use methods such as block strategies to preprocess the radar echo image sample sequence. Each radar echo image sample has 20 frames of radar echo images, and each frame of radar echo image is block-processed. For example, assume that the height and width of a frame of radar echo image are respectively evenly divided into n parts, then each frame of radar echo image will be divided into n 2 radar echo sub-images, that is, each frame of radar echo image corresponds to n2 Zhang's radar echo sub-images. Thus, a new processed sample data set is constructed.

[0085] Among them, in the block processing stage, each frame of radar echo image is further refined. The specific steps are as follows: Image block: The original radar echo image is evenly divided into several radar echo sub-images of equal size. Assuming that the height and width of the radar echo image are respectively evenly divided into n parts, then each image will be divided into n 2 radar echo sub-images.

[0086] Adjust the number of channels: By expanding the channel dimension, the features of each radar echo sub-image can independently express its local spatial information. Specifically, the original single-channel (grayscale image) is expanded to n 2 channels, and each channel corresponds to a radar echo sub-image.

[0087] Data shape adjustment: After the above block processing, the sample sequence of radar echo image with the data structure of [batch, 20, 1, img_height, img_width] forms a block-processed sample sequence with the data structure of [batch, 20, n², img_height / n, img_width / n]. Among them: batch is the batch size, indicating the number of data sequences input into the model at one time; the sequence length is 20, indicating that each radar echo image sample sequence contains 20 frames of radar echo images; the number of channels after the block processing of the radar echo image is n 2 , and this number of channels is equal to the total number of radar echo sub-images of each radar echo image multiplied by the number of channels of the radar echo image; img_height / n and img_width / n are the height and width of each radar echo sub-image.

[0088] Step 4, divide the processed sample data set into a training set, a validation set and a test set.

[0089] Step 5, use the block-processed sample sequence in the training set to train the radar echo image prediction model of PredRNN based on global reinforcement to be trained, and obtain the trained radar echo image prediction model.

[0090] Among them, use the divided training set to train the radar echo image prediction model, and use the validation set for inspection at the end of each training round, and adjust the training parameters of the radar echo image prediction model according to the inspection results for better subsequent training.

[0091] Step 6: Use the sample sequences after chunking in the validation set to verify the trained radar echo image prediction model based on globally reinforced PredRNN. After adjusting the network parameters of the trained radar echo image prediction model according to the verification results, continue the training until the verification results reach the convergence condition, and obtain a trained radar echo image prediction model based on globally reinforced PredRNN.

[0092] Among them, use the training set to train the radar echo image prediction model based on globally reinforced PredRNN, use the validation set to verify the training results during the training process, and adaptively adjust the network parameters for subsequent training.

[0093] Step 7: Use the sample sequences after chunking in the test set to test the trained radar echo image prediction model based on globally reinforced PredRNN, and evaluate the performance of the trained radar echo image prediction model based on globally reinforced PredRNN.

[0094] Among them, after the radar echo image prediction model is trained, use the divided test set to test the prediction performance of the radar echo image prediction model on new data.

[0095] Among them, use the test set data that the radar echo image prediction model based on globally reinforced PredRNN has not seen to test the trained radar echo image prediction model based on globally reinforced PredRNN, evaluate the test results with various indicators, and compare them with the prediction results of the existing PredRNN model and ConvLSTM model.

[0096] Among them, use two evaluation indicators, mean squared error (MSE) and structural similarity index (SSIM), to quantitatively analyze the complete prediction results, and visually display the differences between the prediction results and the actual observations through visualization charts.

[0097] Among them, the radar echo image prediction model includes: the first globally reinforced ST-LSTM module, the second globally reinforced ST-LSTM module, the gradient highway module, the third globally reinforced ST-LSTM module, and the fourth globally reinforced ST-LSTM module; the first globally reinforced ST-LSTM module, the second globally reinforced ST-LSTM module, the gradient highway module, the third globally reinforced ST-LSTM module, and the fourth globally reinforced ST-LSTM module are connected in sequence.

[0098] Among them, at each time step, implement the residual addition strategy to superimpose the network output and the current input, so that the model only needs to learn the increment, thereby further stabilizing the gradient and simplifying the training process.

[0099] In one embodiment, during the process of training a radar echo image prediction model based on globally reinforced PredRNN, a scheduled sampling unit is added before the radar echo image prediction model based on globally reinforced PredRNN.

[0100] Among them, Figure 5 Fig. 5 shows the structural composition of the radar echo image prediction model based on globally reinforced PredRNN in a time series prediction task and its operation process at different time steps during training. The radar echo image prediction model based on globally reinforced PredRNN is overall composed of four stacked GEn-STLSTM (Global Enhanced Spatiotemporal LSTM) modules, and a gradient highway module is inserted after the second layer to optimize the gradient propagation problem in long sequences.

[0101] It should be understood that the basic structure of the radar echo image prediction model includes a sampling layer, a hidden layer, and an output layer. The sampling layer is used to receive the input data, and the hidden layer is responsible for extracting the features of the data, that is, it is composed of the first globally reinforced ST-LSTM module, the second globally reinforced ST-LSTM module, the gradient highway module, the third globally reinforced ST-LSTM module, and the fourth globally reinforced ST-LSTM module. The output layer generates the final prediction result.

[0102] Among them, during the training process, the radar echo sub-images corresponding to the radar echo images at each time step in the chunked sample sequence are input into the radar echo image prediction model based on globally reinforced PredRNN to be trained in chronological order (that is, all the radar echo sub-images of the radar echo image). Each training process (that is, completing the learning and inference of the radar echo images at all time steps of a chunked sample sequence) can be divided into a learning stage and an inference stage. Taking a chunked sample sequence including 20 time steps of radar echo sub-images (that is, real radar echo sub-image data) as an example, the radar echo sub-images of the first 10 time steps are input into the radar echo image prediction model based on globally reinforced PredRNN in sequence, so that the radar echo image prediction model based on globally reinforced PredRNN can learn the features in the sample. The radar echo sub-images of the last 9 time steps are used to assist the radar echo image prediction model based on globally reinforced PredRNN to infer and predict the radar echo prediction sub-images (radar echo prediction sub-image data) of the last 10 time steps.

[0103] Among them, to clearly show the information interaction between the front and back time steps, Figure 5The following is an example of a processing flow that shows a typical scenario representing the learning stage (learning using radar echo images from the 1st time step to the 10th time step) and the inference stage (inferring using radar echo images from the 11th time step to the 19th time step), taking the learning process using the radar echo image at the 10th time step and the inference process using the radar echo image at the 11th time step as examples: In the learning stage, the radar echo image prediction model receives the real input frame X 10 (i.e., all radar echo sub-images of the radar echo image at the 10th time step) as the input for the current time step. After the hidden state, cell state, and spatio-temporal memory are generated at the previous time step, they are sequentially updated in each layer of the GEn-STLSTM, and are used to generate the hidden state, cell state, and spatio-temporal memory for the current time step. Among them, the hidden state and spatio-temporal memory not only achieve stride transfer in the time dimension (i.e., transferred from the previous time step to the current time step), but also perform vertical information flow between different network levels within the same time step (i.e., cross-layer flow, such as from the first layer to the second layer), enhancing the radar echo image prediction model's ability to model multi-level spatio-temporal dependencies. After completing the state update of the four-layer globally reinforced ST-LSTM module, to improve the stability of the radar echo image prediction model and enhance the ability to restore image details, the radar echo image prediction model performs a residual connection between the output of the last layer of the globally reinforced ST-LSTM module GEn-STLSTM and the original input frame (i.e., all radar echo sub-images of the real radar echo image), thereby generating the final prediction frame O 11 (i.e., all radar echo prediction sub-images of the radar echo prediction image at the 11th time step). This residual structure helps to strengthen the modeling and restoration of local features while maintaining the overall spatial structure consistency. In this stage, the Teacher Forcing strategy is adopted, that is, the input for each time step is provided by the real radar echo image, ensuring that the model effectively learns short-term and long-term temporal dependencies.

[0104] In the inference stage, the radar echo image prediction model enters the autoregressive prediction mode and no longer completely relies on real data as input. At this time, the input of the radar echo image prediction model is the prediction frame O generated at the previous time step 11 and the real frame at the current time step (i.e., all radar echo sub-images of the radar echo image at the 11th time step) are jointly fed into the Scheduled Sampling Unit. The Scheduled Sampling Unit decides whether to select the prediction frame O according to the set scheduling probability η 11Or the real frame is used as the input of the first-layer GEn-STLSTM network, so as to alleviate the distribution shift problem in the learning and inference stages. After that, the sampled input enters the same four-layer GEn-STLSTM network structure again for state update and prediction result generation. The hidden state output by the last-layer GEn-STLSTM network is connected with the input of the first-layer GEn-STLSTM network by residual connection, and the final predicted frame O is output. 12 (i.e., all radar echo prediction sub-images of the radar echo prediction image at the 12th time step).

[0105] Among them, in order to understand the realization of stride transfer in the time dimension and the longitudinal information flow between different network levels within the same time step, Figure 5 In the figure, the unfolded view of the information transfer of multiple time steps (i.e., the radar echo images at the 10th time step and the 11th time step) is drawn. The structure of the radar echo image prediction model (GEn-PredRNN model) (i.e., the first globally reinforced ST-LSTM module GEn-STLSTM1, the second globally reinforced ST-LSTM module GEn-STLSTM2, the gradient highway module GHU, the third globally reinforced ST-LSTM module GEn-STLSTM3, and the fourth globally reinforced ST-LSTM module GEn-STLSTM4 are connected in sequence) remains consistent throughout the sequence and shares weights (SharedParameters) among all time steps.

[0106] Among them, when learning or inferring the radar echo image of the previous time step, the fourth globally enhanced spatio-temporal memory generated by GEn-STLSTM4 will be passed to the learning or inference of the current time step and passed layer by layer. For example, when learning the radar echo image at the 10th time step, the fourth globally enhanced spatio-temporal memory M generated by GEn-STLSTM4 l=4 10 is passed to the inference of the radar echo image at the 11th time step as the input of GEn-STLSTM1, and the first globally enhanced spatio-temporal memory M output by GEn-STLSTM1 l=1 11 is passed to the next layer GEn-STLSTM2, and the second globally enhanced spatio-temporal memory M output by GEn-STLSTM2 l=2 11 is passed to the next layer GEn-STLSTM3, and the third globally enhanced spatio-temporal memory M output by GEn-STLSTM3 l=3 11 is passed to the next layer GEn-STLSTM4, and finally the fourth globally enhanced spatio-temporal memory M output by GEn-STLSTM4 l=4 11Passed to the inference of the next time step.

[0107] Among them, when learning or inferring the radar echo image of the previous time step, the new cell states generated by each layer of GEn-STLSTM update the cell states of each layer. For example, when inferring the radar echo image of the 11th time step, GEn-STLSTM1 generates the first cell state C based on the current cell state (i.e., the first cell state C generated when learning the radar echo image of the 10th time step) l=1 10 ). Generate the first cell state C l=1 11 , and update the current cell state to the first cell state C l=1 11 for GEn-STLSTM1 to generate a new first cell state in the next time step; similarly, GEn-STLSTM2 generates the second cell state C based on the current cell state (i.e., the second cell state C generated when learning the radar echo image of the 10th time step) l=2 10 ). Generate the second cell state C l=2 11 , and update the current cell state to the second cell state C l=2 11 for GEn-STLSTM2 to generate a new second cell state in the next time step; GEn-STLSTM3 generates the third cell state C based on the current cell state (i.e., the third cell state C generated when learning the radar echo image of the 10th time step) l=3 10 ). Generate the third cell state C l=3 11 , and update the current cell state to the third cell state C l=3 11 for GEn-STLSTM3 to generate a new third cell state in the next time step; GEn-STLSTM4 generates the fourth cell state C based on the current cell state (i.e., the fourth cell state C generated when learning the radar echo image of the 10th time step) l=4 10 ). Generate the fourth cell state C l =4 11 , and update the current cell state to the fourth cell state C l=4 11 for GEn-STLSTM4 to generate a new fourth cell state in the next time step.

[0108] Among them, when learning or inferring the radar echo image of the previous time step, the hidden states generated by each layer of GEn-STLSTM update the hidden states of each layer and are passed layer by layer. For example, when inferring using the radar echo image of the 11th time step, GEn-STLSTM1 is based on the current hidden state (i.e., the first hidden state H generated when learning the radar echo image of the 10th time step) l=1 10 ) and the fourth hidden state H generated when learning the radar echo image of the 10th time step l=4 10 to generate the first hidden state H l=1 11 , update the current hidden state to the first hidden state H l=1 11 for GEn-STLSTM1 to generate a new first hidden state at the next time step, and input the first hidden state H l=1 11 into GEn-STLSTM2; GEn-STLSTM2 is based on the first hidden state H l=1 11 and the current hidden state (i.e., the second hidden state H generated when learning the radar echo image of the 10th time step l=2 10 ) to generate the second hidden state H l=2 11 , update the current hidden state to the second hidden state H l=2 11 for GEn-STLSTM2 to generate a new second hidden state at the next time step, and input the second hidden state H l=2 11 into GHU; GHU is based on the second hidden state H l=2 11 and the third hidden state H generated by GEn-STLSTM3 when learning the radar echo image of the 10th time step l=3 10 to generate the highway state R 11 , input the highway state R 11 into GEn-STLSTM3; GEn-STLSTM3 is based on the highway state R 11 and the current hidden state (i.e., the third hidden state H generated when learning the radar echo image of the 10th time step l=3 10 ) to generate the third hidden state H l=3 11 , update the current hidden state to the third hidden state H l=3 11For GEn-STLSTM3 to generate a new third hidden state at the next time step and pass the third hidden state H l=3 11 to the input of GEn-STLSTM4; GEn-STLSTM4 generates a fourth hidden state H based on the third hidden state H l=3 11 and the current hidden state (i.e., the fourth hidden state H generated during the learning of the radar echo image at the 10th time step) l=4 10 ), and updates the current hidden state to the fourth hidden state H l=4 11 For GEn-STLSTM4 to generate a new fourth hidden state at the next time step and output the fourth hidden state H l=4 11 For GEn-STLSTM4 to generate a new fourth hidden state at the next time step and output the fourth hidden state H l =4 11 Output

[0109] Thus, the hidden state, cell state, global enhanced spatio-temporal memory, and highway state are passed layer by layer between time steps, forming a complete spatio-temporal memory chain, thereby supporting the radar echo image prediction model to model and predict the complex dynamic changes in the radar echo image sequence

[0110] In one embodiment, the radar echo image prediction model based on globally reinforced PredRNN is trained using the sample sequences after chunking in the training set to obtain the trained radar echo image prediction model, including: Input the radar echo sub-image at the 1st time step of the chunked sample sequence into the radar echo image prediction model based on globally reinforced PredRNN to be trained, and output the radar echo prediction sub-image at the 2nd time step; input the radar echo sub-image at the 2nd time step of the chunked sample sequence into the radar echo image prediction model based on globally reinforced PredRNN to be trained, and output the radar echo prediction sub-image at the 3rd time step; and so on, until the radar echo sub-image at the time step of the chunked sample sequence is input into the radar echo image prediction model based on globally reinforced PredRNN to be trained, and output the radar echo prediction sub-image at the + 1 time step; then input the radar echo sub-image at the + 1 time step and the radar echo prediction sub-image at the + 1 time step into the scheduled sampling unit, so that the scheduled sampling unit determines according to the preset scheduling probability η to use the + The radar echo sub-image or radar echo prediction sub-image of 1 time step is input into the radar echo image prediction model of PredRNN based on global reinforcement to be trained, and the radar echo prediction sub-image of the future + 2 time steps is output; and so on. Then, the radar echo sub-image and the radar echo prediction sub-image of the (q - 1)-th time step of the segmented sample sequence are input into the scheduled sampling unit, so that the scheduled sampling unit determines whether to input the radar echo sub-image or the radar echo prediction sub-image of the (q - 1)-th time step into the radar echo image prediction model of PredRNN based on global reinforcement to be trained according to the preset scheduling probability η, and outputs the radar echo prediction sub-image of the q-th time step.

[0111] Among them, during the training process of the radar echo image prediction model, the scheduled sampling strategy is first applied. By dynamically adjusting the scheduling probability η, it is decided whether to use the real data (i.e., the radar echo sub-image) as input or rely on the output predicted by the radar echo image prediction model itself (i.e., the radar echo prediction sub-image) at each time step. This strategy mainly relies on real data in the initial stage, and as the number of iterations increases, η gradually decreases, so that the training data gradually changes from real input to the data predicted by the radar echo image prediction model, thus effectively alleviating the problem of inconsistent distribution between learning and inference, and laying a foundation for the subsequent autoregressive prediction of the radar echo image prediction model.

[0112] Among them, the initial value of the preset scheduling probability η is 1, and a hyperparameter μ for adjusting the scheduling probability η is set. After each iteration training is completed, μ is decreased based on the current scheduling probability η.

[0113] Among them, the value of the hyperparameter μ can be set according to the actual situation. For example, the value of the hyperparameter μ can be set to 0.001, 0.002, 0.003, etc.

[0114] In an example, the value of the hyperparameter μ is 0.002. After each iteration training is completed, 0.002 is decreased based on the current scheduling probability η. Taking the reduction of the scheduling probability η after the first iteration training as an example, the scheduling probability η before adjustment is 1, and the scheduling probability η after adjustment is 0.998.

[0115] The above-mentioned radar echo image prediction method based on globally enhanced PredRNN is oriented to the spatio-temporal prediction task of radar echo images. Taking the VIL (vertically integrated liquid) data of the target area in the SEVIR dataset as the research object, a set of efficient and stable deep learning frameworks are constructed around the main links such as data preparation, data preprocessing, model learning and inference, and result post-processing and evaluation. In the network design, key technologies such as the scheduled sampling strategy, globally enhanced ST-LSTM module, gradient highway module, and dual-source self-attention mechanism are introduced, effectively improving the accuracy and stability of the model in long-time series prediction; at the same time, the combination of patching and residual connection technologies further enhances the ability to capture and recover detailed features. Finally, this application can comprehensively analyze the prediction results with multiple evaluation indicators and visualization methods, providing strong support for applications in fields such as weather forecasting, basin flood control, and disaster warning.

[0116] The above-mentioned radar echo image prediction method based on globally enhanced PredRNN introduces a gradient highway module, effectively alleviating the problem of gradient vanishing or explosion in deep recurrent networks, thereby improving the ability to capture long-term dependencies; at the same time, multi-source self-attention is adopted to enhance the model's comprehensive understanding of spatial structure and temporal dynamics; and the global context enhancement mechanism is used to enrich spatio-temporal memory and improve the overall expression ability of the model. Compared with traditional prediction models, this method significantly improves the phenomenon that the quality of predicted images rapidly decays over time, providing more reliable technical support for the accurate prediction of radar echo images and weather forecasting.

[0117] In one embodiment, the performance comparison of three models, namely ConvLSTM, PredRNN, and GEn-PredRNN (the radar echo image prediction model of this application), is as follows: As Figure 6 shown in the schematic diagram of the MSE change curve over time, this figure takes the time step as the abscissa and the mean square error (MSE) as the ordinate, showing the change of the prediction error of the three models over the prediction time; the results show that the MSE value of GEn-PredRNN is always the lowest at most time steps, especially in the later stage, it can maintain a lower error, indicating that it has higher accuracy in long-term prediction. PredRNN is the second, but the error rises significantly in the middle and later stages and then drops slightly. The curve of ConvLSTM is the highest, showing its deficiency in capturing long-time series dependencies.

[0118] As Figure 7The schematic diagram of the SSIM curve varying with time shows, by marking the predicted time steps on the abscissa and the structural similarity (SSIM) values on the ordinate, the comparison of the prediction quality of three models, namely ConvLSTM, PredRNN, and GEn-PredRNN, at multiple future moments. From the overall trend of the curve, it can be seen that the SSIM value of GEn-PredRNN always remains at a relatively high level, indicating that it can better retain the structural details of the image at different prediction moments. PredRNN ranks second, while the curve of ConvLSTM shows a significant decline at later moments, reflecting the relatively insufficient ability of this model to maintain the structure for long-term prediction.

[0119] As Figure 8 shown in the schematic diagram of the visual comparison of the prediction results, it shows the visual differences between the prediction results of different models and the ground truth at the same moment. Each column represents a predicted time step or the ground truth, and each row is, in sequence, the true observation sequence, the output of ConvLSTM, PredRNN, and GEn-PredRNN. It can be clearly seen that GEn-PredRNN can still generate images closer to the true distribution at later moments, and there is less blurring or distortion in the detail areas. In contrast, the results of ConvLSTM are distorted to a certain extent at subsequent time steps, and although the prediction of PredRNN is better than that of ConvLSTM, it still lags behind GEn-PredRNN in the detail restoration degree in key areas.

[0120] It should be understood that although Figure 1 the steps in the flowchart of Figure 1 are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover,

[0121] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0122] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A radar echo image prediction method based on global reinforcement PredRNN, characterized in that The radar echo image prediction method based on global reinforcement PredRNN includes: Obtaining radar echo images from the (t - m)-th time step to the t-th time step of the target area, where t is the current time step and m is a positive integer; Preprocessing each of the radar echo images and combining them into a radar echo image sequence of a fixed length in chronological order; Performing a block processing on the radar echo image of each time step in the radar echo image sequence to obtain a block-processed radar echo image sequence, where the block-processed radar echo image sequence includes radar echo sub-images of each time step; Sequentially inputting the radar echo sub-images of each time step in chronological order into a trained radar echo image prediction model of PredRNN based on global reinforcement to obtain radar echo prediction images for the future (t + 1)-th time step to the (t + m + 1)-th time step; Wherein, the radar echo image prediction model includes: a first globally reinforced ST-LSTM module, a second globally reinforced ST-LSTM module, a gradient highway module, a third globally reinforced ST-LSTM module, and a fourth globally reinforced ST-LSTM module; The first globally reinforced ST-LSTM module, the second globally reinforced ST-LSTM module, the gradient highway module, the third globally reinforced ST-LSTM module, and the fourth globally reinforced ST-LSTM module are connected in sequence.

2. The method for predicting radar echo images based on global reinforcement PredRNN according to claim 1, wherein The step of sequentially inputting the radar echo sub-images of each time step in chronological order into a trained radar echo image prediction model of PredRNN based on global reinforcement to obtain radar echo prediction images for the future (t + 1)-th time step to the (t + m + 1)-th time step includes: Inputting the radar echo sub-image of the (t - m)-th time step into a trained radar echo image prediction model of PredRNN based on global reinforcement to output a radar echo prediction sub-image of the (t - m + 1)-th time step; Inputting the radar echo sub-image of the (t - m + 1)-th time step into a trained radar echo image prediction model of PredRNN based on global reinforcement to output a radar echo prediction sub-image of the (t - m + 2)-th time step; And so on, until the radar echo sub-image of the (t - m + m)-th time step is input into a trained radar echo image prediction model of PredRNN based on global reinforcement to output a radar echo prediction sub-image of the future (t + 1)-th time step; Then inputting the radar echo prediction sub-image of the (t + 1)-th time step into a trained radar echo image prediction model of PredRNN based on global reinforcement to output a radar echo prediction sub-image of the future (t + 2)-th time step; And so on, then inputting the radar echo prediction sub-image of the (t + m)-th time step into a trained radar echo image prediction model of PredRNN based on global reinforcement to output a radar echo prediction sub-image of the future (t + m + 1)-th time step; Stitch the radar echo prediction sub-images from the (t + 1)-th time step to the (t + m + 1)-th time step to obtain the radar echo prediction image for the future from the (t + 1)-th time step to the (t + m + 1)-th time step.

3. The radar echo image prediction method based on global reinforcement PredRNN according to claim 2, wherein The way the radar echo image prediction model predicts the radar echo prediction sub-image at the i-th time step is as follows: Input the sub-image at the (i - 1)-th time step of the radar echo image prediction model described in the input and the fourth global enhanced spatio-temporal memory generated during the prediction at the (i - 1)-th time step into the first globally reinforced ST-LSTM module. The first globally reinforced ST-LSTM module generates the first hidden state, the first cell state, and the first global enhanced spatio-temporal memory during the prediction at the i-th time step based on the first hidden state, the first cell state, the fourth global enhanced spatio-temporal memory, and the sub-image at the (i - 1)-th time step, where, , when i is to any one of the time steps, the sub-image at the (i - 1)-th time step is the radar echo sub-image at the (i - 1)-th time step, and when i is to any one of the time steps, the sub-image at the i-th time step is the radar echo prediction sub-image at the (i - 1)-th time step; Input the first hidden state and the first global enhanced spatio-temporal memory at the prediction of the i-th time step into the second globally enhanced ST-LSTM module. The second globally enhanced ST-LSTM module generates the second hidden state, the second cell state, and the second global enhanced spatio-temporal memory at the prediction of the i-th time step based on the second hidden state, the second cell state generated at the prediction of the (i - 1)-th time step, the first hidden state, and the first global enhanced spatio-temporal memory at the prediction of the i-th time step. Input the second hidden state generated at the prediction of the i-th time step and the third hidden state generated at the prediction of the (i - 1)-th time step into the gradient highway module. The gradient highway module generates the highway state at the prediction of the i-th time step based on the second hidden state generated at the prediction of the i-th time step and the third hidden state generated at the prediction of the (i - 1)-th time step. Input the highway state at the prediction of the i-th time step and the second global enhanced spatio-temporal memory into the third globally enhanced ST-LSTM module. The third globally enhanced ST-LSTM module generates the third hidden state, the third cell state, and the third global enhanced spatio-temporal memory at the prediction of the i-th time step based on the third hidden state, the third cell state generated at the prediction of the (i - 1)-th time step, the highway state, and the second global enhanced spatio-temporal memory at the prediction of the i-th time step. Input the third hidden state at the prediction of the i-th time step and the third global enhanced spatio-temporal memory into the fourth globally enhanced ST-LSTM module. The fourth globally enhanced ST-LSTM module generates the fourth hidden state, the fourth cell state, and the fourth global enhanced spatio-temporal memory at the prediction of the i-th time step based on the fourth hidden state, the fourth cell state generated at the prediction of the (i - 1)-th time step, the third hidden state, and the third global enhanced spatio-temporal memory at the prediction of the i-th time step. Element-wise add the fourth hidden state at the prediction of the i-th time step and the sub-image at the (i - 1)-th time step to obtain the radar echo prediction sub-image at the i-th time step.

4. The method for predicting radar echo images based on global reinforcement PredRNN according to claim 3, wherein The step of inputting the second hidden state generated at the prediction of the i-th time step and the third hidden state generated at the prediction of the (i - 1)-th time step into the gradient highway module, where the gradient highway module generates the highway state at the prediction of the i-th time step based on the second hidden state generated at the prediction of the i-th time step and the third hidden state generated at the prediction of the (i - 1)-th time step, includes: Perform an addition operation on the second hidden state generated at the prediction of the i-th time step after being processed by the tanh activation function and the third hidden state generated at the prediction of the (i - 1)-th time step to obtain a candidate tensor. The second hidden state generated during the prediction at the i-th time step is processed through the σ activation function and added to the third hidden state generated during the prediction at the (i - 1)-th time step to obtain the first gating weight and the second gating weight; The candidate tensor is weighted according to the first gating weight to obtain a weighted candidate tensor; The third hidden state generated during the prediction at the (i - 1)-th time step is weighted according to the second gating weight to obtain a weighted third hidden state; Feature fusion is performed on the weighted candidate tensor and the weighted third hidden state to generate a highway state during the prediction at the i-th time step.

5. The radar echo image prediction method based on global reinforcement PredRNN according to claim 3, wherein The first globally enhanced ST-LSTM module, the second globally enhanced ST-LSTM module, the third globally enhanced ST-LSTM module, and the fourth globally enhanced ST-LSTM module are globally enhanced ST-LSTM networks with the same structure, and the expression of the processing process of the globally enhanced ST-LSTM network is: ， ， ， ， ， ， ， ， ， ， ， ， ， ， ， ； Among them, is a convolution that maps the input to 7 gate vectors, is a convolution that maps the input to 4 gate vectors, is a convolution that maps the input to 3 gate vectors, where l is the number of the globally enhanced ST-LSTM module, , is the hidden state, highway state, or sub-image input to the l-th globally enhanced ST-LSTM module at the i-th time step prediction, is the hidden state generated by the l-th globally enhanced ST-LSTM module at the (i - 1)-th time step prediction, is the globally enhanced spatio-temporal memory generated by the (l - 1)-th globally enhanced ST-LSTM module at the i-th time step prediction, is from the input gate pre-activation, is from the forget gate pre-activation, is from the candidate memory, is the global memory branch version of the input gate, is the global memory branch version of the forget gate, is the candidate memory of the global memory branch, is the element-wise multiplication, is the contribution to the output gate, is from the pre-activation of the input gate, is from the pre-activation of the forget gate, is from the candidate memory, is the contribution to the output gate, is from the pre-activation of the input gate, is from the pre-activation of the forget gate, is from the candidate memory of the input gate, is the output of the input gate, is the Sigmoid activation function, is the output of the forget gate, is the forget gate bias, is the output of the input modulation gate, is the hyperbolic tangent activation function, is the cell state generated by the l-th globally enhanced ST-LSTM module at the i-th time step prediction, is the cell state generated by the l-th globally enhanced ST-LSTM module at the (i-1)-th time step for prediction. is the output of the input gate of the globally enhanced memory branch. is the output of the forget gate of the globally enhanced memory branch. is the output of the input modulation gate of the globally enhanced memory branch. is the globally enhanced information. is the adaptive average pooling layer. is the fully connected layer. is to broadcast the 1*1 feature back to the same spatial size. is the globally enhanced spatio-temporal memory generated by the l-th globally enhanced ST-LSTM module at the i-th time step for prediction. is the adaptive weight parameter. is the comprehensive memory tensor. is to perform channel dimension concatenation. is the output of the output gate. denotes compressed and fused to obtain the features required to drive the output gate. denotes compressed and fused to obtain the features required to generate the hidden state. is the multi-source self-attention unit. is the preliminary hidden state generated by the l-th globally enhanced ST-LSTM module at the i-th time step for prediction. is the hidden state generated by the l-th globally enhanced ST-LSTM module at the i-th time step for prediction.

6. The method for predicting radar echo images based on global reinforcement PredRNN according to claim 5, wherein The processing process of the multi-source self-attention unit is: The preliminary hidden state generated during the prediction at the i-th time step by the l-th globally enhanced ST-LSTM module and the hidden state generated during the prediction at the (i - 1)-th time step by the l-th globally enhanced ST-LSTM module are added and then adjusted in shape to obtain an adjusted hidden state; The adjusted hidden state is processed through a multi-head self-attention sub-unit to obtain a self-attention enhanced feature; The self-attention enhanced feature and the adjusted hidden state are added and then processed through a normalization layer to obtain a normalized hidden state; The normalized hidden state is processed through a feed-forward neural network and added to the normalized hidden state, and then sequentially processed through a normalization layer and a shape adjustment to obtain the hidden state generated during the prediction at the i-th time step by the l-th globally enhanced ST-LSTM module.

7. The radar echo image prediction method based on global reinforcement PredRNN according to claim 1, characterized in that The training method of the radar echo image prediction model based on globally enhanced PredRNN is: Obtain a sample data set containing multiple radar echo image samples, and each radar echo image sample includes radar echo images at q time steps; Preprocess each radar echo image sample in the sample data set, and combine the radar echo images in each radar echo image sample into a radar echo image sample sequence with a fixed length in chronological order; Each radar echo image at each time step of each radar echo image sample sequence is block-processed to obtain each block-processed sample sequence, and all block-processed sample sequences constitute a processed sample data set. Each block-processed sample sequence includes radar echo sub-images corresponding to radar echo images at q time steps; The processed sample data set is divided into a training set, a validation set, and a test set; Use the block-processed sample sequences in the training set to train the radar echo image prediction model of the globally enhanced PredRNN to be trained to obtain a trained radar echo image prediction model; Use the sample sequences after chunking in the validation set to validate the trained radar echo image prediction model. After adjusting the network parameters of the trained radar echo image prediction model according to the validation results, continue the training until the validation results reach the convergence condition, and obtain a trained radar echo image prediction model based on globally reinforced PredRNN. Use the sample sequences after chunking in the test set to test the trained radar echo image prediction model based on globally reinforced PredRNN, and evaluate the performance of the trained radar echo image prediction model based on globally reinforced PredRNN.

8. The method for predicting radar echo images based on global reinforcement PredRNN according to claim 7, wherein The step of using the sample sequences after chunking in the training set to train the radar echo image prediction model based on globally reinforced PredRNN to be trained, and obtaining the trained radar echo image prediction model includes: Input the radar echo sub-image at the first time step of the chunked sample sequence into the radar echo image prediction model based on globally reinforced PredRNN to be trained for training, and output the radar echo prediction sub-image at the second time step. Input the radar echo sub-image at the second time step of the chunked sample sequence into the radar echo image prediction model based on globally reinforced PredRNN to be trained, and output the radar echo prediction sub-image at the third time step. And so on until the radar echo sub-image at the time step of the segmented sample sequence is input into the radar echo image prediction model of PredRNN based on global reinforcement for training, and the radar echo prediction sub-image at the + 1 time step is output; Then, the radar echo sub-image at the +1 time step and the radar echo prediction sub-image at the +1 time step of the segmented sample sequence are input into the scheduled sampling unit, so that the scheduled sampling unit determines whether to input the radar echo sub-image or the radar echo prediction sub-image at the +1 time step into the radar echo image prediction model of PredRNN based on global reinforcement to be trained, and outputs the radar echo prediction sub-image at the future +2 time steps; And so on. Then input the radar echo sub-image and the radar echo prediction sub-image at the (q - 1)-th time step of the chunked sample sequence into the scheduled sampling unit, so that the scheduled sampling unit determines whether to input the radar echo sub-image or the radar echo prediction sub-image at the (q - 1)-th time step into the radar echo image prediction model based on globally reinforced PredRNN to be trained according to the preset scheduling probability η, and output the radar echo prediction sub-image at the q-th time step.

Images