A short-term precipitation prediction method based on radar echo map extrapolation
By using an improved UNet network model, combined with multi-scale feature embedding and the trans-temporal Transformer algorithm, the problem of insufficient representation of radar echo sequence features was solved, thereby improving the accuracy of short-term precipitation forecasting.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SOUTHEAST UNIV
- Filing Date
- 2022-11-27
- Publication Date
- 2026-06-09
AI Technical Summary
Existing short-term precipitation prediction methods based on radar echoes do not fully extract the spatial and temporal features of radar echo sequences, resulting in low prediction accuracy.
An improved UNet network model is adopted, introducing a multi-scale feature embedding module and a multi-level fusion module. Combined with CNN and Transformer algorithms, the feature extraction capability is enhanced, and the temporal dependency of radar echo sequences is mined through the cross-time Transformer algorithm.
It significantly improves the accuracy of short-term precipitation forecasts, especially in the forecasting of heavy rainfall, thus enhancing the accuracy of predictions.
Smart Images
Figure CN115761261B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of target detection in image processing, specifically relating to a short-term precipitation prediction method based on radar echo image extrapolation. Background Technology
[0002] my country is a country with diverse geographical environments, fragile ecosystems, and complex climate conditions, making it one of the countries most severely affected by extreme weather disasters. At the same time, short-term precipitation forecasting, due to its characteristics of suddenness, complex and diverse mechanisms, and short lifecycle, is also a very complex and challenging field. Therefore, research on short-term precipitation forecasting is of great significance. However, current accuracy and forecast timeliness are still insufficient to meet operational needs. Therefore, further improving the accuracy of short-term forecasts is crucial for people's production and daily life.
[0003] Numerical weather forecasting, as a traditional forecasting method used in my country, can no longer meet the needs of today's rapidly developing economy and society because its temporal and spatial resolution is not fine enough and it can only provide simple qualitative descriptions. More refined short-term forecasts, especially refined forecasts of severe convective weather, have become the main direction of meteorological operational research and development.
[0004] In recent years, with the development of meteorological monitoring equipment and artificial intelligence technology, short-term precipitation forecasting methods based on radar echo data and utilizing machine learning and deep learning techniques have rapidly become a research hotspot. This method does not focus on complex atmospheric dynamic processes; it simply predicts future radar echo sequences using past radar echo sequences. This approach clearly divides short-term precipitation forecasting into two steps: first, predicting future radar echo sequences, and then estimating precipitation based on the obtained echo distribution.
[0005] Deep learning-based RNN methods are widely used for precipitation prediction based on radar echo extrapolation. For example, the ConvLSTM model was proposed by combining CNN and LSTM architectures. Predictive Recurrent Neural Network (PredRNN) models and their improved versions (PredRNN++) have also been proposed for short-term precipitation prediction. However, these RNN-based methods always suffer from the vanishing gradient problem, resulting in low accuracy in short-term precipitation prediction.
[0006] Besides RNN-based methods, CNN-based methods have also been proposed for short-term precipitation prediction. For example, the FCN (Fully Convolution Networks) model based on UNet and the SmaAt-UNet model have been proposed. However, while CNN-based methods are generally effective for extracting local features, they are insufficient for representing global features, thus resulting in low accuracy in short-term precipitation prediction.
[0007] Existing methods leverage the advantages of CNNs focusing on local features and Transformers' self-attention mechanism in mining global features, combining the two for short-term precipitation prediction. An AA-UNetTransUnet network combining U-Net and Transformer is proposed for short-term precipitation prediction, achieving a CSI of 0.6651 (r≥0.5). Furthermore, a Rainformer model combining Swin-Transformer, UNet, and a gate fusion unit is proposed, achieving a CSI of 0.6678 (r≥0.5) for short-term precipitation prediction (the CSI index considers both missed and false alarm errors, with values between 0 and 1; 0 indicates complete failure, and 1 indicates complete success). However, these models do not effectively extract the time-series information of radar echoes, resulting in low accuracy in short-term precipitation prediction. Directly training a short-term precipitation predictor using UNet also results in low accuracy, mainly due to insufficient representation of the spatial and temporal features of the radar echo sequence. Therefore, the original UNet network needs further improvement to meet practical applications. Summary of the Invention
[0008] Objective: This invention aims to provide a short-term precipitation prediction method based on radar echo extrapolation. Firstly, it employs the UNet network model framework and improves its feature encoding blocks. By introducing a multi-scale feature embedding module, it obtains local and global short-term precipitation feature information, thereby increasing the model's multi-scale prediction capability. Secondly, it introduces a multi-level fusion module to achieve information transfer between local, intermediate, and global features at different levels, fully combining the advantages of CNN's focus on local information and Transformer's establishment of global dependencies, improving the model's feature representation in the spatial dimension of radar echo sequences. Thirdly, it uses a cross-temporal Transformer algorithm to mine the temporal dependencies of radar echo sequences, learning the temporal variation trend of short-term precipitation, further enhancing the model's feature representation in the temporal dimension of radar echo sequences. The improved model enhances its ability to extract features from both the spatial and temporal dimensions of radar echo sequences, improving the accuracy of short-term precipitation prediction.
[0009] Technical solution: To achieve this objective, the present invention adopts the following technical solution:
[0010] The present invention discloses a short-term precipitation prediction method based on radar echo image extrapolation, comprising the following steps:
[0011] S1: Collect radar precipitation images and perform data preprocessing on the raw precipitation images;
[0012] S2: Using a multi-scale feature embedding module within the UNet network framework, a feature extraction network front-end is constructed, yielding the output features of three branches, denoted as feature X. l X m X g ;
[0013] S3: Use a multi-level fusion algorithm to fuse feature X l X m X g Construct and extract the middle part of the network;
[0014] S4: Add a time-series Transformer algorithm to build an extraction network backend;
[0015] S5: Transfer learning is performed on the improved network model to obtain a short-term precipitation prediction model based on radar echo map extrapolation.
[0016] As a further technical solution of the present invention, in step S1, the data preprocessing method includes center cropping of the image. The data preprocessing method of the present invention is as follows:
[0017] S11: Collect the raw data of radar precipitation images, and perform center cropping on the raw precipitation images to remove the blank background area at the circular edge to obtain training samples.
[0018] As a further technical solution of the present invention, the method for constructing the front end of the feature extraction network using a multi-scale feature embedding module in step S2 is as follows:
[0019] S21: The first pure convolutional network structure that separates the encoder feature extraction network from the original UNet network;
[0020] S22: Replace the single-branch pure convolutional module separated from S21 with a multi-scale feature embedding module with three branches (two Transformer block branches and one CNN block branch). Connect the output of the previous layer to the input of the multi-scale feature embedding module. This module will compute multiple different transformations on the same mapping in parallel, that is, parallel convolutional block operations and Transformer block operations. Use this module to perform multi-scale learning on a fixed-size image and extract X. l X m X g feature;
[0021] S23: The outputs of the three branches of the multi-scale feature module in S22 are processed to the next layer to obtain the features of the same image at different scales;
[0022] S24: Process the pure convolutional feature extraction network structure in the original UNet network continuously according to S21 to S22 until all pure convolutional feature extraction modules in the original UNet feature extraction network are replaced with multi-scale feature embedding modules. This step ends here.
[0023] As a further technical solution of the present invention, in step S3, a multi-level fusion algorithm is used to fuse feature X. l X m X g The method for constructing and extracting the middle part of the network is as follows:
[0024] S31: The output feature derived from the convolutional branch of the new multi-scale feature embedding module in S2 is denoted as feature X. l The output feature derived from the branch of the Transformer block after two 3x3 convolutions is denoted as feature X. m The output feature derived from the branch of the Transformer block after three 3x3 convolutions is denoted as feature X. g ;
[0025] S32: Feature X l spliced to feature X m middle;
[0026] S33: Feature X m spliced to feature X g middle,;
[0027] S34: The outputs obtained from S32 and S33 are then mapped and transformed by a multilayer perceptron to achieve feature fusion.
[0028] As a further technical solution of the present invention, in step S4, a time-spanning Transformer algorithm is added, and the method for extracting the network backend is as follows:
[0029] 41: Connect the outputs of the three branches of S3 to the inputs of the time-spanning Transformer block respectively;
[0030] S42: Connect the three branches of S31 to the output features of the time-transitional Transformer block, and denote them as X' respectively. l , X' m , X' g The algorithm formula for the Transformer block spanning multiple time intervals is:
[0031]
[0032]
[0033]
[0034] in For adaptive mask parameters;
[0035] S43: The outputs of the three branches in S41 are concatenated and fed into the next layer of the model structure through skip connections and convolution operations.
[0036] As a further technical solution of the present invention, in step S5, the method for performing transfer learning on the improved network model to obtain a short-term precipitation prediction model based on radar echo image extrapolation is as follows:
[0037] S51: Download the pre-trained default parameters and perform fine-tuning operations on the weights from the UNet website, load the resulting parameters into our improved UNet network, and obtain the optimized objective function as follows:
[0038]
[0039] X t+1 , ..., X t+k =f(X) t-j+1 , ..., X t )
[0040] Where X t Let be the radar echo sequence at time t;
[0041] S52: Use the prepared dataset to perform transfer learning on the model and train it until it converges.
[0042] Beneficial Effects: This invention discloses a short-term precipitation prediction method based on radar echo image extrapolation. Firstly, it employs the UNet network model framework, introducing a multi-scale feature embedding module and a multi-level fusion module to integrate multi-scale features from different levels of CNN blocks and Transformer blocks, thereby improving the model's feature representation in the spatial dimension of the radar echo sequence. Furthermore, it uses a cross-temporal Transformer algorithm to mine the temporal dependencies of the radar echo sequence, enhancing the model's feature representation in the temporal dimension of the radar echo sequence. This invention enhances the feature representation capabilities of radar echo sequences in both spatial and temporal dimensions, resulting in a significant improvement in the accuracy of short-term precipitation prediction. Attached Figure Description
[0043] Figure 1 This is a flowchart of the short-term precipitation prediction method based on radar echo image extrapolation of the present invention.
[0044] Figure 2 This is a schematic diagram of the training samples after center cropping according to the present invention.
[0045] Figure 3This is a schematic diagram of the multi-scale feature embedding module of the present invention.
[0046] Figure 4 This is a schematic diagram of the multi-level fusion module of the present invention.
[0047] Figure 5 This is a schematic diagram illustrating the introduction of a time-transitional Transformer module into the UNet coding layer network structure according to the present invention.
[0048] Figure 6 This is a schematic diagram of the overall network framework of the improved UNet coding block of the present invention.
[0049] Figure 7 This is a schematic diagram illustrating the actual prediction performance of the improved UNet coding layer network of this invention. Detailed Implementation
[0050] The technical solution of the present invention will be further described below with reference to specific embodiments and accompanying drawings.
[0051] Example 1: This specific implementation discloses a short-term precipitation prediction method based on radar echo image extrapolation, such as... Figures 1 to 7 As shown, it includes the following steps:
[0052] S1: As Figure 1 As shown, before the samples are input into the network for training, preprocessing is required. Radar precipitation images are collected, and the original precipitation images are cropped to obtain training samples (such as...). Figure 2 (as shown);
[0053] S2: Replace the pure convolutional modules of the original UNet feature extraction network with a multi-scale feature embedding module containing Transformer and CNN blocks to obtain an improved feature extraction network front-end (e.g., Figure 3 As shown), the output features of the three branches are derived, denoted as feature X. l X m X g ;
[0054] S3: Use a multi-level fusion algorithm to fuse feature X l X m X g Construct and extract the middle part of the network (such as Figure 4 As shown), this enables information transfer between different levels and strengthens the original model's feature representation of the spatial dimension of radar echo sequences.
[0055] S4: Add a time-series Transformer algorithm to mine temporal dependencies in radar echo sequences, resulting in an improved feature extraction network backend (such as...). Figure 5(As shown), this enhances the original model's representation of the time dimension of radar echo sequences;
[0056] S5: Download the pre-trained default parameters and perform weight fine-tuning operations from the UNet website and load them into the improved UNet model. Then, adjust the improved UNet network (e.g., ...). Figure 6 (As shown) Transfer learning is performed on the network. The loss function for predicting test set images gradually decreases until convergence, resulting in a short-term precipitation prediction model based on radar echo image extrapolation. The actual prediction effect is illustrated in the diagram. Figure 7 As shown.
[0057] In step S1, the data preprocessing method is image center cropping. The data preprocessing method of this invention is as follows:
[0058] S11: Collect radar precipitation maps. We selected 4,000 sequences as the training set and 1,734 sequences as the validation set.
[0059] S12: For the original radar precipitation images, a center cropping algorithm is used to crop the original images of the network to remove the blank background areas with circular edges, obtaining training and testing samples with a resolution of 288*288. These are then applied to the training of the UNet network. Each training session uses a sequence of 9 frames as input to the model, and another 9 frames as ground truth (GT). t-j+1 ,...,X t This is a sequence of radar echoes at j consecutive timestamps input to the network.
[0060] In step S2, the method of replacing the pure convolutional module of the original UNet feature extraction network with a multi-scale feature embedding module containing Transformer and CNN blocks is as follows:
[0061] S21: The first pure convolutional network structure that separates the encoder feature extraction network from the original UNet network;
[0062] S22: Replace the single-branch pure convolutional module separated from S21 with a multi-scale feature embedding module with three branches (two Transformer block branches and one CNN block branch). Connect the output of the previous layer to the input of the multi-scale feature embedding module. This module computes multiple different transformations on the same mapping in parallel, i.e., parallel convolutional blocks and Transformer blocks. The convolutional block branch (i.e., local feature block) consists of one 1x1 convolution, one 3x3 convolution, and one 1x1 convolution. The intermediate feature branch of the Transformer consists of two 3x3 convolutional layers, layer normalization (LN), multi-head attention mechanism (MHSA), and feedforward network (FFN). The global feature branch of the Transformer consists of three 3x3 convolutional layers, layer normalization (LN), multi-head attention mechanism (MHSA), and feedforward network (FFN). This module is used to perform multi-scale learning on a fixed-size image and extract X. l X m X g feature;
[0063] S23: The outputs of the three branches of the multi-scale feature module in S22 are processed to the next layer to obtain the features of the same image at different scales;
[0064] S24: Process the pure convolutional feature extraction network structure in the original UNet network continuously according to S21 to S22 until all pure convolutional feature extraction modules in the original UNet feature extraction network are replaced with multi-scale feature embedding modules. This step ends here.
[0065] In step S3, a multi-level fusion algorithm is used to fuse feature X. l X m X g The method for constructing and extracting the middle part of the network is as follows:
[0066] S31: The output feature derived from the convolutional branch of the new multi-scale feature embedding module in S2 is denoted as feature X. l The output feature derived from the intermediate feature branch of the Transformer block after two 3x3 convolutions is denoted as feature X. m The output feature derived from the global feature branch of the Transformer block after three 3x3 convolutions is denoted as feature X. g ;
[0067] S32: Feature X l spliced to feature X m middle;
[0068] S33: Feature X m spliced to feature X g middle,;
[0069] S34: The outputs obtained from S32 and S33 are then mapped and transformed by a multilayer perceptron to achieve feature fusion. This feature output can obtain richer spatial dimension features of the radar echo sequence.
[0070] In step S4, a time-spanning Transformer algorithm is added, and the method for extracting the network backend is constructed as follows:
[0071] S41: Connect the outputs of the three branches of S3 to the inputs of the time-spanning Transformer block respectively;
[0072] S42: Connect the three branches of S31 to the output features of the time-transitional Transformer block, and denote them as X' respectively. l , X' m , X' g The algorithm formula for the Transformer block spanning multiple time intervals is:
[0073]
[0074]
[0075]
[0076] in An adaptive masking strategy;
[0077] S43: The outputs of the three branches in S41 are concatenated and fed into the next layer of the model structure through skip connections and convolution operations.
[0078] In step S5, the improved UNet model is trained through transfer learning to obtain the intelligent prediction model. The method is as follows:
[0079] S51: Download the pre-trained default parameters and perform fine-tuning operations on the weights from the UNet website, load the resulting parameters into our improved UNet network, and obtain the optimized objective function as follows:
[0080]
[0081] X t+1 , ..., X t+k =f(X) t-j+1 , ..., X t )
[0082] Where X t Let be the radar echo sequence at time t;
[0083] S52: Transfer learning was performed on the model using the prepared dataset, and training was continued until convergence. An NVIDIA RTX 3090 GPU was configured on the server to train and test our model. The initial learning rate was set to 0.0001, and the stochastic gradient descent method was implemented using the Adam optimizer. We used 18 radar echo sequences as mini-batch input data and employed Balanced Mean Absolute Error (B-MAE) as the validation loss function. When the validation loss no longer decreased during training, the model with the smallest validation loss was selected as the best training prediction model.
[0084] This invention is compared with the state-of-the-art (SOTA) standards, as shown in Tables 1, 2, and 3. The best performance is highlighted in bold. For ease of comparison, results with r ≥ 10 are multiplied by 10, and results with r ≥ 30 are multiplied by 100. Compared with other SOTA standards, our model achieves the best performance level across the four evaluation metrics (CSI, HSS, MSE, MAE).
[0085] Table 1. Comparison of CSI results between the present invention and existing algorithms.
[0086]
[0087] Table 2. HSS Comparison Results between the Invention and Existing Algorithms
[0088]
[0089] Table 3 Comparison of MSE & MAE between the present invention and existing algorithms
[0090] method B-MSE↓ B-MAE↓ ConvLSTM 18.4430 1.2656 PredRNN++ 17.7672 1.2332 SmaAt-Unet 17.5860 1.2162 AA-TransUnet 17.0093 1.2032 Rainformer 16.6708 1.1836 This invention 15.9018 1.1709
[0091] In summary, this invention improves the UNet model using several state-of-the-art methods, achieving the best performance across all four prediction accuracy metrics. Crucially, it demonstrates significant improvements at the 10mm / h and 30mm / h thresholds (heavy rainfall). Compared to the superior PredRNN++ method, the proposed method significantly enhances both the CSI and HSS for heavy rainfall (30mm / h) predictions. Even with the newly proposed Rainformer method, at the 30mm / h threshold, CSI and HSS (skill score of forecast, a metric comparing actual forecasts with stochastic or persistent forecasts to measure the quality of forecasting methods; higher is better) can achieve improvements exceeding 4.2% and 3.1%, respectively. This enhances the accuracy of short-term precipitation forecasting, enabling the detection model to accurately predict the point intensity and scale of short-term precipitation in complex and variable weather conditions, thus meeting the accuracy requirements for short-term precipitation forecasting.
[0092] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any transformations or substitutions that can be conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for predicting short-term precipitation based on radar echo image extrapolation, characterized in that: Includes the following steps: S1: Collect radar precipitation images and perform data preprocessing on the raw precipitation images; S2: Using a multi-scale feature embedding module within the UNet network framework, a feature extraction network front-end is constructed, yielding the output features of three branches, denoted as features. ; S3: Feature fusion is achieved using a multi-level fusion algorithm. Construct and extract the middle part of the network; S4: Add a time-series Transformer algorithm to build an extraction network backend; S5: Transfer learning is performed on the improved network model to obtain a short-term precipitation prediction model based on radar echo map extrapolation. In step S2, the method for constructing the front end of the feature extraction network using the multi-scale feature embedding module is as follows: S21: The first pure convolutional network structure that separates the encoder feature extraction network from the original UNet network; S22: Replace the single-branch pure convolutional module separated from S21 with a multi-scale feature embedding module with three branches, including two Transformer block branches and one CNN block branch. Connect the output of the previous layer to the input of the multi-scale feature embedding module. This module will compute multiple different transformations on the same mapping in parallel, that is, parallel convolutional block operations and Transformer block operations. Use this module to perform multi-scale learning and extract features from a fixed-size image. feature; S23: The outputs of the three branches of the multi-scale feature module in S22 are processed to the next layer to obtain the features of the same image at different scales; S24: Process the pure convolutional feature extraction network structure in the original network continuously according to S21 to S22 until all pure convolutional feature extraction modules in the original UNet feature extraction network are replaced with multi-scale feature embedding modules. This step ends here. In step S3, a multi-level fusion algorithm is used to fuse features. The method for constructing and extracting the middle part of the network is as follows: S31: The output features derived from the convolutional branch of the new multi-scale feature embedding module in S2 are denoted as features. The output features derived from the branch of the Transformer block after two 3x3 convolutions are denoted as features. The output features derived from the branch of the Transformer block after three 3x3 convolutions are denoted as features. ; S32: Features spliced to features middle; S33: Features spliced to features middle; S34: The outputs obtained from S32 and S33 are then mapped and transformed through a multilayer perceptron to achieve feature fusion; In step S4, a time-spanning Transformer algorithm is added, and the method for extracting the network backend is constructed as follows: S41: Connect the outputs of the three branches of S3 to the inputs of the time-spanning Transformer block respectively; S42: Connect the three branches of S31 to the output features of the time-spanning Transformer block, and denote them as follows: , , The algorithm formula for the Transformer block spanning multiple time intervals is: in , , For adaptive mask parameters; S43: The outputs of the three branches in S41 are concatenated and fed into the next layer of the model structure through skip connections and convolution operations.
2. The method for short-term precipitation prediction based on radar echo image extrapolation according to claim 1, characterized in that, In step S1, the data preprocessing method is as follows: S11: Collect radar precipitation images, and crop the original precipitation images to remove the blank background area at the circular edge to obtain training samples.
3. The method for short-term precipitation prediction based on radar echo image extrapolation according to claim 1, characterized in that, In step S5, the improved UNet model is trained through transfer learning to obtain a short-term precipitation prediction method based on radar echo image extrapolation: S51: Download the pre-trained default parameters and perform fine-tuning operations on the weights from the UNet website, load the resulting parameters into our improved UNet network, and obtain the optimized objective function as follows: in for The radar echo sequence at any given time; S52: Use the prepared dataset to perform transfer learning on the model and train it until it converges.