Hybrid perception based timing attention precipitation forecasting method

Abstract

The application provides a timing attention precipitation prediction method based on a hybrid perception. In view of the problem that the U-Net skip connection in the prior art is difficult to effectively fuse multi-modal spatio-temporal features, a multi-modal spatio-temporal attention module and a hybrid multi-layer perception module are used to replace the U-Net skip connection. Specifically, the multi-modal spatio-temporal attention module is introduced, the key spatial features in a single frame are extracted through a static attention mechanism, and a dynamic attention mechanism is combined to model the inter-frame temporal dependence relationship, so that the evolution law of the precipitation system can be accurately captured. Meanwhile, the hybrid multi-layer perception module is used to fuse channel interaction, spatial convolution and residual connection, and the expression ability of multi-scale features is optimized. In addition, in order to solve the aliasing artifact problem generated in the decoder upsampling process, a pixel rearrangement module is introduced, and the clarity of the reconstructed precipitation field image is significantly improved. Experimental results show that the method has more excellent performance compared with the existing precipitation prediction method.

Description

Technical Field

[0001] This invention relates to the field of meteorological precipitation forecasting, and more particularly to a time-series attention-based precipitation forecasting method based on hybrid sensing. Background Technology

[0002] Against the backdrop of global climate change, the intensity and frequency of extreme weather events are increasing in many parts of the world, leading to frequent torrential rains, floods, and droughts, posing severe challenges to the natural environment and human society. In particular, the increased frequency of extreme precipitation events such as torrential rains poses a serious threat to water supply, food security, and human life in water-rich areas. Therefore, the study of precipitation, especially heavy precipitation, has significant social value.

[0003] Currently, precipitation forecasting mainly relies on three types of technologies. The first is numerical weather prediction, which simulates atmospheric evolution based on physical equations, but its resolution and timeliness are still insufficient in capturing short-term, sudden heavy precipitation. The second is statistical methods, which rely on historical observation data for modeling and are difficult to adapt to rapidly changing weather systems. The third is radar echo extrapolation methods, which, due to their high spatiotemporal resolution, have become the core means of short-term nowcasting. However, traditional radar echo extrapolation methods (such as the cross-correlation method TREC and optical flow method) have limited performance in dealing with nonlinear evolution processes such as cloud rotation, scale changes, and formation and dissipation, especially exhibiting poor stability under complex weather conditions such as strong convection, with prediction accuracy rapidly decreasing with forecast duration.

[0004] In recent years, deep learning models have demonstrated excellent performance and broad application potential in precipitation forecasting. From 2015 to 2019, researchers applied deep learning networks such as LSTM, U-Net, and fully convolutional neural networks to precipitation forecasting. However, early deep learning models directly regressed precipitation intensity, resulting in results lacking physical plausibility and failing to model the uncertainties of extreme precipitation events. Subsequently, scholars proposed introducing attention mechanisms into precipitation forecasting. Deep generative radar extrapolation models such as DGMR, SmatUnet, and MetNet all utilize attention mechanisms, which have become a common tool for improving model robustness. CBAM is a representative network among these.

[0005] Shortcomings of existing technology:

[0006] 1. The predictive ability for heavy precipitation events is limited, and models based on recursive structures (such as ConvLSTM and Pred RNN) struggle to maintain the physical plausibility and accuracy of their predictions over extended periods during recursive forecasting. Deep learning models (especially those based on loss functions such as mean squared error) tend to predict near-average conditions, easily underestimating the intensity and extent of extreme precipitation or overestimating the intensity of weak precipitation. Existing models perform well within the spatiotemporal patterns covered by the training data, but their extrapolation prediction ability significantly decreases for conditions outside the training distribution (such as unprecedented weather patterns or exceptionally strong systems). During recursive forecasting, small initial errors amplify exponentially with the prediction step size (time), causing the prediction quality to rapidly degrade from 20 minutes to several hours. Therefore, it is necessary to establish models with strong spatial feature extraction capabilities and efficient temporal dependency models. Effectively capturing inter-frame dynamic changes in the temporal dimension enhances the continuity and stability of temporal feature representation. Strengthening location dependencies in the spatial dimension is also crucial.

[0007] 2. The network lacks sufficient ability to finely characterize local meteorological features, resulting in insufficient ability to accurately predict rainfall under complex underlying weather conditions. Unlike general image recognition, the occurrence and development of rainfall are highly dependent on local meteorological conditions and underlying surface features. However, many current deep learning-based rainfall models use backbone networks (such as standard ResNet and U-Net variants) primarily designed for general visual tasks, and have limitations in capturing highly localized and non-uniformly distributed key features in complex meteorological fields. Therefore, it is necessary to design or introduce network components with stronger local feature extraction and context modeling capabilities, specifically for finely capturing and characterizing key local meteorological factors affecting rainfall. Summary of the Invention

[0008] To address the shortcomings of existing technologies, this invention proposes a time-series attention-based precipitation prediction method based on hybrid sensing. This method constructs a time-series attention network, which introduces a multi-scale attention mechanism to enhance the local feature representation capability of radar echo images. Simultaneously, a hybrid sensing module, including a multimodal spatiotemporal attention module and a hybrid multilayer sensing module, replaces the skip-connection layer. Specifically, the multimodal spatiotemporal attention module extracts key spatial features of single-frame radar echoes through static spatial attention and combines it with dynamic time-series attention to model inter-frame evolution dependencies, thereby capturing the spatiotemporal evolution patterns of the precipitation system. The hybrid multilayer sensing module fuses shallow and deep features to improve the network's ability to locate rainfall. Finally, pixel rearrangement operations eliminate aliasing artifacts introduced during the upsampling process, improving the quality and spatiotemporal consistency of the reconstructed precipitation field image. The method specifically includes:

[0009] Step 1: Prepare the dataset required for precipitation nowcasting, and extract the maximum reflectance data at different elevation angles to synthesize a combined reflectance image;

[0010] Step 2: Preprocess the dataset obtained in Step 1, locate the local precipitation area using a sliding window method, divide it into training set, validation set and test set according to the agreed ratio, and perform data augmentation and normalization on the three datasets.

[0011] Step 3: Construct and initialize a temporal attention-based precipitation prediction network based on radar echo data. The prediction network consists of a multi-scale attention module, a hybrid sensing module, and a pixel rearrangement module. The multi-scale attention module backbone performs multi-level feature extraction and compression on the input image, gradually transforming the original image information into a high-dimensional, abstract feature representation. The multi-modal spatiotemporal attention module backbone extracts key spatial features of single-frame radar echoes through static spatial attention and combines dynamic temporal attention to model inter-frame evolution dependencies to capture the spatiotemporal evolution patterns of the precipitation system. The hybrid multi-layer sensing module integrates channel interaction and spatial modeling functions, making full use of the multi-scale features generated by the multi-modal spatiotemporal attention module to achieve efficient feature mixing and cross-channel mapping. The pixel rearrangement module is used to decode the features extracted by the previous two networks and output the predicted radar echo image.

[0012] Step 4: Input the training set processed in Step 2 into the constructed prediction network to train the network, specifically including:

[0013] Step 41: Input the original radar echo images in the training set into the multi-scale attention module to extract feature information. Then, obtain feature maps of different scales by passing downsampling and convolution blocks in sequence, and output the first feature map T1, the second feature map T2, the third feature map T3, the fourth feature map T4 and the fifth feature map T5.

[0014] Step 42: Input feature maps T1, T2, and T3 of different scales into the hybrid sensing module for feature fusion. Specifically, after processing, the first feature map T1, the second feature map T2, and the third feature map T3 are respectively obtained as the first hybrid feature map C1, the second hybrid feature map C2, and the third hybrid feature map C3.

[0015] Step 43: Concatenate the third hybrid feature map C3 and the fifth feature map T5 through channels, and then pass them through the pixel rearrangement module to obtain the third decoded feature map W. out3 The second mixed feature map C2 and the third decoded feature map W are combined. out3 The second decoded feature map W is obtained by concatenating channels and then rearranging pixels. out2 The first mixed feature map C1 and the second decoded feature map W are combined. out2 The first decoded feature map W is obtained by concatenating channels and then rearranging pixels. out1 ;

[0016] Step 44: Combine the first feature map T1 with the first decoded feature map W out1 The output feature map F is obtained by fusing the features using the feature summation method. F is then adjusted by 3×3 convolution to adjust the number of channels in the output feature map, and the predicted radar echo image P is output.

[0017] Step 5: Calculate the loss value between the predicted radar echo image P and the actual radar echo image;

[0018] Step 6: The precipitation prediction network is trained with the initially set model parameters. The network automatically changes the network model parameters based on the validation loss value obtained in Step 5. After a preset number of training rounds, the validation set loss value gradually stabilizes. The current training model parameters are saved, and training is stopped.

[0019] Step 7: Select the network that performs best on the validation set, then feed the test set into the network, calculate the mean absolute error, peak signal-to-noise ratio and perceptual similarity ratio, predict the average value of the three indicators of ten radar echo images and the three indicator values ​​of ten single-frame radar echo images, and save the final prediction result.

[0020] According to a preferred embodiment, the data processing flow of the multi-scale attention module includes:

[0021] Step 411: The multi-scale attention module consists of three feature extraction layers, each of which comprises a downsampling layer and a double convolutional block;

[0022] Step 412: The first feature map T1 is obtained by passing the input radar echo image through a double convolutional block, wherein the double convolutional block is a combination of two consecutive convolutional blocks, and each convolutional block includes: 3×3 convolution, batch normalization and linear rectified function;

[0023] Step 413: Next, the first feature map T1 is processed through the first feature extraction layer to obtain T2. ​​The feature extraction layer includes a 2×2 max pooling layer and a double convolutional block. Similarly, T2 is processed through the second feature extraction layer to obtain T3, and T3 is processed through the third feature extraction layer to obtain T4.

[0024] Step 414: Finally, T4 is processed by a convolutional block to obtain feature map T5.

[0025] According to a preferred embodiment, the data processing of the multimodal spatiotemporal attention module includes:

[0026] Step 421: The hybrid sensing module includes a multimodal spatiotemporal attention module and a hybrid multilayer sensing module, which are connected in the form of residuals;

[0027] Step 422: Input the first feature map T1, the second feature map T2 and the third feature map T3 into the multimodal spatiotemporal attention module respectively. First, adjust the channel dimension through batch normalization layer, 1×1 convolution and Gaussian error linear unit activation function to obtain feature map W;

[0028] Step 423: The multimodal spatiotemporal attention module includes two branches, specifically a static attention branch and a dynamic attention branch. The feature map W is simultaneously input into both branches.

[0029] In the static attention branch, W sequentially passes through depthwise separable convolution, depthwise expanded convolution, and 1×1 convolution. First, it expands the receptive field to model intra-frame spatial information. Then, it adjusts the input feature map W through a 1×1 convolution kernel, finally obtaining the static attention weight vector W. sta ;

[0030] In the dynamic attention branch, W first passes through an adaptive average pooling layer to obtain global information about the channels, and then learns the correlation between channels through a fully connected layer with a bottleneck structure, outputting a dynamic attention weight vector W. dyn Finally, W and W sta and W dyn Multiply the results, adjust the number of channels by a 1×1 convolution, multiply the result by the channel scaling parameter, and then perform residual concatenation with the input feature map to obtain feature map M;

[0031] Step 424: Feature map M is then fed into the hybrid multilayer perception module. After convolution, the resulting perceptual feature map is multiplied by the channel scaling parameter and then residually connected with feature map M to obtain hybrid feature map C. The input features are feature map M1, second feature map M2 and third feature map M3. After processing by the multilayer perception module, the first hybrid feature map C1, the second hybrid feature map C2 and the third hybrid feature map C3 are obtained respectively.

[0032] According to a preferred embodiment, the data processing of the pixel rearrangement module specifically includes:

[0033] Step 431: First, concatenate the fifth feature map T5 and the third mixed feature map C3 along the channel dimension, and then pass them into the pixel rearrangement module to obtain W. out3 ;

[0034] Step 432: Place W out3 The second mixed feature map C2 is first concatenated along the channel dimension, and then fed into the pixel rearrangement module to obtain W. out2 Similarly, W out2 After concatenating the first mixed feature map C1 along the channel dimension, it is fed into the pixel rearrangement module to obtain W. out1 .

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] 1. To address the issue of insufficient depth in temporal modeling, this invention innovatively proposes a Multimodal Spatiotemporal Attention Module (MM-TA) to solve this problem. The MM-TA module includes two complementary attention mechanisms: a static attention mechanism and a dynamic attention mechanism. The static attention mechanism focuses on spatial features within the frame, while the dynamic attention mechanism explicitly simulates the temporal dependence of precipitation to accurately capture precipitation evolution patterns.

[0037] 2. To address the issue of insufficient prediction accuracy, this invention innovatively proposes a hybrid multilayer perception module to integrate spatial modeling, channel interaction, and nonlinear representation capabilities, effectively enhancing the model's ability to represent and model complex dynamic scenes such as precipitation. Furthermore, a residual connection mechanism is used to integrate the channel modeling capabilities of the multimodal spatiotemporal attention module and the hybrid multilayer perception module, preserving hierarchical spatial information while optimizing input multi-scale features through channel feature interaction, spatial convolution, and residual connections. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the precipitation prediction network structure proposed in this invention;

[0039] Figure 2 This is a schematic diagram of the structure of a multimodal spatiotemporal attention module;

[0040] Figure 3 This is a schematic diagram of the structure of a hybrid multilayer sensing module;

[0041] Figure 4 This is a comparison chart of the predictions for the first hour and the next hour on a radar dataset based on the present invention.

[0042] Figure 5 This is a comparison chart of the predicted 2-meter temperature on the Weather Bench dataset based on the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0044] The TAU in this invention refers to Temporal Attention Unit, which is an existing efficient spatiotemporal prediction network.

[0045] In this invention, MM-TA refers to Multi-Modal Spatiotemporal Attention.

[0046] The MixMLP of this invention refers to Mixed Multi-Layer Perceptron.

[0047] This invention relates to the field of nowcasting research, primarily addressing the problems of insufficient prediction accuracy and inadequate temporal modeling depth. Deep learning, as one of the mainstream methods in nowcasting precipitation, has demonstrated superior performance far exceeding traditional physical methods. However, most neural network architectures in deep learning lack sufficient ability to finely characterize local meteorological features, making it difficult to maintain the physical plausibility and accuracy of predictions over long periods.

[0048] To address the problem that U-Net skip connections in existing technologies struggle to effectively fuse multimodal spatiotemporal features, this invention proposes replacing them with a multimodal spatiotemporal attention module and a hybrid multilayer sensing module. Furthermore, to address the insufficient prediction accuracy and temporal modeling depth of existing precipitation prediction networks, this invention proposes a precipitation prediction network based on spatiotemporal feature extraction. Figure 1 This is a schematic diagram of the network structure proposed in this invention. Its input data is a pre-processed set of the first ten spatiotemporal sequences, and its output is a predicted set of the next ten spatiotemporal sequences. The input image resolution is 1×64×64, where 1 represents the number of channels, and 64 represents the image height and width, respectively.

[0049] The core and main innovation of the target detection network in this invention lies in proposing a hybrid perception mechanism, which replaces the U-Net skip connections by connecting the multimodal spatiotemporal attention module (MM-TA) and the hybrid multilayer perception module (MixMLP) in a residual manner.

[0050] This invention proposes a temporal attention-based precipitation prediction method based on radar echo data. The constructed detection network TAUNet comprises three parts: a multi-scale attention module, a hybrid sensing module (multimodal spatiotemporal attention module MM-TA + hybrid multilayer sensing module MixMLP), and a pixel rearrangement module. The multi-scale attention module performs multi-level feature extraction and compression on the input image, gradually transforming the original image information into a high-dimensional, abstract feature representation, thereby more accurately extracting target features. The multimodal spatiotemporal attention module MM-TA extracts key spatial features of single-frame radar echoes through static spatial attention and combines it with dynamic temporal attention to model inter-frame evolution dependencies, capturing the spatiotemporal evolution patterns of the precipitation system. The hybrid multilayer sensing module MixMLP integrates channel interaction and spatial modeling functions, fully utilizing the multi-scale features generated by the multimodal spatiotemporal attention module MM-TA to achieve efficient feature mixing and cross-channel mapping. The pixel rearrangement module decodes the features extracted by the previous two networks and outputs the predicted detection image. The method specifically includes:

[0051] Step 1: Prepare the dataset required for nowcasting precipitation, namely the Xichang radar echo dataset. The Xichang radar station records data every 6 minutes, covering a radius of 230km, an elevation angle of 0.5°, and a tangential resolution of 0.15km.

[0052] Step 11: The radar acquires reflectivity data from low to high altitudes through multiple scans at different elevation angles. In this experiment, the maximum reflectivity value is selected for each grid, and the reflectivity data from different elevation angles are combined into a 1000×1000 composite reflectivity image. The dataset resolution is 1×1000×1000.

[0053] Step 2: Preprocess the dataset obtained in Step 1, locate the local precipitation area using the sliding window mechanism, divide it into training set, validation set and test set according to the agreed ratio, and perform data augmentation processing on the training set, validation set and test set.

[0054] Due to the large image size and the lack of precipitation in most areas, a sliding window mechanism was used to locate local precipitation areas. The window size was 64×64, and 20 frames before and after this area were extracted to form a sequence, with the first 10 frames used as input and the last 10 frames as the prediction target. The training set consisted of 18,705 sequences, the validation set of 2,672 sequences, and the test set of 5,344 sequences.

[0055] For the Xichang radar echo dataset, the data preprocessing method uses max-min normalization to normalize the data to between 0 and 1.

[0056] Step 3: Construct and initialize a precipitation prediction network based on spatiotemporal feature extraction. The prediction network consists of a multi-scale attention module, a multimodal spatiotemporal attention machine module (MM-TA), a hybrid multilayer perception module (MixMLP), and a pixel rearrangement module (PixelShuffle).

[0057] The multi-scale attention module extracts multi-scale features from radar echo data. The multimodal spatiotemporal attention module (MM-TA) is based on the feature extraction backbone of the temporal attention unit (TAU). Based on multi-scale features, it introduces a static attention mechanism integrating deep convolution and deep extended convolution to model intra-frame spatial information, ensuring independent operation of each frame even without temporal background. A dynamic attention mechanism is introduced, integrating average pooling and fully connected layers to learn inter-frame dependencies, generate frame importance weights, enhance key evolutionary states, and reduce noise in the time series. The hybrid multilayer perceptron (MixMLP) module integrates spatial modeling, channel interaction, and nonlinear expression capabilities, effectively enhancing the model's representation and modeling capabilities for complex dynamic scenes such as precipitation. The pixel rearrangement module rearranges low-resolution, high-channel-count feature maps into high-resolution, low-channel-count feature maps, transferring information from the channel dimension to the spatial dimension, thus achieving an upsampling effect.

[0058] Step 4: Input the training set processed in Step 2 into the constructed precipitation prediction network to train the network, specifically including:

[0059] Step 41: Input the original radar echo images in the training set into the multi-scale attention module to extract feature information. Then, obtain feature maps of different scales by passing downsampling and convolution blocks in sequence, and output the first feature map T1, the second feature map T2, the third feature map T3, the fourth feature map T4 and the fifth feature map T5.

[0060] Step 411: The multi-scale attention module consists of three feature extraction layers, each consisting of a downsampling layer and a double convolutional block. Each convolutional block uses a 3×3 convolution kernel to adjust the input feature map.

[0061] Step 412: The first feature map T1 is obtained by passing the input radar echo image through a double convolutional block, wherein the double convolutional block is a combination of two consecutive convolutional blocks, and each convolutional block includes: 3×3 convolution, batch normalization and linear rectified function;

[0062] Step 413: Next, the first feature map T1 is processed through the first feature extraction layer to obtain T2. ​​The feature extraction layer includes a 2×2 max pooling layer and a double convolutional block. Similarly, T2 is processed through the second feature extraction layer to obtain T3, and T3 is processed through the third feature extraction layer to obtain T4.

[0063] Step 414: Finally, T4 is processed by a convolutional block to obtain T5. The convolutional block is a 3×3 convolution, batch normalization, and linear rectified function.

[0064] Step 42: Input the feature maps T1, T2, and T3 of different scales into the multimodal spatiotemporal attention module for feature fusion. Specifically, the first feature map T1, the second feature map T2, and the third feature map T3 are input into the hybrid perception module to obtain the first hybrid feature map C1, the second hybrid feature map C2, and the third hybrid feature map C3, respectively.

[0065] Step 421: The hybrid sensing module includes a multimodal spatiotemporal attention module and a hybrid multilayer sensing module, which are connected in the form of residuals;

[0066] Figure 2 This is a schematic diagram of the multimodal spatiotemporal attention mechanism MM-TA structure proposed in this invention. This module takes the backbone outputs T1, T2 and T3 as inputs and features W1, W2 and W3 as outputs, effectively capturing dynamic changes between frames and enhancing the continuity and stability of temporal feature representation.

[0067] Figure 3 This is a schematic diagram of the Mixed Multilayer Perception Module (MixMLP) introduced in this invention. It optimizes input multi-scale features through channel feature interaction, spatial convolution, and residual connections.

[0068] Step 422: Input the first feature map T1, the second feature map T2 and the third feature map T3 into the multimodal spatiotemporal attention module MM-TA. First, adjust the channel dimension through batch normalization layer, 1×1 convolution and Gaussian error linear unit activation function to obtain feature map W.

[0069] Step 423: The Multimodal Spatiotemporal Attention Module (MM-TA) includes two branches: a static attention branch and a dynamic attention branch. The feature map W is simultaneously input into both branches. In the static attention branch, W sequentially passes through a depthwise separable convolution, a depth-expanding convolution, and a 1×1 convolution. This first expands the receptive field to model intra-frame spatial information, then adjusts the input feature map W through a 1×1 convolution kernel, finally yielding the static attention weight vector W. sta In the dynamic attention branch, the feature map W first passes through an adaptive average pooling layer to obtain global information about the channels, and then learns the correlation between channels through a fully connected layer with a bottleneck structure (i.e., dimensionality reduction followed by dimensionality increase), outputting a dynamic attention weight vector W. dyn This is used to adjust the channel weights of the original feature map. Finally, W, W sta W dynThe results are multiplied, and the number of channels is adjusted by a 1×1 convolution. Then, the result is multiplied by the channel scaling parameter and residually concatenated with the input feature map to obtain the feature map M.

[0070] like Figure 1 , Figure 2 As shown, the input data consists of the first feature map T1, the second feature map T2, and the third feature map T3. After processing, feature maps M1, M2, and M3 of the same size are obtained respectively.

[0071] Step 424: Feature map M is then fed into a hybrid multilayer perceptron module, which specifically consists of a 1×1 convolution, a 3×3 depthwise separable convolution, and a 1×1 convolution. The resulting perceptron feature map is multiplied by the channel scaling parameter and then residually concatenated with feature map M to obtain a hybrid feature map C. The input features are feature map M1, the second feature map M2, and the third feature map M3. After processing by the multilayer perceptron module, the first hybrid feature map C1, the second hybrid feature map C2, and the third hybrid feature map C3 are obtained, respectively.

[0072] like Figure 3 As shown, the input features are feature map M1, second feature map M2 and third feature map M3. After the same processing, the first mixed feature map C1, second mixed feature map C2 and third mixed feature map C3 are obtained.

[0073] Step 43: Concatenate the third hybrid feature map C3 and the fifth feature map T5 through channels, and then pass them through the pixel rearrangement module to obtain the third decoded feature map W. out3 The second mixed feature map C2 and the third decoded feature map W are combined. out3 The second decoded feature map W is obtained by concatenating channels and then rearranging pixels. out2 The first mixed feature map C1 and the second decoded feature map W are combined. out2 The first decoded feature map W is obtained by concatenating channels and then rearranging pixels. out1 ;

[0074] The pixel rearrangement module specifically consists of: transposed convolution, pixel rearrangement, group normalization, and Swish activation function.

[0075] Step 431: First, concatenate the fifth feature map T5 and the third mixed feature map C3 along the channel dimension, and then pass them into the pixel rearrangement module to obtain W. out3 .

[0076] Step 432: Place W out3 The second mixed feature map C2 is first concatenated along the channel dimension, and then fed into the pixel rearrangement module to obtain W. out2 Similarly, W out2After concatenating the first mixed feature map C1 obtained in step 423 along the channel dimension, it is fed into the pixel rearrangement module to obtain W. out1 .

[0077] Step 44: Combine the first feature map T1 with the first decoded feature map W out1 The output feature map F is obtained by fusing the features using the feature summation method. F is then adjusted by 3×3 convolution to adjust the number of channels in the output feature map, and the predicted radar echo image P is output.

[0078] Step 5: Calculate the loss value between the predicted radar echo image P and the actual radar echo image;

[0079] Step 6: The precipitation prediction network is trained with the initially set model parameters. The network automatically changes the network model parameters based on the validation loss value obtained in Step 5. After a preset number of training rounds, the validation set loss value gradually stabilizes. The current training model parameters are saved, and training is stopped.

[0080] Step 7: Select the network that performs best on the validation set, then feed the test set into the network, calculate the mean absolute error, peak signal-to-noise ratio and perceptual similarity ratio, predict the average value of the three indicators of ten radar echo images and the three indicator values ​​of ten single-frame radar echo images, and save the final prediction result.

[0081] To further illustrate the effectiveness of the method of this invention, it is compared with other existing methods. For a fair comparison, the officially released code of other methods is used, and their experimental settings are followed, with all methods implemented in the same computational environment and subjected to both quantitative and qualitative analysis. The eight methods are as follows: Method 1 is PredRNN, which proposes a novel recurrent structure, ST-LSTM, for modeling complex spatiotemporal dynamics, maintaining both temporal and spatial memory within the recurrent units; Method 2 is PredRN-v2, an improvement on PredRNN by introducing a "backward recursive path" and a "gradient highway" mechanism to address the gradient explosion problem when training deep PredRNNs; Method 3 is MAU, focusing on modeling motion information by decomposing the prediction task into two sub-tasks: motion estimation and content generation; Method 4 is STCINet, utilizing the interaction of 3D and 2D convolutions to model spatiotemporal dependencies, avoiding the use of recurrent structures and improving parallel computation efficiency; and Method 5 is SimVP, proposing a simple model based on an encoder-predictor-decoder architecture. Method 6 is a lightweight, non-recurrent spatiotemporal sequence prediction model with fast extrapolation speed; Method 7 is TAU, which introduces an attention mechanism into video prediction, dynamically focusing on the parts of historical frames relevant to the current prediction; Method 8 is PredRNN++, which adds a spatial memory state to the traditional LSTM, allowing each LSTM unit to simultaneously transmit temporal and spatial memories. This allows information to flow in both temporal and spatial dimensions; Method 9 is MIM, which introduces multi-level memory units and allows interaction between these units, forming a "memory within a memory," enabling the model to learn transitions from non-stationary to stationary states and adapt to dynamic changes.

[0082] Table 1 presents the quantitative comparison results of mean absolute error (MAE), peak signal-to-noise ratio (PSNR), and perceptual similarity ratio (LPIPS) for six different network structures on the test set.

[0083] Table 1 Comparison results of different methods on the test set

[0084]

[0085] Furthermore, Table 2 presents the Key Success Index (CSI) scores for each model under different radar echo thresholds. Here, 50 dB, 40 dBZ, 30 dBZ, and 20 dBZ correspond to the intensities of heavy rain, torrential rain, moderate rain, and light rain, respectively. The results shown in the table demonstrate that TAUNet consistently outperforms other models across all rainfall intensities, with its performance advantage becoming more pronounced at higher intensities (50 dBZ and 40 dBZ). For example, TAUNet improves upon the best baseline performance achieved by the TAU method below at 50 dBZ and 40 dBZ, reducing it by 6.54% and 11.14%, respectively. These results further demonstrate the robustness and superiority of the method presented in this invention. A comparison of the results in Tables 1 and 2 shows that the method presented in this invention achieves higher accuracy and more precise predictions compared to all other methods.

[0086] Table 2 compares the key success indices of different methods on the Xichang radar dataset.

[0087]

[0088]

[0089] To verify the generalization ability of the proposed method in other spatiotemporal series prediction tasks, we used 2-meter temperature (t2m) data from the WeatherBench dataset. This task involves predicting the temperature evolution over a 12-hour period based on observations from the previous 12 hours. Table 3 presents a quantitative comparison of the mean squared error (MSE), peak signal-to-noise ratio (PSNR), and key index (SUCR) for four different network architectures on the test set. As shown in Table 3, the proposed method achieves the best performance across all evaluation metrics, demonstrating its strong generalization ability.

[0090] Table 3 Comparison results of different methods on the t2m test set from 2010 to 2018

[0091]

[0092] To more intuitively illustrate the effectiveness of the method of the present invention, a comparison is made between the prediction results of existing methods and the method of the present invention on the prediction of spatiotemporal sequence datasets. Figure 4 This is a typical large-scale precipitation process selected from radar echo data from Xichang. Figure 4 The rectangular region represents a representative center of heavy precipitation, highlighting the performance differences between different models. It can be observed that the baseline models being compared generally exhibit significant dissipation in these heavy precipitation areas. Although PredRNNv2 captures some of the heavy precipitation areas, its predicted locations deviate considerably from actual observations. In contrast, the method of this invention demonstrates good fitting ability in terms of the location and shape of heavy precipitation centers.

[0093] This invention essentially predicts from spatiotemporal datasets. As long as the data is spatiotemporal and its shape meets the requirements, prediction can be performed. For example, this invention can also predict temperature. The network model remains unchanged; only the input and output data are modified. For instance, precipitation is predicted from 10 frames of data, using radar data as the input, while temperature is predicted from 12 frames of data, using the Weather Bench t2m 2010-2018 dataset. Figure 5 This example shows the predicted temporal evolution from 12:00 to 24:00 UTC on January 1, 2017, selected from the 2-meter temperature (t2m) data in the Weather Bench dataset. The input data spans from 00:00 to 11:00 UTC. Notably, the comparison model exhibits asymptotic dissipation artifacts in these regions, deviating significantly from the ground-based ground observations.

[0094] The effectiveness of each proposed module was verified through ablation experiments. The evaluation dataset for the ablation experiments used the test set of the Xichang radar echo dataset.

[0095] Table 4. Ablation experiment results. √ indicates that the module was used, × indicates that the module was not used.

[0096]

[0097] The quantitative results are shown in Table 4. The proposed method uses U-Net as the baseline model. When modules are added to the baseline one by one, the proposed method shows significant improvement. MM-TA-dynamical represents the multimodal dynamic attention mechanism, which integrates average pooling layers and fully connected layers to learn inter-frame dependencies, generate frame importance weights, enhance key evolutionary states, and reduce noise in the temporal series. MM-TA-statical represents the multimodal static attention mechanism, which integrates depthwise convolution and depthwise expanded convolution to model intra-frame spatial information, operating independently for each frame without temporal background. Furthermore, the impact of adding a single module on the baseline was tested. Experimental results show that adding any module improves detection performance. This demonstrates the effectiveness of the proposed module.

[0098] It should be noted that the specific embodiments described above are exemplary, and those skilled in the art can devise various solutions inspired by the disclosure of this invention. These solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents.

Claims

1. A temporal attention-based precipitation prediction method based on hybrid sensing, characterized in that, The precipitation prediction method constructs a temporal attention network, which introduces a multi-scale attention mechanism to enhance the local feature representation capability of radar echo images. Simultaneously, a hybrid sensing module, including a multimodal spatiotemporal attention module and a hybrid multilayer sensing module, replaces the skip-connection layer. Specifically, the multimodal spatiotemporal attention module extracts key spatial features of single-frame radar echoes through static spatial attention and combines it with dynamic temporal attention to model inter-frame evolution dependencies, thereby capturing the spatiotemporal evolution patterns of the precipitation system. The hybrid multilayer sensing module fuses shallow and deep features to improve the network's ability to locate precipitation. Furthermore, pixel rearrangement operations eliminate aliasing artifacts introduced during the upsampling process, improving the quality and spatiotemporal consistency of the reconstructed precipitation field image. The method specifically includes: Step 1: Prepare the dataset required for precipitation nowcasting, and extract the maximum reflectance data at different elevation angles to synthesize a combined reflectance image; Step 2: Preprocess the dataset obtained in Step 1, locate the local precipitation area using a sliding window method, divide it into training set, validation set and test set according to the agreed ratio, and perform data augmentation and normalization on the three datasets. Step 3: Construct and initialize a temporal attention-based precipitation prediction network based on radar echo data. The prediction network consists of a multi-scale attention module, a hybrid sensing module, and a pixel rearrangement module. The multi-scale attention module backbone performs multi-level feature extraction and compression on the input image, gradually transforming the original image information into a high-dimensional, abstract feature representation. The multi-modal spatiotemporal attention module backbone extracts key spatial features of single-frame radar echoes through static spatial attention and combines dynamic temporal attention to model inter-frame evolution dependencies to capture the spatiotemporal evolution patterns of the precipitation system. The hybrid multi-layer sensing module integrates channel interaction and spatial modeling functions, making full use of the multi-scale features generated by the multi-modal spatiotemporal attention module to achieve efficient feature mixing and cross-channel mapping. The pixel rearrangement module is used to decode the features extracted by the previous two networks and output the predicted radar echo image. The multi-scale attention module consists of three feature extraction layers, each including a downsampling layer and a double convolutional block; The multimodal spatiotemporal attention module first sets up a batch normalization layer, a 1×1 convolution, and a Gaussian error linear unit activation function, and then connects two branches, specifically a static attention branch and a dynamic attention branch. The static attention branch includes depthwise separable convolution, depthwise expanded convolution, and a 1×1 convolution; the dynamic attention branch includes an adaptive average pooling layer and a fully connected layer. Finally, the outputs of the two branches are passed through a 1×1 convolution. The hybrid multilayer sensing module consists of: 1×1 convolution, 3×3 depthwise separable convolution, and 1×1 convolution. The pixel rearrangement module specifically consists of: transposed convolution, pixel rearrangement, group normalization, and Swish activation function; Step 4: Input the training set processed in Step 2 into the constructed prediction network to train the network, specifically including: Step 41: Input the original radar echo images in the training set into the multi-scale attention module to extract feature information. Then, obtain feature maps of different scales by passing downsampling and convolution blocks in sequence, and output the first feature map T1, the second feature map T2, the third feature map T3, the fourth feature map T4 and the fifth feature map T5. Step 42: Input feature maps T1, T2, and T3 of different scales into the hybrid sensing module for feature fusion. Specifically, after processing, the first feature map T1, the second feature map T2, and the third feature map T3 are respectively obtained as the first hybrid feature map C1, the second hybrid feature map C2, and the third hybrid feature map C3. Step 43: Concatenate the third hybrid feature map C3 and the fifth feature map T5 through channels, and then pass them through the pixel rearrangement module to obtain the third decoded feature map W. out3 The second mixed feature map C2 and the third decoded feature map W are combined. out3 The second decoded feature map W is obtained by concatenating channels and then rearranging pixels. out2 The first mixed feature map C1 and the second decoded feature map W are combined. out2 The first decoded feature map W is obtained by concatenating channels and then rearranging pixels. out1 ; Step 44: Combine the first feature map T1 with the first decoded feature map W out1 The output feature map F is obtained by fusing the features using the feature summation method. F is then adjusted by 3×3 convolution to adjust the number of channels in the output feature map, and the predicted radar echo image P is output. Step 5: Calculate the loss value between the predicted radar echo image P and the actual radar echo image; Step 6: The precipitation prediction network is trained with the initially set model parameters. The network automatically changes the network model parameters based on the validation loss value obtained in Step 5. After a preset number of training rounds, the validation set loss value gradually stabilizes. The current training model parameters are saved, and training is stopped. Step 7: Select the network that performs best on the validation set, then feed the test set into the network, calculate the mean absolute error, peak signal-to-noise ratio and perceptual similarity ratio, predict the average value of the three indicators of ten radar echo images and the three indicator values ​​of ten single-frame radar echo images, and save the final prediction result.

2. The time-series attention precipitation prediction method as described in claim 1, characterized in that, The data processing flow of the multi-scale attention module includes: Step 411: The first feature map T1 is obtained by passing the input radar echo image through a double convolutional block, wherein the double convolutional block is a combination of two consecutive convolutional blocks, and each convolutional block includes: 3×3 convolution, batch normalization and linear rectified function; Step 412: Next, the first feature map T1 is processed through the first feature extraction layer to obtain T2. ​​The feature extraction layer includes a 2×2 max pooling layer and a double convolutional block. Similarly, T2 is processed through the second feature extraction layer to obtain T3, and T3 is processed through the third feature extraction layer to obtain T4. Step 413: Finally, T4 is processed by a convolutional block to obtain feature map T5.

3. The time-series attention precipitation prediction method as described in claim 1, characterized in that, The data processing of the multimodal spatiotemporal attention module includes: Step 421: The hybrid sensing module includes a multimodal spatiotemporal attention module and a hybrid multilayer sensing module, which are connected in the form of residuals; Step 422: Input the first feature map T1, the second feature map T2 and the third feature map T3 into the multimodal spatiotemporal attention module respectively. First, adjust the channel dimension through batch normalization layer, 1×1 convolution and Gaussian error linear unit activation function to obtain feature map W; Step 423: The multimodal spatiotemporal attention module includes two branches, specifically a static attention branch and a dynamic attention branch. The feature map W is simultaneously input into both branches. In the static attention branch, W sequentially passes through depthwise separable convolution, depthwise expanded convolution, and 1×1 convolution. First, it expands the receptive field to model intra-frame spatial information. Then, it adjusts the input feature map W through a 1×1 convolution kernel, finally obtaining the static attention weight vector W. sta ; In the dynamic attention branch, W first passes through an adaptive average pooling layer to obtain global information about the channels, and then learns the correlation between channels through a fully connected layer with a bottleneck structure, outputting a dynamic attention weight vector W. dyn Finally, W and W sta and W dyn Multiply the results, adjust the number of channels by a 1×1 convolution, multiply the result by the channel scaling parameter, and then perform residual concatenation with the input feature map to obtain feature map M; Step 424: Feature map M is then fed into the hybrid multilayer perception module. After convolution, the resulting perceptual feature map is multiplied by the channel scaling parameter and then residually connected with feature map M to obtain hybrid feature map C. The input features are feature map M1, second feature map M2 and third feature map M3. After processing by the multilayer perception module, the first hybrid feature map C1, the second hybrid feature map C2 and the third hybrid feature map C3 are obtained respectively.

4. The time-series attention precipitation prediction method as described in claim 1, characterized in that, The data processing of the pixel rearrangement module specifically includes: Step 431: First, concatenate the fifth feature map T5 and the third mixed feature map C3 along the channel dimension, and then pass them into the pixel rearrangement module to obtain W. out3 ; Step 432: Place W out3 The second mixed feature map C2 is first concatenated along the channel dimension, and then fed into the pixel rearrangement module to obtain W. out2 Similarly, W out2 After concatenating the first mixed feature map C1 along the channel dimension, it is fed into the pixel rearrangement module to obtain W. out1 .

Images