Precipitation Prediction Method and System Based on Multi-Source Data Fusion and Dynamic and Static Spatiotemporal Network

Abstract

The present invention provides a precipitation prediction method and system based on multi-source data fusion and dynamic and static spatio-temporal network, which relates to the field of data processing. The method includes: obtaining multiple sets of sample data, where the sample data includes historical multi-source meteorological maps and historical rainfall data of a sample area; preprocessing the multiple sets of sample data to generate multiple sets of training samples; establishing a precipitation prediction model, where the precipitation prediction model at least includes a feature fusion module and a dynamic and static spatio-temporal network module; training the precipitation prediction model based on a global inter-frame perception loss function and multiple sets of training samples; obtaining multi-source meteorological maps of a target area; preprocessing the multi-source meteorological maps of the target area; and generating future precipitation prediction data of the target area through the precipitation prediction model according to the preprocessed multi-source meteorological maps of the target area, which has the advantage of improving the accuracy of short-term precipitation prediction.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and particularly to a precipitation prediction method and system based on multi-source data fusion and dynamic and static spatio-temporal networks. Background Art

[0002] Precipitation is a common meteorological event that has a significant impact on human production and life and is also the cause of various natural disasters. Although weather forecasting methods and technologies have made effective progress in recent years, due to the high requirements for refinement, short-term precipitation forecasting remains the focus of meteorological research. Short-term precipitation forecasting focuses on high spatio-temporal resolution precipitation prediction in a local area in the next few hours, and the forecast within 1 hour is of great practical significance for actual operations. However, short-term precipitation prediction has a short cycle and rapid changes, and how to accurately forecast remains an urgent problem to be solved in the meteorological field.

[0003] Traditional precipitation prediction methods mainly include numerical weather prediction and precipitation prediction methods based on radar echo extrapolation. Among them, numerical weather prediction has the problem of low computational efficiency, and the precipitation prediction based on radar echo extrapolation is difficult to meet the actual needs in terms of accuracy. In recent years, deep learning technology has developed rapidly and shown more excellent application results than traditional methods in many fields, which has opened up a new development path for short-term precipitation forecasting. Most studies tend to regard short-term precipitation forecasting as the prediction of radar sequences. The short-term precipitation prediction algorithm based on deep learning focuses on studying the spatio-temporal evolution characteristics of precipitation events from a large amount of historical data. Given the highly non-linear and complex characteristics of the precipitation system, radar sequence prediction is more difficult than conventional image sequence prediction. In this context, many valuable and meaningful methods have emerged, but there are still certain defects, such as:

[0004] 1. Traditional numerical weather prediction needs to solve complex atmospheric physical equations, consuming a large amount of computing resources and computing time, and it is difficult to accurately predict short-term precipitation. The method based on radar echo extrapolation is inaccurate in precipitation prediction because it is difficult to consider complex atmospheric movements;

[0005] 2. The development of meteorological information technology has spawned rich and multi-source meteorological data. Diversified cross-source meteorological data can make up for the deficiencies of single-source data in terms of accuracy and reliability. However, current short-term precipitation prediction methods based on deep learning usually only use single-source meteorological data and are difficult to effectively integrate and utilize multi-source data for short-term precipitation prediction;

[0006] 3. Short-term and imminent precipitation is highly complex and non-linear. Common spatio-temporal prediction networks vary greatly due to different application scenarios, making it difficult to directly transfer them to the precipitation prediction field to capture complex spatio-temporal mapping relationships. Moreover, such methods ignore the internal connection between the whole and frames, and the prediction performance drops sharply as the prediction duration increases. Therefore, it is difficult to effectively improve the accuracy of precipitation forecasting.

[0007] Therefore, there is a need to provide a precipitation prediction method and system based on multi-source data fusion and dynamic and static spatio-temporal networks to improve the accuracy of short-term and imminent precipitation prediction. Summary of the Invention

[0008] The present invention provides a precipitation prediction method based on multi-source data fusion and dynamic and static spatio-temporal networks, including: obtaining multiple sets of sample data, where the sample data includes historical multi-source meteorological maps and historical rainfall data of a sample area; preprocessing the multiple sets of sample data to generate multiple sets of training samples; establishing a precipitation prediction model, where the precipitation prediction model at least includes a feature fusion module and a dynamic and static spatio-temporal network module; training the precipitation prediction model based on a global inter-frame perception loss function and multiple sets of training samples; obtaining multi-source meteorological maps of a target area; preprocessing the multi-source meteorological maps of the target area; and generating future precipitation prediction data of the target area through the precipitation prediction model according to the preprocessed multi-source meteorological maps of the target area.

[0009] Further, preprocessing the multiple sets of sample data includes: for each set of sample data, extracting the feature maps of each meteorological map included in the historical multi-source meteorological map through a convolutional layer, scaling the feature maps of each meteorological map included in the extracted historical multi-source meteorological map to a preset size using bicubic linear interpolation, and concatenating the scaled feature maps of each meteorological map in the channel dimension to generate a multi-dimensional feature map corresponding to the sample data.

[0010] Further, the feature fusion module includes a channel fusion mechanism and a spatial fusion mechanism. The channel fusion mechanism is used to generate channel weights corresponding to the multi-dimensional feature map, and perform matrix multiplication on the multi-dimensional feature map and the channel weights to obtain a channel fusion output. The spatial fusion mechanism is used to generate spatial weights based on the channel fusion output, and perform matrix multiplication on the channel fusion output and the spatial weights to obtain a feature fusion output.

[0011] Further, the channel fusion mechanism at least includes a channel global average pooling layer, a channel global max pooling layer, and a multi-layer perceptron. Among them, the input of the multi-layer perceptron includes the outputs of the channel global average pooling layer and the channel global max pooling layer, and the channel weights corresponding to the multi-dimensional feature map are generated based on the output of the multi-layer perceptron using element-wise addition and an activation function; the spatial fusion mechanism at least includes a spatial global average pooling layer and a spatial global max pooling layer, and spatial weights are generated based on the outputs of the spatial global average pooling layer and the spatial global max pooling layer through a convolutional layer and an activation function.

[0012] Further, the static-dynamic spatio-temporal network module includes an encoder, a static-dynamic spatio-temporal network, and a decoder. Among them, the encoder is used to perform spatial encoding on the output of the feature fusion module, the static-dynamic spatio-temporal network is used to extract spatial feature information and temporal feature information from the output of the encoder, and the decoder is used to decode the output of the static-dynamic spatio-temporal network.

[0013] Further, the static-dynamic spatio-temporal network includes a plurality of static networks and a plurality of dynamic networks. Among them, the static networks are used to extract spatial feature information from the output of the encoder, and the dynamic networks are used to extract temporal feature information from the output of the encoder; the static networks and the dynamic networks are arranged alternately.

[0014] Further, the static networks extracting spatial feature information from the output of the encoder includes: fusing the input features in the batch dimension and the temporal dimension to generate fused data; sending the fused data into a plurality of static network branches, where the plurality of static network branches use grouped convolutions with different convolutional kernel sizes to extract feature information from the mixed-transformed data at multiple scales and perform non-linear mapping; using element-wise addition to fuse the outputs of each static network branch to generate spatial feature information.

[0015] Further, the dynamic networks extracting temporal feature information from the output of the encoder includes: fusing the input features in the temporal dimension and the channel dimension to generate fused data; sending the fused data into a first dynamic network branch and a second dynamic network branch. Among them, the first dynamic network branch consists of a depthwise separable convolutional layer, a depthwise dilated convolutional layer, and a convolutional layer with a convolutional kernel size of 1, and the second dynamic network branch consists of an average pooling layer and a fully connected layer; performing element-wise multiplication on the outputs of the first dynamic network branch and the second dynamic network branch to generate intermediate features; performing matrix multiplication on the intermediate features and the input features to generate temporal feature information.

[0016] Further, the global inter-frame perception loss function is:

[0017] ;

[0018] ;

[0019] ;

[0020] wherein, is the global inter-frame perception loss, is the mean square error, is the inter-frame loss, and are weights, and are both greater than 0, , is the true precipitation corresponding to the i-th training sample, is the predicted precipitation corresponding to the i-th training sample output by the precipitation prediction model, is the number of training samples included in a training batch, , , is the predicted precipitation corresponding to the (i + 1)-th training sample output by the precipitation prediction model, is the true precipitation corresponding to the (i + 1)-th training sample, is the activation function, is the KL divergence, is the number of adjacent frame pairs.

[0021] The present invention provides a precipitation prediction system based on multi-source data fusion and dynamic and static spatio-temporal network, which applies the above precipitation prediction method based on multi-source data fusion and dynamic and static spatio-temporal network, including: a data acquisition module for acquiring multiple sets of sample data, wherein the sample data includes historical multi-source meteorological maps and historical rainfall data of a sample area; a data preprocessing module for preprocessing the multiple sets of sample data to generate multiple sets of training samples; a model establishment module for establishing a precipitation prediction model, wherein the precipitation prediction model at least includes a feature fusion module and a dynamic and static spatio-temporal network module; a model training module for training the precipitation prediction model based on the global inter-frame perception loss function and multiple sets of training samples; the data acquisition module is further configured to acquire multi-source meteorological maps of a target area; the data preprocessing module is further configured to preprocess the multi-source meteorological maps of the target area; a precipitation prediction module for generating future precipitation prediction data of the target area according to the preprocessed multi-source meteorological maps of the target area through the precipitation prediction model.

[0022] Compared with the prior art, the precipitation prediction method and system based on multi-source data fusion and dynamic and static spatio-temporal network provided by the present invention at least have the following beneficial effects:

[0023] 1. It can reasonably mine and integrate the potential precipitation information in multi-source meteorological data, solve the problem of insufficient accuracy and reliability of single-source data, and the static spatial information captured by the static spatio-temporal network and the dynamic temporal information captured by the dynamic spatio-temporal network complement each other, solve the problem of insufficient spatio-temporal mapping ability between multi-source meteorological elements and precipitation, and improve the accuracy of precipitation prediction.

[0024] 2. The global inter-frame perception loss function can adaptively and dynamically perceive the internal relationship between the global and inter-frame, which is conducive to alleviating the adverse situation that the prediction performance drops sharply with the increase of the prediction duration, solve the problem that it is difficult for previous precipitation prediction methods to establish the internal relationship between frames, and optimize the internal ability of the model to capture precipitation continuity. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] This specification will be further described by way of exemplary embodiments, which will be described in detail through the accompanying drawings. These embodiments are not restrictive. In these embodiments, the same numbers represent the same structures, where:

[0026] Figure 1 is a schematic flowchart of a precipitation prediction method based on multi-source data fusion and static and dynamic spatio-temporal networks according to some embodiments of this specification;

[0027] Figure 2 is a schematic structural diagram of a precipitation prediction model according to some embodiments of this specification;

[0028] Figure 3 is a schematic structural diagram of a feature fusion module according to some embodiments of this specification;

[0029] Figure 4 is a schematic structural diagram of a channel fusion mechanism according to some embodiments of this specification;

[0030] Figure 5 is a schematic structural diagram of a spatial fusion mechanism according to some embodiments of this specification;

[0031] Figure 6 is a schematic structural diagram of a static and dynamic spatio-temporal network module according to some embodiments of this specification;

[0032] Figure 7 is a schematic structural diagram of a static network according to some embodiments of this specification;

[0033] Figure 8 is a schematic structural diagram of a dynamic network according to some embodiments of this specification;

[0034] Figure 9is a line graph of various meteorological indicators at different thresholds using different methods as shown in some embodiments of this specification;

[0035] Figure 10 is a line graph comparing prediction results at different times using different methods as shown in some embodiments of this specification;

[0036] Figure 11 is a visualization graph of the overall effect of short-term and nowcasting precipitation prediction using different methods as shown in some embodiments of this specification;

[0037] Figure 12 is a schematic diagram of the visualization results of ablation experiments using different meteorological data as shown in some embodiments of this specification;

[0038] Figure 13 is a schematic diagram of the visualization results of ablation experiments on the dynamic and static spatio-temporal network as shown in some embodiments of this specification;

[0039] Figure 14 is a schematic diagram of the visualization results of ablation experiments on the loss function as shown in some embodiments of this specification;

[0040] Figure 15 is a schematic diagram of the modules of a precipitation prediction system based on multi-source data fusion and dynamic and static spatio-temporal network as shown in some embodiments of this specification. Detailed implementation manners

[0041] To more clearly illustrate the technical solutions of the embodiments of this specification, the accompanying drawings required for the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some examples or embodiments of this specification. For those of ordinary skill in the art, without creative efforts, this specification can also be applied to other similar scenarios based on these drawings. Unless obvious from the language context or otherwise stated, the same reference numerals in the drawings represent the same structure or operation.

[0042] Figure 1 is a schematic flowchart of a precipitation prediction method based on multi-source data fusion and dynamic and static spatio-temporal network as shown in some embodiments of this specification. As Figure 1 shown, the precipitation prediction method based on multi-source data fusion and dynamic and static spatio-temporal network may include the following steps.

[0043] Step 110, obtain multiple groups of sample data, where the sample data includes historical multi-source meteorological maps and historical rainfall data of the sample area.

[0044] For example, based on the SEVIR (Storm EVent ImagRy) dataset, multiple sets of sample data can be obtained. The SEVIR dataset includes meteorological data captured from satellites and radars. Three different types of data are used as inputs.

[0045] By way of example only, historical multi-source meteorological maps can respectively correspond to VIL, IR069, and IR107. Among them, IR069 is a map of atmospheric water vapor distribution, IR107 is a map of the temperature of the surface and clouds, and VIL is a map of vertical liquid water content.

[0046] Table 1

[0047]

[0048] Step 120: Preprocess the multiple sets of sample data to generate multiple sets of training samples.

[0049] In some embodiments, step 120 specifically includes:

[0050] For each set of sample data, extract the feature maps of each meteorological map included in the historical multi-source meteorological map through a convolutional layer, scale the feature maps of each meteorological map included in the extracted historical multi-source meteorological map to a preset size using bicubic interpolation, and splice the scaled feature maps of each meteorological map in the channel dimension to generate a multi-dimensional feature map corresponding to the sample data.

[0051] Specifically, since the spatial resolutions and sizes of meteorological maps from different sources are different, in order to avoid deviations and errors due to different resolutions in subsequent processing, after extracting their features through a convolutional layer in data preprocessing, use bicubic interpolation to uniformly scale the meteorological maps to a size of 384×384 to be applicable to subsequent processing. Finally, all the processed different meteorological maps are spliced together in the channel dimension to form a new multi-dimensional data. This process not only ensures the spatial consistency of the multi-source meteorological maps, but also effectively integrates the multi-source meteorological maps, providing a data basis for subsequent data analysis and modeling.

[0052] Step 130: Establish a precipitation prediction model.

[0053] Figure 2 is a schematic structural diagram of the precipitation prediction model shown in some embodiments of this specification, as Figure 2 shown, the precipitation prediction model at least includes a feature fusion module and a dynamic and static spatio-temporal network module.

[0054] Figure 3 is a schematic structural diagram of the feature fusion module shown in some embodiments of this specification, as Figure 3As shown, in some embodiments, the feature fusion module includes a channel fusion mechanism and a spatial fusion mechanism. Among them, the channel fusion mechanism is used to generate channel weights corresponding to the multi-dimensional feature map, and perform matrix multiplication on the multi-dimensional feature map and the channel weights to obtain a channel fusion output. The spatial fusion mechanism (Spatial Fusion Mechanism, SFM) is used to generate spatial weights based on the MLP (Multilayer Perceptron), and perform matrix multiplication on the channel fusion output and the spatial weights to obtain a feature fusion output.

[0055] Figure 4 is a schematic structural diagram of the channel fusion mechanism shown in some embodiments of this specification. As Figure 4 shown, in some embodiments, the channel fusion mechanism at least includes a channel global average pooling layer, a channel global maximum pooling layer, and a multi-layer perceptron. Among them, the input of the multi-layer perceptron includes the outputs of the channel global average pooling layer and the channel global maximum pooling layer, and the channel weights corresponding to the multi-dimensional feature map are generated based on the output of the multi-layer perceptron using element-wise addition and an activation function. Among them, C represents the number of channels.

[0056] Specifically, first use global average pooling and global maximum pooling to obtain the channel representation of the multi-dimensional feature map:

[0057] ;

[0058] ;

[0059] Among them, X is the input multi-dimensional feature map, MaxPool(·) is the maximum pooling function, AvgPool(·) is the average pooling function, is the maximum pooling result of the c-th channel, is the average pooling result of the c-th channel.

[0060] The output of the MLP is fused using element-wise addition and the sigmoid activation function to generate channel weights:

[0061] ;

[0062] Among them, X is a multi-dimensional feature map with dimensions ( C , H , W ), where C is the number of channels, H is the height of the multi-dimensional feature map, W is the width of the multi-dimensional feature map, is a one-dimensional vector of length C, representing the weight of each channel, is is the input of the output, is is the input of the output, and σ(·) is the sigmoid activation function.

[0063] Perform matrix multiplication on the channel fusion output and the spatial weight to obtain the channel fusion output:

[0064] ;

[0065] where is the channel fusion output.

[0066] Figure 5 is a schematic structural diagram of the spatial fusion mechanism shown in some embodiments of this specification. As Figure 5 shown, in some embodiments, the spatial fusion mechanism at least includes a spatial global average pooling layer and a spatial global maximum pooling layer, and generates a spatial weight through a convolutional layer and an activation function based on the outputs of the spatial global average pooling layer and the spatial global maximum pooling layer.

[0067] Specifically, use average pooling and maximum pooling in the channel dimension and concatenate the obtained results.

[0068] ;

[0069] where is the concatenation result of the outputs of the spatial global average pooling layer and the spatial global maximum pooling layer.

[0070] Generate the spatial weight by passing the concatenation result of the outputs of the spatial global average pooling layer and the spatial global maximum pooling layer through a convolutional layer and a sigmoid activation function.

[0071] ;

[0072] where is the spatial weight.

[0073] Perform matrix multiplication on the channel fusion output and the spatial weight to obtain the feature fusion output.

[0074] ;

[0075] where is the feature fusion output, is the output of the channel fusion mechanism.

[0076] Figure 6 is a schematic structural diagram of the dynamic and static spatio-temporal network module shown in some embodiments of this specification. As Figure 6As shown, in some embodiments, the static and dynamic spatio-temporal network module includes an encoder, a static and dynamic spatio-temporal network, and a decoder. Among them, the encoder is used to perform spatial encoding on the output of the feature fusion module, the static and dynamic spatio-temporal network is used to extract spatial feature information and temporal feature information from the output of the encoder, and the decoder is used to decode the output of the static and dynamic spatio-temporal network.

[0077] Specifically, the encoder first uses a convolutional layer (Conv) to change the dimension and spatial scale of the input data, then uses a group normalization layer (GroupNorm) to normalize the data to stabilize the model training process and accelerate convergence, and finally uses an activation layer (LeakyReLU) to enhance the non-linear representation ability of the features, thereby enhancing the representation ability of the model.

[0078] As Figure 6 shown, in some embodiments, the static and dynamic spatio-temporal network includes multiple static networks and multiple dynamic networks. Among them, the static network is used to extract spatial feature information from the output of the encoder, and the dynamic network is used to extract temporal feature information from the output of the encoder; the static networks and dynamic networks are arranged alternately.

[0079] Figure 7 is a schematic structural diagram of the static network shown according to some embodiments of this specification. As Figure 7 shown, in some embodiments, the static network extracts spatial feature information from the output of the encoder, including:

[0080] Fuse the batch dimension and the time dimension of the input features to generate the fused data. Specifically, mix the batch dimension (B) and the time dimension (T) with the shape of and convert them into data with the shape of ;

[0081] Send the fused data into multiple static network branches. Among them, the multiple static network branches use grouped convolutions with different convolutional kernel sizes to extract feature information from the mixed-transformed data at multiple scales and perform non-linear mapping;

[0082] Use element-wise addition to fuse the outputs of each static network branch to generate spatial feature information.

[0083] Figure 8 is a schematic structural diagram of the dynamic network shown according to some embodiments of this specification. As Figure 8 shown, in some embodiments, the dynamic network extracts temporal feature information from the output of the encoder, including:

[0084] Fuse the time dimension and the channel dimension of the input features to generate the fused data. Specifically, convert the input features from to ;

[0085] The fused data is fed into the first dynamic network branch and the second dynamic network branch. Among them, the first dynamic network branch consists of a depthwise separable convolutional layer, a depthwise dilated convolutional layer, and a convolutional layer with a kernel size of 1. The second dynamic network branch consists of an average pooling layer and a fully connected layer;

[0086] Perform element-wise multiplication on the outputs of the first dynamic network branch and the second dynamic network branch to generate intermediate features;

[0087] Perform matrix multiplication on the intermediate features and the input features to generate temporal feature information.

[0088] The decoder decodes the low-dimensional spatial features to obtain the final output. Similar to the encoder, the decoder also consists of a convolutional layer, a group normalization layer, and an activation layer. The input is restored to the high-dimensional feature space to generate the final prediction result.

[0089] Step 140, train the precipitation prediction model based on the global inter-frame perception loss function and multiple groups of training samples.

[0090] The global inter-frame perception loss function is specially designed to alleviate the adverse situation that the prediction effect gradually decreases over time. In some embodiments, the global inter-frame perception loss function is:

[0091] ;

[0092] ;

[0093] ;

[0094] where is the global inter-frame perception loss, is the mean squared error, is the inter-frame loss, and are weights, and are both greater than 0, , is the true precipitation corresponding to the i-th training sample, is the predicted precipitation corresponding to the i-th training sample output by the precipitation prediction model, is the number of training samples included in a training batch, , , is the predicted precipitation corresponding to the (i + 1)-th training sample output by the precipitation prediction model, is the true precipitation corresponding to the (i + 1)-th training sample is the activation function, is the KL divergence, is the number of adjacent frame pairs.

[0095] Preferably, in order to automatically find the appropriate weight coefficient, the weight coefficient is parameterized into a learnable variable, that is, , where is the parameterized weight coefficient.

[0096] The parameters of the network model can be optimized by gradient backpropagation. Stop training until the loss converges.

[0097] Step 150, obtain multi-source meteorological maps of the target area.

[0098] Step 160, preprocess the multi-source meteorological maps of the target area.

[0099] Step 170, generate future precipitation prediction data for the target area through a precipitation prediction model based on the preprocessed multi-source meteorological maps of the target area.

[0100] The beneficial effects will be described below in combination with experiments.

[0101] Compare from objective evaluation indicators and subjective visual effects. The objective evaluation indicators adopted by the present invention are Mean Square Error (MSE), Mean Absolute Error (MAE), Peak Signal to Noise Ratio (PSNR), False Alarm Rate (FAR), Critical Success Index (CSI), and Heidke Skill Score (HSS).

[0102] The mean square error is an index that measures the average absolute difference between the predicted value and the true value. A smaller MSE usually indicates a better model prediction result. Its formula is:

[0103] ;

[0104] The mean absolute error measures the average absolute difference between the model prediction result and the true value, and can intuitively reflect the prediction ability of the model. The smaller the error, the stronger the fitting ability of the model. Its formula is:

[0105] ;

[0106] Among them, represents the prediction result of the model, represents the true value, represents the number of samples.

[0107] The peak signal-to-noise ratio is the ratio of the energy of the peak signal to the average energy of the noise, and is usually expressed in decibels (dB) after multiplying by 10. Its calculation formula is: It is expressed in decibels (dB). Its calculation formula is:

[0108] ;

[0109] where, represents the square of the maximum pixel value. The PSNR ranges from 0 to infinity, and the larger the value, the smaller the difference between the two images.

[0110] The false alarm rate represents the proportion of the area where precipitation is predicted but there is actually no precipitation in the area where precipitation is forecast. The FAR value ranges from 0 to 1. When it is 1, it means the worst prediction effect, and when it is 0, it means the best prediction effect. Its formula is:

[0111] ;

[0112] where, FP represents the number of positive samples mispredicted by the model, and TP represents the number of positive samples correctly predicted by the model.

[0113] The critical achievement index represents the proportion of the area where precipitation is correctly predicted in the total area where precipitation actually occurs or is predicted. The closer its value is to 1, the better the effect. Its formula is:

[0114] ;

[0115] where, FN represents the number of negative samples mispredicted.

[0116] The Heidke skill score represents the forecast accuracy after removing the influence of random events. The closer its value is to 1, the better. Its calculation formula is:

[0117] ;

[0118] where, TN represents the number of negative samples correctly predicted.

[0119] The present invention is compared with other methods in terms of the prediction effect of short-term and imminent precipitation. Multisource meteorological data for the previous hour is input into the model to predict the precipitation situation every 10 minutes for the next hour, with a total of 6 outputs totaling 60 minutes. The comparison methods used are ConvLSTM (Convolutional Long Short-Term Memory), PhyDNet (Physical Dynamics Network), PredRNN (Predictive Recurrent Neural Network), Earthformer, Rainformer, SimVP (Simpler yet Better Video Prediction), and PastNet (Physical-Assisted Spatio-Temporal Network). They are mainly models based on CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), and Transformer, and they have also achieved excellent prediction effects in precipitation prediction. All methods are trained and tested on the same dataset, and all experimental results are objective and fair. The experimental results are shown in Table 2.

[0120] Table 2

[0121]

[0122] According to the experimental results in Table 2, the present invention not only performs best in the traditional image metrics MSE, MAE, and PSNR, but also shows the best quantization effect in the meteorological metrics FAR, CSI, and HSS. These results prove the excellent performance of our method in short-term and imminent precipitation prediction.

[0123] To prove the effectiveness of the present method under different precipitation thresholds, the line graphs of the FAR, CSI, and HSS metrics of all methods under different thresholds are plotted in Figure 9 . Figure 9 This illustrates the prediction capabilities of different methods under different precipitation thresholds. As can be seen from Figure 9 , the present method is superior to the comparison methods at all thresholds, and the present method shows excellent results in the prediction of high thresholds, i.e., heavy precipitation. In addition, to analyze the performance of the present method at different prediction times, the line graph comparing the prediction results of all methods at different times is plotted, as shown in Figure 10 . Figure 10The results shown in [figure] indicate that the proposed method still exhibits the best prediction performance among the prediction results at different times. Combining the information presented in the two line charts, it is evident that the proposed method demonstrates a more excellent performance in terms of spatio-temporal continuity compared to the comparison methods. This can be attributed to the dynamic and static spatio-temporal network proposed in the present invention, which effectively captures and establishes the complex spatio-temporal relationships between multi-source meteorological data and precipitation information. In addition, the global inter-frame perception loss guides the model to establish the internal relationships between frames, optimizing the model's ability to capture precipitation continuity, thereby alleviating the adverse situation where the prediction performance of the model significantly deteriorates over time.

[0124] To visually demonstrate the effectiveness of the proposed method, a set of data was randomly selected as the prediction results of different input models for visualization. As Figure 11 shown, the method of the present invention exhibits visualization results closer to the true values at each time frame, which is consistent with the results presented in Table 2. Figure 11 The visualization results in [figure] further confirm the Figure 9 findings in [figure], indicating that the proposed method also achieves the best results in terms of spatio-temporal consistency compared to the comparison methods.

[0125] To further verify the effectiveness of multi-source meteorological data fusion, the dynamic and static spatio-temporal network, and the global inter-frame perception loss, a comprehensive ablation experiment was conducted to evaluate the effectiveness of these methods separately. Precipitation prediction neural networks based on CNNs or Transformers often neglect the interactions between multi-source meteorological data. These models fail to capture and integrate the potential relationships between meteorological data from different sources and precipitation information. To address this limitation, the present invention constructs the potential relationships between meteorological data from different sources to improve the prediction accuracy of the model. The effectiveness of this multi-feature fusion method was verified through ablation experiments, and the results are shown in Table 3.

[0126] As shown in Table 3, the proposed method achieved the best evaluation metrics in the ablation experiment. Figure 12 The visualization results in [figure] further support this finding. Compared to the comparison methods, the prediction results of the proposed method are closer to the true values. These findings indicate that integrating the mutual relationships between multi-source meteorological data can significantly improve the performance of the model, demonstrating the positive significance of the proposed method for short-term precipitation prediction.

[0127] Table 3

[0128]

[0129] Existing precipitation prediction methods are difficult to model the potential relationships between multi-source data and precipitation information, which limits their capabilities in precipitation prediction. To address this issue, the proposed dynamic and static spatio-temporal network module in the present invention effectively establishes a mapping relationship between multi-source meteorological data and precipitation information, capturing the complex spatio-temporal evolution process in meteorological data. As shown in Table 4, ablation experiments were respectively conducted on the static network module and the dynamic network module of this module, where "Ours-SN" and "Ours-DN" represent the results of ablation on the static network and the dynamic network respectively, and Ours represents Figure 2 the precipitation prediction model shown, including the feature fusion module and the dynamic and static spatio-temporal network module, demonstrating that the dynamic and static spatio-temporal network has a favorable impact on the overall performance of the model. In addition, Figure 13 the visualization results in

[0130] Table 4

[0131]

[0132] further strengthen this finding, indicating that the dynamic and static spatio-temporal network makes an important contribution to the short-term precipitation prediction ability of the model. Figure 14 The visualization of the prediction results using the methods with two different losses (as shown in

[0133] Table 5

[0134]

[0135] Figure 15 shows) indicates that the method using the global inter-frame perception loss still shows prediction results closer to the true values as time increases. This shows that the global inter-frame perception loss function can effectively establish the internal connection between frames, alleviating the adverse situation where the model performance drops sharply as the prediction time increases. Figure 15 is a schematic diagram of the modules of the precipitation prediction system based on multi-source data fusion and dynamic and static spatio-temporal network according to some embodiments of this specification. As shown in

[0136] The data acquisition module is used to acquire multiple sets of sample data, where the sample data includes historical multi-source meteorological maps and historical rainfall data of the sample area;

[0137] The data preprocessing module is used to preprocess multiple sets of sample data to generate multiple sets of training samples;

[0138] A model establishment module for establishing a precipitation prediction model, where the precipitation prediction model at least includes a feature fusion module and a dynamic and static spatio-temporal network module;

[0139] A model training module for training the precipitation prediction model based on a global inter-frame perception loss function and multiple sets of training samples;

[0140] The data acquisition module is also used to acquire multi-source meteorological maps of the target area;

[0141] The data preprocessing module is also used to preprocess the multi-source meteorological maps of the target area;

[0142] A precipitation prediction module for generating future precipitation prediction data of the target area through the precipitation prediction model according to the preprocessed multi-source meteorological maps of the target area.

[0143] The precipitation prediction system based on multi-source data fusion and dynamic and static spatio-temporal network can be used to execute the precipitation prediction method based on multi-source data fusion and dynamic and static spatio-temporal network, which will not be elaborated here.

[0144] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other deformations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification can be regarded as consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.

Claims

1. A precipitation prediction method based on multi-source data fusion and dynamic and static spatiotemporal network, characterized in that: include: Acquire multiple sets of sample data, wherein the sample data include historical multi-source meteorological maps and historical rainfall data of the sample area; Preprocessing the multiple groups of sample data to generate multiple groups of training samples; Establishing a precipitation prediction model, wherein the precipitation prediction model at least includes a feature fusion module and a dynamic and static spatiotemporal network module; Training the precipitation prediction model based on a global inter-frame perceptual loss function and multiple groups of training samples; Obtain multi-source weather maps of the target area; Preprocessing of multi-source meteorological images of the target area; Generate future precipitation prediction data of the target area according to the preprocessed multi-source meteorological map of the target area through the precipitation prediction model; Preprocessing the multiple groups of sample data includes: For each set of sample data, the feature map of each meteorological map included in the historical multi-source meteorological map is extracted through the convolution layer, and the feature map of each meteorological map included in the extracted historical multi-source meteorological map is scaled to a preset size using bicubic linear interpolation. In the channel dimension, the scaled feature map of each meteorological map is spliced ​​to generate a multidimensional feature map corresponding to the sample data; The feature fusion module includes a channel fusion mechanism and a spatial fusion mechanism, wherein the channel fusion mechanism is used to generate channel weights corresponding to the multidimensional feature map, and perform matrix multiplication operations on the multidimensional feature map and the channel weights to obtain channel fusion outputs, and the spatial fusion mechanism is used to generate spatial weights based on the channel fusion outputs, and perform matrix multiplication operations on the channel fusion outputs and the spatial weights to obtain feature fusion outputs; The dynamic-static spatiotemporal network module includes an encoder, a dynamic-static spatiotemporal network and a decoder, wherein the encoder is used to perform spatial encoding on the output of the feature fusion module, the dynamic-static spatiotemporal network is used to extract spatial feature information and temporal feature information of the output of the encoder, and the decoder is used to decode the output of the dynamic-static spatiotemporal network; The dynamic-static spatiotemporal network includes a plurality of static networks and a plurality of dynamic networks, wherein the static network is used to extract spatial feature information of the output of the encoder, and the dynamic network is used to extract temporal feature information of the output of the encoder; The static network and the dynamic network are arranged alternately.

2. The precipitation prediction method based on multi-source data fusion and dynamic and static spatiotemporal network according to claim 1 is characterized in that: The channel fusion mechanism at least includes a channel global average pooling layer, a channel global maximum pooling layer and a multi-layer perceptron, wherein the input of the multi-layer perceptron includes the output of the channel global average pooling layer and the channel global maximum pooling layer, and the channel weight corresponding to the multi-dimensional feature map is generated based on the output of the multi-layer perceptron using element-by-element addition and activation function; The spatial fusion mechanism includes at least a spatial global average pooling layer and a spatial global maximum pooling layer, and generates spatial weights based on the outputs of the spatial global average pooling layer and the spatial global maximum pooling layer through a convolutional layer and an activation function.

3. The precipitation prediction method based on multi-source data fusion and dynamic and static spatiotemporal network according to claim 1 is characterized in that: The static network extracts spatial feature information of the output of the encoder, including: Fuse the input features in batch dimension and time dimension to generate fused data; The fused data is fed into a plurality of static network branches, wherein the plurality of static network branches use grouped convolutions with different convolution kernel sizes to extract feature information of the mixed transformed data from a plurality of scales and perform nonlinear mapping; The output of each static network branch is fused using element-wise addition to generate spatial feature information.

4. The precipitation prediction method based on multi-source data fusion and dynamic and static spatiotemporal network according to claim 1 is characterized in that: The dynamic network extracts temporal feature information of the output of the encoder, including: Fuse the input features in the time dimension and channel dimension to generate fused data; The fused data is sent to the first dynamic network branch and the second dynamic network branch, wherein the first dynamic network branch is composed of a depthwise separable convolution layer, a depthwise atrous convolution layer and a convolution layer with a convolution kernel size of 1, and the second dynamic network branch is composed of an average pooling layer and a fully connected layer; Performing element-by-element multiplication on the outputs of the first dynamic network branch and the second dynamic network branch to generate intermediate features; A matrix multiplication operation is performed on the intermediate feature and the input feature to generate time feature information.

5. The precipitation prediction method based on multi-source data fusion and dynamic and static spatiotemporal network according to any one of claims 1 to 4, characterized in that: The global inter-frame perception loss function is: ， ， ； in, is the global inter-frame perceptual loss, is the mean square error, is the inter-frame loss, and is the weight, and are greater than 0, , is the actual precipitation corresponding to the i-th training sample, is the predicted precipitation corresponding to the i-th training sample output by the precipitation prediction model, is the number of training samples included in a training batch, , , is the predicted precipitation corresponding to the i+1th training sample output by the precipitation prediction model, is the actual precipitation corresponding to the i+1th training sample, is the activation function, is the KL divergence, is the number of adjacent frame pairs.

6. The precipitation prediction system based on multi-source data fusion and dynamic and static spatiotemporal network is characterized by: The precipitation prediction method based on multi-source data fusion and dynamic and static spatiotemporal network described in any one of claims 1 to 5 is applied, comprising: A data acquisition module, used to acquire multiple sets of sample data, wherein the sample data includes historical multi-source meteorological maps and historical rainfall data of the sample area; A data preprocessing module, used to preprocess the multiple groups of sample data to generate multiple groups of training samples; A model building module, used to build a precipitation prediction model, wherein the precipitation prediction model at least includes a feature fusion module and a dynamic and static spatiotemporal network module; A model training module, used for training the precipitation prediction model based on a global inter-frame perception loss function and multiple groups of training samples; The data acquisition module is also used to obtain a multi-source meteorological map of the target area; The data preprocessing module is also used to preprocess the multi-source meteorological map of the target area; The precipitation prediction module is used to generate future precipitation prediction data for the target area according to the preprocessed multi-source meteorological map of the target area through the precipitation prediction model.

Images