Short-time rainfall prediction method based on MSPTS-Net network model

Abstract

The invention provides a short-time rainfall prediction method based on an MSPTS-Net network model. The short-time rainfall prediction method comprises the following steps: acquiring a radar echo data set formed by radar echo images; a network model is constructed based on MSPTS-Net, a spatial-temporal feature extraction and fusion mechanism is embedded in a framework of the MSPTS-Net, and an initial short-term rainfall prediction model is generated; training the initial short-time rainfall prediction model by using the radar echo data set, and carrying out reverse adjustment and optimization on network parameters through a combined loss function to obtain a final short-time rainfall prediction model; and inputting a radar echo image to be measured into the final short-time rainfall prediction model to obtain a short-time rainfall prediction result. Through the MSPTS-Net model, spatio-temporal change information in radar echo data can be more comprehensively captured, multi-scale feature extraction and a self-attention fusion mechanism are utilized, data changes of different time scales and spatial scales are processed, a combined loss function is combined, the model training process is more efficient, convergence to an optimal solution can be faster, and the method is suitable for being applied to the field of radar imaging. And the precision of short-time rainfall prediction is improved.

Description

Technical Field

[0001] The present application relates to the technical field of weather forecasting, and in particular to a short-term precipitation prediction method based on the MSPTS-Net network model. Background Art

[0002] Short-term precipitation forecast refers to the prediction of precipitation in a local area within 0-2 hours in the future. It plays a pivotal role in the field of weather forecasting and widely affects people's daily lives. For example, it can predict extreme weather (rainstorms and heavy rainstorms) in advance, thereby reducing the huge impact of natural disasters such as mudslides and floods, and help people plan their travel and prepare for countermeasures. Especially for the aviation industry, which relies on weather, it is very important to predict the weather in advance. However, due to the complexity of the weather system, how to obtain real-time, accurate and high-resolution forecast results is still a topic that needs to be improved.

[0003] Mainstream traditional forecasting methods include numerical weather forecasting and optical flow method, but due to the high computational cost of numerical weather forecasting and the large amount of resources it occupies, as well as the poor forecasting effect of optical flow method in a short period of time, ignoring historical records and only speculating based on current observations, it is limited in practical application. With the rapid rise of deep learning, and benefiting from the development of large amounts of data collection and image processing units, the prediction accuracy and deduction speed have been greatly improved, and its application in meteorological forecasting has become more and more common. The current mainstream methods mainly include convolutional neural network CNN, recurrent neural network RNN, Transformer, etc. The core idea is to learn spatiotemporal information from historical observation data to predict future echo motion.

[0004] Convolutional neural networks (CNNs) are good at learning spatial features. In the field of rainfall prediction, Lei et al. (Han, Lei, et al. "Convective precipitation nowcasting using U-Net Model." IEEE Transactions on Geoscience and Remote Sensing 60 (2021): 1-8.) applied the pure convolutional network UNet used in the field of image segmentation to rainfall prediction. This architecture obtains the predicted radar reflectivity image by performing three operations: upsampling, downsampling, and skip connections on the input radar image. However, since convolutional neural networks focus more on capturing local features, they are inefficient for modeling global relationships, but long-range spatial dependencies are also very important for rainfall information. In recurrent neural networks, since the input and output of precipitation nowcasting are both spatiotemporal sequences, recurrent neural networks are better at learning sequence information than convolutional neural networks. In recent years, Wang et al. proposed predRNN (Predrnn: Recurrent neural networks for predictive learning using spatiotemporal lstms." Advances in neural information processing systems 30 (2017).) and its variants predRNN++, predRNN_v2 and other network structures, which have achieved excellent results in rainfall prediction. However, due to their structural characteristics, there is still a risk of gradient explosion. In addition, long-range dependencies will gradually be lost during information transmission. Another Transformer architecture originally used in natural language processing has been found to have excellent performance in global feature extraction due to its self-attention mechanism. However, it has a large amount of computation and is difficult to deploy on devices with limited resources. In the case that each network has its own advantages and disadvantages, integrating the advantages of different networks has become a focus of attention.

[0005] Most of the networks mentioned above are based on autoregressive models, that is, the output will be fed back to the input and combined with the input to predict the next frame. Although this method effectively learns the prior information, if there are outliers or incomplete data in the spatiotemporal series, the prediction results of the model will be biased. As the number of predicted frames increases, the small errors in the prediction results of earlier frames will accumulate and deteriorate rapidly. In particular, meteorological data is complex and non-stationary, and autoregressive models are difficult to fit complex nonlinear relationships. The initial small errors will become larger and larger over time, resulting in inaccurate rainfall prediction results.

[0006] The Chinese invention patent application with publication number CN117574074A and publication date of February 20, 2024, "A Transformer-based Short-term Forecast Method for Severe Convective Precipitation", studies the spatial distribution characteristics of dual-polarization radar data and designs a deep learning model. Its structure consists of a feature encoder, a Transformer model and a feature decoder. Through this structure, spatial multi-scale features are obtained, and in order to alleviate the fuzzy prediction problem of the forecast, the radar observation data is divided into high and low frequencies, and the energy of high and low frequencies in the frequency domain is used to calculate the weight of the loss function to design a new loss function, and finally the precipitation is calculated through the ZR relationship. The method is based on a similar UNet architecture as a whole, and multi-scale spatial features are extracted by up and down sampling in space, but no cascading is performed between the levels, which may lose some information. In addition, it ignores that precipitation prediction is a direction of spatiotemporal series prediction, and time information also occupies an important position in the prediction process.

[0007] The Chinese invention patent application "A method, device and equipment for short-term precipitation forecasting based on sequence length" with publication number CN117647855A and publication date March 5, 2024, is based on precipitation radar echo sequences. The method obtains the spatial distribution and motion characteristics of precipitation by stacking prediction combination layers, wherein the combination layer contains short sequence feature extraction modules and long sequence feature extraction modules. The short sequence feature extraction module is based on Transformer and non-local attention mechanism. The long sequence feature extraction module is based on the prediction results of the short sequence and then uses the long short-term memory network for modeling. The prediction results are then used as the input of the next layer through the encoding and decoding module, and the generated prediction results are supervised by the loss functions MSE and MAE. Although this method takes into account the correlation between long-term and short-term time features in rainfall prediction, it ignores the characteristics of radar echo sequences at multiple spatial scales, and because it is an autoregressive model, there is still the problem of error accumulation.

[0008] The Chinese invention patent application "A precipitation prediction method based on a deep dual-branch structure" with publication number CN117908164A and publication date April 19, 2024, also uses radar echo sequences as data sets to train the network. The deep dual-branch structure is mainly divided into two parts. One part uses the PhyCell physical branch unit to learn the potential physical characteristics in the radar echo, so as to predict future radar echoes according to physical laws. The other part uses SimVP as a residual branch unit to obtain residual features, that is, dynamic detail features of radar echoes, and sums the physical features with the residual features to obtain the prediction results. This method improves the accuracy of the prediction by introducing physical features, but the structure has limitations in obtaining the global features of the radar echo and as the extrapolation process time increases, it may lose early time features, resulting in poor prediction results.

[0009] The Chinese invention patent application "Short-term precipitation prediction method based on multi-scale attention and convolution fusion" with publication number CN118298222A and publication date July 5, 2024, uses meteorological data and geographic information as input, and performs adaptive position encoding on the input. After the information from two different sources is fused, it is sent to the network for learning. The network structure mainly uses the Transformer-based encoding and decoding structure to learn global features, and introduces a convolutional neural network in the decoding layer to learn local features. During the training process, the loss function combines MSE, MAE and regularization terms for constraints. The characteristic of this method is that it integrates information from two different sources. However, although the network structure used embeds the convolutional neural network into the Transformer structure to extract global to local spatial information, the temporal information is ignored, and the feature extraction of only the spatial resolution at a single scale is difficult to effectively capture the change information of spatiotemporal data at different spatial levels. Summary of the invention

[0010] In view of the defects in the prior art, the purpose of this application is to provide a network architecture based on MSPTS-Net, which not only improves the accuracy of short-term precipitation prediction, but also effectively alleviates the image blur problem when the extrapolation time is long.

[0011] In one aspect of the present application, a short-term precipitation prediction method based on the MSPTS-Net network model is provided, and the prediction method comprises the following steps:

[0012] Acquire a radar echo data set consisting of radar echo images;

[0013] A network model is constructed based on MSPTS-Net, and a spatiotemporal feature extraction and fusion mechanism is embedded in the framework of the MSPTS-Net to generate an initial short-term rainfall prediction model;

[0014] Using the radar echo data set, the initial short-term rainfall prediction model is trained, and the network parameters are optimized by reverse tuning through a combined loss function to obtain a final short-term rainfall prediction model;

[0015] The radar echo image to be tested is input into the final short-term rainfall prediction model to obtain a short-term rainfall prediction result.

[0016] Furthermore, after acquiring the radar echo data set, the radar echo image is preprocessed, and the preprocessing includes:

[0017] The radar echo image in the radar echo data set is split into a plurality of image blocks, and the position of each image block is encoded.

[0018] Furthermore, the network model is constructed based on MSPTS-Net, and a spatiotemporal feature extraction and fusion mechanism is embedded in the framework of the MSPTS-Net to generate an initial short-term rainfall prediction model, including:

[0019] Construct a spatial coding block to obtain spatial scale features of radar echoes of different scales, namely, spatial multi-scale features;

[0020] In parallel, a time multi-scale coding block is introduced to obtain time scale features of radar echoes of different scales, namely, time multi-scale features, and the long-term and short-term time evolution laws are learned through the time multi-scale features;

[0021] Sending the spatial multi-scale features and the temporal multi-scale features into a self-attention mechanism for fusion;

[0022] The initial short-term rainfall prediction model is generated.

[0023] Furthermore, the construction of the spatial coding block to obtain spatial scale features of radar echoes of different scales includes:

[0024] Construct a dual-branch spatial coding block and use convolution kernels of different sizes to process radar echo images;

[0025] The self-attention mechanism and CBAM channel space attention mechanism of the Transformer framework are used to extract the global features and local features of the radar echo image respectively. The residual connection (Add) and group normalization (Norm) are embedded after the self-attention mechanism to ensure the stable transmission and training of the features in the network. The data is batch-normalized before the CBAM channel space attention mechanism, and the activation function (ReLU) is used to filter irrelevant information.

[0026] The global features and local features are adaptively fused through a gating mechanism, and a fused feature map is output to obtain spatial scale features of radar echoes of different scales.

[0027] Furthermore, the dual-branch coding block is two parallel branches, including:

[0028] The first branch: uses convolution operations to map the data into a high-dimensional latent space and downsamples the radar echo image to half the input size;

[0029] The second branch uses a convolution kernel of a different size from that of the first branch to perform a greater degree of downsampling on the radar echo image.

[0030] Furthermore, a time multi-scale coding block is introduced in parallel to obtain the time multi-scale features of the radar echo, and the long-term and short-term time evolution laws are learned through the time multi-scale features, including:

[0031] By reshaping spatiotemporal series data to implicitly encode temporal information;

[0032] A temporal feature extraction unit is introduced, which is mapped into a high-dimensional space through convolution, and convolution kernels of different sizes are used for multi-scale convolution to extract temporal features of different scales.

[0033] By stacking the time feature extraction units and using the shallow network and deep network of the network model, the short-term and long-term time scale features in the radar echo image are obtained.

[0034] Furthermore, the step of feeding the spatial multi-scale features and the temporal multi-scale features into a self-attention mechanism for fusion includes:

[0035] The temporal multi-scale feature is used as a query vector, and the spatial multi-scale feature is used as a key vector and a value vector, and fused through a self-attention mechanism;

[0036] The fused features are then sent to the feedforward network for mapping transformation to further improve the fused features.

[0037] Furthermore, the initial short-term rainfall prediction model is trained by using a part of the radar echo data set as a training set, and the network parameters are reversely tuned and optimized through a combined loss function to obtain a final short-term rainfall prediction model, including:

[0038] The first n frames of images in the radar echo data set are used as input, and the last m frames of images are used to train the initial short-term rainfall prediction model;

[0039] Determine the initial number of training rounds, enhance the generalization ability of the model by rotating the training data, set the initial learning rate, and adjust it through the cosine annealing adjustment strategy;

[0040] The training data is sent to the MSPTS-Net network for forward propagation to obtain the predicted frame;

[0041] By combining the loss function MSE and Charbonnier Loss, the difference between the predicted image and the real image is calculated to obtain the loss value;

[0042] Reverse tuning is performed according to the loss value to minimize the difference between the real frame and the predicted frame to train the model to the optimal performance and obtain the final short-term rainfall prediction model.

[0043] Furthermore, the combined loss function MSE and Charbonnier Loss is specifically as follows:

[0044] Loss = aMSE(Y pred -Y truth )+βCharb(Y pred -Y truth )

[0045] Among them, set α to 2, β to 1, and Y pred refers to the predicted radar echo image, Y truth Refers to the actual radar echo image;

[0046] Among them, the calculation formula of Charbonnier Loss is:

[0047]

[0048] In the Charbonnier Loss formula, Y pred is the predicted radar echo image, Y truth is the real radar echo image, and ε is a constant.

[0049] The second aspect of the present application provides a short-term precipitation prediction system based on the MSPTS-Net network model, comprising:

[0050] An acquisition module is used to obtain radar echo images and form a radar echo data set;

[0051] The prediction model building module is used to build a network model based on MSPTS-Net, embed the spatiotemporal feature extraction and fusion mechanism in the framework of the MSPTS-Net, and generate the initial short-term rainfall prediction model

[0052] A training module, used to train the initial short-term rainfall prediction model using the radar echo data set, and to reversely tune and optimize network parameters through a combined loss function to obtain a final short-term rainfall prediction model;

[0053] The output module is used to input the radar echo image to be measured into the final short-term rainfall prediction model to obtain the short-term rainfall prediction result.

[0054] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0055] This application uses the self-attention mechanism and CBAM attention in the spatial feature encoding module to learn global local features, uses convolution kernels of different sizes on the time scale to learn the long-term and short-term time evolution laws, and finally further integrates through self-attention to obtain the characteristic expressions of radar echo time and space scales, effectively improving the prediction ability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments with reference to the following drawings:

[0057] Figure 1 This is a flow chart of a short-term precipitation prediction method based on the MSPTS-Net network model according to an embodiment of the present application.

[0058] Figure 2 This is a flow chart of a short-term precipitation prediction method based on the MSPTS-Net network model, which is an embodiment of the present application.

[0059] Figure 3 A diagram of a short-term precipitation network model provided in an embodiment of the present application.

[0060] Figure 4 This is a comparison chart of the visualization results of an embodiment of the present application on various network models. DETAILED DESCRIPTION

[0061] The present application is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements can also be made without departing from the concept of the present application. These all belong to the protection scope of the present application.

[0062] Reference Figure 1 As shown, a short-term precipitation prediction method based on the MSPTS-Net network model is provided in accordance with an embodiment of the present application. The prediction method comprises the following steps:

[0063] S1. Obtain a radar echo data set consisting of radar echo images.

[0064] S2. Build a network model based on MSPTS-Net, embed the spatiotemporal feature extraction and fusion mechanism in the framework of MSPTS-Net, and generate the initial short-term rainfall prediction model.

[0065] S3. The initial short-term rainfall prediction model is trained using the radar echo dataset, and the network parameters are optimized through reverse tuning using the combined loss function to obtain the final short-term rainfall prediction model.

[0066] S4. Input the radar echo image to be measured into the final short-term rainfall prediction model to obtain the short-term rainfall prediction result.

[0067] By embedding the spatiotemporal feature extraction and fusion mechanism, the MSPTS-Net model can more comprehensively capture the spatiotemporal change information in the radar echo data and improve the accuracy of short-term rainfall prediction; by utilizing multi-scale feature extraction and self-attention fusion mechanism, the model can better handle data changes at different time scales and spatial scales, and enhance the generalization ability for different weather conditions; by using the combined loss function, the training process of the MSPTS-Net model is more efficient and can converge to the optimal solution faster, thereby improving the accuracy of short-term rainfall prediction through the final short-term rainfall prediction model.

[0068] In a specific embodiment, when acquiring a radar data set, the radar echo image is continuously sampled for a fixed number of frames in the radar echo image, and the continuously sampled radar echo images are formed into a radar echo data set to form a training sample.

[0069] In the above specific embodiment, when forming training samples, image augmentation is performed on the radar echo images in the radar echo data set. The image augmentation method adopts a method of flipping the radar echo images to increase the diversity of training samples and improve the accuracy of training.

[0070] For example, the time interval between each frame of radar echo data set of HKO-7 provided by a certain observatory is 6 minutes. In this embodiment, by continuously sampling 20 frames, that is, radar echo sequence data of the past hour and radar echo sequence data of the future hour, the radar echo sequence data of the past hour (10 frames) is used to train the rainfall prediction model and obtain ten frames of predicted radar echo images. The last ten frames of real radar echo images are used to compare with the predicted radar echo images to obtain the prediction performance.

[0071] The radar echo dataset refers to a collection of multiple radar echo images, which may be acquired at different times, locations or under different conditions.

[0072] Radar echo sequence data: refers to a group of radar echo image data arranged in time series.

[0073] For example, S1 uses the HKO-7 public dataset released by an observatory, which includes 20,949 continuous radar echo sequences with an interval of 6 minutes, of which 18,177 are used for training, 808 are used for verification, and 1,964 are used for testing. Each training uses sequential sampling to obtain a continuous 20-frame radar echo sequence from the training dataset. The first ten frames are used to train the rainfall prediction model and obtain ten frames of predicted radar echo images. The last ten frames of real radar echoes are used to compare with the predicted ones to obtain the predicted performance.

[0074] In some possible embodiments, after acquiring the radar echo data set, the method further includes: preprocessing the radar echo image, wherein the preprocessing includes: splitting the radar echo image in the radar echo data set into a plurality of image blocks, and encoding the positions of the image blocks.

[0075] By preprocessing the radar echo image, the radar echo image is split into image blocks and the position information is encoded to reduce the computational complexity and retain the spatial position information of the image block. The split coded block image is sent to the spatial and temporal multi-scale extraction module in parallel to obtain the temporal and spatial characteristics of the radar echo sequence.

[0076] Reference Figure 3 As shown in the figure, the processing block diagram of the radar echo image is performed. First, the input radar echo data set is sorted by time to form radar echo sequence data, so that the network model can better distinguish the position of each part of the radar echo image in the radar echo data set to better understand the context. Then, the radar echo image in the radar echo sequence data is split and positionally encoded to form a coded block image. The coded block image is sent to the spatial and temporal multi-scale extraction module in parallel to obtain the temporal and spatial features in the radar echo sequence data. In order to fully understand the correlation between the temporal and spatial features, they are fused through the self-attention mechanism, and finally the prediction result is obtained through the feedforward network.

[0077] Exemplarily, before being sent to the network model, the radar echo images in the input radar echo data set must first be split to reduce the complexity of the model. For example, the original radar echo image of size 480*480 is divided into 16 non-overlapping image blocks of size 120*120, and then the image blocks are position-encoded and sent to the spatial encoder and time encoder in parallel. The spatial correlation and temporal correlation between echo distributions are learned respectively through the two encoding methods.

[0078] In some possible embodiments, a network model is constructed in S2 based on MSPTS-Net, a spatiotemporal feature extraction and fusion mechanism is embedded in the framework of MSPTS-Net, and an initial short-term rainfall prediction model is generated, including the following steps:

[0079] S21. Construct a spatial coding block to obtain spatial scale features of radar echoes of different scales, namely, spatial multi-scale features.

[0080] S22. In parallel, a time multi-scale coding block is introduced to obtain the time scale features of radar echoes of different scales, namely, time multi-scale features, and the long-term and short-term time evolution laws are learned through time multi-scale features.

[0081] S23. Send the spatial multi-scale features and temporal multi-scale features into the self-attention mechanism for fusion.

[0082] S24. Generate an initial short-term rainfall prediction model.

[0083] By constructing the MSPTS-Net network model and embedding the spatiotemporal feature extraction and fusion mechanism, a short-term rainfall prediction model can be generated. The spatial characteristics of radar echoes at different scales are obtained by constructing spatial coding blocks. At the same time, multi-scale temporal coding blocks are introduced in parallel to learn the long-term and short-term evolution laws. Then, the spatiotemporal features are effectively fused using the self-attention mechanism, which improves the accuracy and reliability of short-term rainfall prediction.

[0084] In the above embodiment, the spatial coding block is mainly composed of two branches, each of which downsamples the radar echo image data at different scales and obtains multi-scale spatial features through the global to local feature extraction unit.

[0085] Specifically, the Transformer network model is first used as the framework, and the spatial coding block is divided into two parts. The first part is the spatial multi-scale module, which includes global feature extraction and local feature extraction and adaptive fusion, so as to obtain information on spatial features of each scale and increase the multi-scale prediction ability of the network model. The second part is the temporal multi-scale module, which obtains short-term time dependencies through convolution kernels of different scales. The stacked temporal multi-scale modules transfer temporal features to further learn long-term time evolution. Different scales help the network learn spatial features at different resolutions and enhance the feature expression of the network model in the time dimension of radar echo. Finally, the spatial multi-scale features and the temporal multi-scale features are sent to the self-attention mechanism for fusion to generate the initial short-term rainfall prediction model. By adaptively focusing on key features, the effective integration of spatiotemporal information is achieved, which improves the accuracy and reliability of the initial short-term rainfall prediction model in short-term rainfall prediction.

[0086] In a specific embodiment, in order to construct a spatial coding block and obtain spatial scale features of radar echoes of different scales, in step S21, it specifically includes: constructing a spatial coding block containing two branches, and processing the radar echo image using convolution kernels of different sizes.

[0087] Specifically, the dual-branch spatial encoding block consists of two parallel branches: the first branch: uses convolution operations to map data to a high-dimensional latent space and downsamples the image to half the input size; the second branch: uses convolution kernels of different sizes to downsample the image to a greater extent than the first branch, usually to a quarter of the input size. Using different scales helps the network learn spatial features at different resolutions.

[0088] For example: For spatial multi-scale features, this application proposes a spatial to local multi-scale extraction unit (SpatialGlobal to Local Multi-scale Extractor), which is specifically divided into two branches. The input image is converted through convolution kernels of different sizes, and feature maps of different resolutions are processed at the same time. The model can better understand features of different scales for learning.

[0089] Furthermore, the global and local features of the radar echo image are extracted through the self-attention mechanism and CBAM channel space attention mechanism of the Transformer framework, and residual connection (Add) and group normalization (Norm) are embedded after the self-attention mechanism to ensure stable transmission and training of features in the network. Before the CBAM channel space attention mechanism, the data is batch normalized and the activation function (ReLU) is used to filter irrelevant information.

[0090] The spatial feature maps at different resolutions are then passed through the spatial feature extraction unit, where global and local correlations are established for the radar echo images. The process is expressed as:

[0091]

[0092] Among them, x i Refers to the input radar echo map, which has the shape of And set i=1, SGLME is a spatial global to local multi-scale extraction unit.

[0093] SGLME includes global feature extraction and local feature extraction. The global feature extraction is composed of a multi-head self-attention mechanism (MHSA) and a feed-forward network (FFN). The local features are first convolved once, and then the spatial feature map is sent to the channel space attention mechanism (CBAM) to focus on important features. The calculation process of global and local feature extraction can be summarized as follows:

[0094] g1=GN(MHSA(x i )+x i )

[0095] g=GN(FFN(g1)+g1)

[0096]

[0097] Among them, MHSA refers to multi-head self-attention, GN refers to group normalization, FFN refers to feedforward network, F refers to input features, M s refers to channel attention, M c Refers to spatial attention. This method can simultaneously focus on useful information between channels and in space, partially suppress unimportant information, and improve the representation ability of the model.

[0098] Exemplarily, the global feature extraction part includes a multi-head self-attention mechanism and a feedforward network. The multi-head self-attention mechanism is used to obtain long-term dependencies in the radar echo sequence, and the feedforward network is used to introduce nonlinearity to enable the network to learn more complex functional relationships, and understand the overall distribution of echoes over a large range through global features.

[0099] The spatial feature extraction of each branch is divided into two parts. The self-attention module of Transformer is embedded to extract global features, and residual connection (Add) and group normalization (Norm) are embedded to ensure stable transmission and training of features in the network. In parallel, batch normalization is used to process data and activation function (ReLU) is used to filter irrelevant information, and then channel space attention mechanism (CBAM) is used to extract local features. This method can focus on the significant features in the channel dimension and suppress secondary information.

[0100] The channel space attention mechanism, namely CBAM, learns the correlation between feature channels and local spatial features, and identifies local feature changes such as convective clouds in echoes. By combining these two different types of attention, the network can learn the characteristics of radar echo data more fully and comprehensively.

[0101] Furthermore, the global features and local features are adaptively fused through a gating mechanism, and the fused feature map is output to obtain the spatial scale features of radar echoes of different scales.

[0102] Finally, the global features and local features are adaptively fused through a gating mechanism. This fusion method can adaptively focus on important information at different scales and reduce the numerical difference between global features and local features.

[0103] In the above preferred embodiment of the present application, a gating mechanism is used to discriminately select two different types of features, so as to achieve a more optimized feature combination, that is, the global and local features are spliced ​​and fused through convolution, and then divided into two identical parts as gating, and feature selection and extraction are performed again on the local features and the global features respectively, and finally the features are added and further fused, which enhances the richness and accuracy of the spatial information, and is conducive to improving the accuracy of the prediction.

[0104] In some specific embodiments, in order to introduce the time multi-scale coding block in parallel, obtain the time multi-scale features of the radar echo, and learn the long-term and short-term time evolution laws through the time multi-scale features, in step S22, specifically including:

[0105] Reshape spatiotemporal series data to implicitly encode temporal information.

[0106] Among them, the spatiotemporal sequence data is obtained by reshaping the spatiotemporal sequence data with a shape of [B, T, C, H, W] into [B, T×C, H, W], that is, merging the channel dimension of the data with the time dimension to implicitly encode the time information.

[0107] In this embodiment, in order to understand the short-term motion trend of radar echoes and the potential long-term regularity of weather systems, and thus improve the accuracy of predictions, a time multi-scale module is proposed. Before being sent to the time feature extraction unit, the shape of the spatiotemporal sequence is reshaped from [B, T, C, H, W] to [B, T*C, H, W]. By merging the channel dimension with the time dimension, that is, implicitly including time information in the channel dimension, the features of the time step can be processed simultaneously in the subsequent multi-scale convolution.

[0108] Furthermore, a temporal feature extraction unit is introduced, which is mapped into a high-dimensional space through convolution, and multi-scale convolution is performed using convolution kernels of different sizes to extract temporal features of different scales.

[0109] Specifically, first, by reshaping the spatiotemporal sequence data to implicitly encode the time information, the channel dimension of the original data is merged with the time dimension, so that the time information is implicitly expressed in the channel dimension, so that the features of the time steps can be processed simultaneously in the subsequent multi-scale convolution and the information in the time dimension can be extracted.

[0110] Next, the time feature extraction unit is introduced. This unit uses 1*1 convolution for high-dimensional mapping and uses convolution kernels of different sizes for multi-scale convolution to extract time features of different scales. This is done on the basis of reshaping the spatiotemporal series data to further extract the time information in the data.

[0111] Exemplarily, the time feature extraction unit maps the data into a high-dimensional space through a 1*1 convolution, and uses four convolution kernels of different sizes, for example: (1, 5, 7, 11); to perform group convolution to obtain short-term time features at different scales, and finally concatenates the features. The formula can be expressed as:

[0112]

[0113] Where k refers to the convolution kernel size, refers to the high-dimensional features of the input, Refers to the time-encoded features. It uses a method similar to the encoding and decoding architecture in UNet, but unlike UNet, it does not have upsampling and downsampling operations.

[0114] Furthermore, by stacking the time feature extraction units, the short-term and long-term time scale features in the radar echo image are obtained through the shallow network and deep network of the network model.

[0115] Time features are extracted by stacking time feature extraction units. The shallow layer obtains features within a certain time step, and the deep time block obtains the short-term time features passed from the shallow layer and performs longer-term feature extraction again, thereby obtaining the evolution law of long-term and short-term time features in this process.

[0116] The feature extraction unit stacks eight units in the vertical direction and performs residual connection between two units in the horizontal direction. The temporal feature extraction module is parallel to the spatial feature extraction module.

[0117] Specifically, the temporal feature extraction module is similar to a U-shaped stacked feature extraction unit, which first performs high-dimensional mapping on the input, and then extracts and fuses the temporal features through four branches with different-sized convolution kernels, so as to simulate short-term time evolution. The temporal multi-scale modules are then stacked, and the shallow temporal feature information is passed to the deep network to further learn the long-term feature evolution.

[0118] For example, in order to model time correlation and understand the short-term movement trends and potential long-term laws in the weather system, the time encoding module stacks multiple time feature extraction units and performs residual connections. The input data is first mapped to a high-dimensional space through convolution, and then four convolution kernels of different sizes (1, 5, 7, 11) are used to obtain features at different spatial scales. At the same time, short-term time features are implicitly learned. Time features are extracted by stacking time feature extraction units. The shallow network obtains features within a certain time step. The deep network obtains the short-term time features passed from the shallow network and performs longer-term feature extraction again, thereby obtaining the evolution laws of long-term and short-term time features in this process.

[0119] In some specific embodiments, in order to send the spatial multi-scale features and the temporal multi-scale features into the self-attention mechanism for fusion, in step S23, specifically including:

[0120] The temporal multi-scale features are used as query vectors, and the spatial multi-scale features are used as key vectors and value vectors, which are fused through the self-attention mechanism. The fused features are then sent to the feedforward network for mapping transformation to further improve the fused features.

[0121] Specifically, the obtained spatiotemporal multi-scale features are finally fed into the Transformer decoding block, where the temporal multi-scale features are used as query vectors and the spatial multi-scale features are used as key vectors and value vectors. The process can be expressed as:

[0122] Among them, Q t The query vector is the time scale feature, K s is the key vector for the spatial scale feature, V s is the value vector of the spatial scale feature, d k Q t ,K s The dimension of the vector. In this way, the temporal multi-scale features and the spatial multi-scale features interact effectively, and the self-attention mechanism is used to adaptively adjust the weights between different features to achieve more accurate predictions. The feedforward network is then used for mapping transformation to further improve the fusion features, and finally the final short-term rainfall prediction model is generated.

[0123] In this fusion process, Q t K s The interaction between spatiotemporal features is quantified, and the interactive information in the spatiotemporal sequence is captured. s Dot product is performed, and more complex spatiotemporal features are further integrated through secondary weighting. The spatiotemporal interaction relationship in radar data can be captured by self-attention fusion. The spatiotemporal fusion features obtained from the attention mechanism are then sent to the feedforward network for mapping transformation to further improve the fusion features. The feedforward network includes normalization (group normalization), activation function (SiLU) and 3D convolution blocks. Finally, the final short-term rainfall prediction model is generated through the above settings.

[0124] In some specific embodiments, in order to use a part of the radar echo data set as a training set, the initial short-term rainfall prediction model is trained, and the network parameters are reversely tuned and optimized through the combined loss function to obtain the final short-term rainfall prediction model. In step S3, it specifically includes:

[0125] The first n frames of images in the radar echo dataset are used as input, and the last m frames are used to train the initial short-term rainfall prediction model. The initial number of training rounds is determined, the generalization ability of the model is enhanced by rotating the training data, the initial learning rate is set, and it is adjusted through the cosine annealing adjustment strategy. The training data is sent to the MSPTS-Net network for forward propagation to obtain the predicted frame. By combining the loss function MSE and Charbonnier Loss, the difference between the predicted image and the real image is calculated to obtain the loss value. According to the loss value, reverse tuning is performed to minimize the difference between the real frame and the predicted frame to train the model to the optimal performance and obtain the final short-term rainfall prediction model.

[0126] Specifically, the first ten frames of the acquired radar echo data set are used as input, and the images in the last ten frames are used to train the network model; the initial number of training rounds is determined, and during the training process, the generalization ability of the model is enhanced by rotating the training data, setting the initial learning rate, and adjusting it through the cosine annealing adjustment strategy; the training data is sent to the network for forward propagation to obtain the predicted frame; by combining the loss function MSE and Charbonnier Loss, the difference between the predicted image and the real image is calculated to obtain the loss value; reverse tuning is performed according to the loss value to minimize the difference between the real frame and the predicted frame to train the model to the optimal performance.

[0127] This embodiment adjusts the number of training rounds of the network so that the model can keep up with the distribution and characteristics of sufficient learning data. At the same time, the generalization ability of the model is enhanced by rotating the training data during the model training process. The initial learning rate is set to 0.0001, and the cosine annealing adjustment strategy is used to verify that the loss can continue to converge. The predicted frame is obtained through forward propagation, and the difference between the predicted frame and the real frame is calculated through the loss function for reverse tuning. The difference between the real frame and the predicted frame is minimized to make the model tend to the optimal performance, and the model is continuously optimized in the back propagation process by combining the loss function MSE and Charbonnier Loss.

[0128] In the above embodiment, the loss function is combined as follows:

[0129] Loss = aMSE(Y pred -Y truth )+βCharb(Y pred -Y truth )

[0130] Among them, set α to 2, β to 1, and Y pred refers to the predicted radar echo image, Y truth Refers to the actual radar echo image.

[0131] Among them, the calculation formula of Charbonnier Loss is:

[0132]

[0133] In the Charbonnier Loss formula, Y pred is the predicted radar echo image, Y truth is the real radar echo image, ε is a constant. Due to the existence of its constant ε, the loss function can avoid the gradient vanishing when close to the zero point and the existence of the square root to avoid the problem of excessive gradient caused by excessive difference.

[0134] In this application, the Charbonnier Loss function is similar to the L1 loss function, namely the MAE mean absolute error, and its calculation formula is L1 = |Y pred -Y truth |, both loss functions measure the loss based on the absolute error between the predicted value and the true value, but Charbonnier Loss provides better numerical stability and smoother gradient performance when the error is small compared to L1. The loss function proposed in this application combines MSE and Charbonnier Loss to retain the MSE loss function's ability to fit most of the data while enhancing the robustness of the model. At the same time, this embodiment sets the initial learning rate to 0.0001, uses the Adam optimizer to adjust the learning rate, and introduces the cosine annealing schedule learning rate decay strategy, the formula of which is:

[0135]

[0136] where η t is the learning rate at a certain moment, η max , η min is the maximum and minimum value of the learning rate, T cur refers to the number of training rounds, and T refers to the total number of training rounds.

[0137] Reference Figure 2As shown, as a specific embodiment, a radar echo data set formed by radar echo images of the past 1h is first obtained, and then a model is built based on MSPTS-Net. A spatiotemporal multiscale feature extraction module is introduced simultaneously using the Transformer framework. The spatiotemporal multiscale feature extraction module includes a spatial multiscale module and a temporal multiscale module. The spatial multiscale module is divided into two branches for processing inputs of radar echo images with different resolutions, and multiscale feature extraction is performed on images with fixed resolution, wherein the radar echo image is spatially encoded by the spatial multiscale module, and the radar echo image is temporally encoded by the temporal multiscale module, and then spatiotemporally decoded by the spatiotemporal fusion module (feature module) to realize the construction of a short-term rainfall prediction model. Then, the model is trained and combined with the loss function optimization model to realize the optimization of the short-term rainfall prediction model, and the final short-term rainfall prediction model is generated to improve the accuracy of the prediction result.

[0138] The following experiments are conducted using radar echo images from the public dataset HKO-7 (see Shi, Xingjian, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-Kin Wong, and Wang-chun Woo. "Convolutional LSTM network: A machine learning approach for precipitation nowcasting." Advances in neural information processing systems 28 (2015): 802-810.) to evaluate the effectiveness of the short-term rainfall prediction method based on MSTSNet proposed in this application.

[0139] The environment of this experiment is Ubuntu 20.04 operating system, with 25GB memory, the deep learning framework is Pytorch platform, and the server configuration uses V100-32GB (32GB) for training. A total of 32,500 rounds of training are performed, and the round with the smallest verification loss is selected as the training test model.

[0140] This application is compared with SOTA, using the structural similarity index (SSIM), mean square error (Mean Square Error), critical success index (CSI), and probability of detection (POD) as evaluation indicators for comparison, where the critical success index (CSI) refers to the correct ratio of precipitation prediction, and the probability of detection (POD) refers to the proportion of predicted actual precipitation areas to all actual precipitation areas. As shown in the following table, the model of this application has achieved the best performance in all four evaluation indicators.

[0141] Table 1 CSI comparison results of this application and existing algorithms

[0142]

[0143] Table 2 POD comparison results of this application and existing algorithms

[0144]

[0145]

[0146] Table 3 MSE&SSIM comparison results of this application and existing algorithms

[0147] method MSE SSIM ConvLSTM 93.12 0.7230 PredRNN 84.01 0.7611 PredRNN++ 81.63 0.7708 MotionRNN 78.62 0.7730 MIMO 65.11 0.8067 This application 52.11 0.8164

[0148] In summary, the present application is compared with the RNN-based model and the Transformer-based model in four experimental indicators. The experimental results are shown in Tables 1, 2, and 3. It can be seen that the present application has achieved the best performance at each threshold, especially under 5mm / h and 10mm / h. Compared with the suboptimal method, the present application has improved CSI and POD by 5.37% and 12.53%, and in terms of MSE indicators, the MIMO of the multi-input multi-output architecture and the present application have been significantly improved compared with the RNN model. Further, as Figure 4 As shown, the predicted effect of this application is presented subjectively.

[0149] This application improves the accuracy of rainfall forecasting for the next hour, and uses spatiotemporal multi-scale coding to fully learn the evolution information of the precipitation system in time and space, and can reliably predict complex radar echo distribution and movement trends.

[0150] The second aspect of the present application provides a short-term precipitation prediction system based on the MSPTS-Net network model, including: an acquisition module for acquiring radar echo images and forming a radar echo data set; a prediction model construction module for constructing a network model based on MSPTS-Net, embedding a spatiotemporal feature extraction and fusion mechanism in the MSPTS-Net framework, and generating an initial short-term rainfall prediction model; a module for training the initial short-term rainfall prediction model using the radar echo data set, and reversely tuning and optimizing network parameters through a combined loss function to obtain a final short-term rainfall prediction model; an output module for inputting the radar echo image to be tested into the final short-term rainfall prediction model to obtain a short-term rainfall prediction result.

[0151] Through the acquisition module, prediction model building module, training module and output module, the MSPTS-Net network model is used in combination with the spatiotemporal feature extraction and fusion mechanism to effectively improve the accuracy of short-term precipitation prediction. Through the reverse tuning of the training prediction model of the radar echo data set and the combined loss function, accurate prediction of short-term rainfall is achieved.

[0152] Specifically, the prediction model construction module includes a time multi-scale module, a spatial multi-scale module and feature fusion; the prediction model in the prediction model construction module is trained using a radar dataset, and the radar data is echo extrapolated based on MSPTS-Net, which effectively improves the accuracy of rainfall prediction; the spatial multi-scale module introduces multi-scale information extraction and fusion to enhance the richness and accuracy of spatial information; the time multi-scale module stacks convolution kernels of different sizes to obtain long-term and short-term temporal evolution trend information; the combination of MSE and Charbonnier Loss is introduced as a loss function to capture the detail information in the radar echo image and improve the quality of the predicted image, which effectively improves the structural similarity of the image and reduces the mean square error.

[0153] The third aspect of the present application provides a terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it can be used to execute the short-term precipitation prediction method based on the MSPTS-Net network model described in any of the above embodiments.

[0154] The fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to execute the short-term precipitation prediction method based on the MSPTS-Net network model described in any of the above embodiments.

[0155] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0156] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0157] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0158] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0159] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0160] The above describes the specific embodiments of the present application. It should be understood that the present application is not limited to the above specific embodiments, and those skilled in the art can make various modifications or variations within the scope of the claims, which does not affect the substantive content of the present application. The above preferred features can be used in any combination without conflicting with each other.

Claims

1. A short-term precipitation prediction method based on the MSPTS-Net network model, characterized in that: The prediction method comprises the following steps: Acquire a radar echo data set consisting of radar echo images; A network model is constructed based on MSPTS-Net, and a spatiotemporal feature extraction and fusion mechanism is embedded in the framework of the MSPTS-Net to generate an initial short-term rainfall prediction model; Using the radar echo data set, the initial short-term rainfall prediction model is trained, and the network parameters are optimized by reverse tuning through a combined loss function to obtain a final short-term rainfall prediction model; The radar echo image to be tested is input into the final short-term rainfall prediction model to obtain a short-term rainfall prediction result.

2. A short-term precipitation prediction method based on the MSPTS-Net network model according to claim 1, characterized in that: After acquiring the radar echo data set, the method further includes preprocessing the radar echo image, wherein the preprocessing includes: The radar echo image in the radar echo data set is split into a plurality of image blocks, and the position of each image block is encoded.

3. A short-term precipitation prediction method based on the MSPTS-Net network model according to claim 2, characterized in that: The network model is constructed based on MSPTS-Net, and a spatiotemporal feature extraction and fusion mechanism is embedded in the framework of the MSPTS-Net to generate an initial short-term rainfall prediction model, including: Construct a spatial coding block to obtain spatial scale features of radar echoes of different scales, namely, spatial multi-scale features; In parallel, a time multi-scale coding block is introduced to obtain time scale features of radar echoes of different scales, namely, time multi-scale features, and the long-term and short-term time evolution laws are learned through the time multi-scale features; Sending the spatial multi-scale features and the temporal multi-scale features into a self-attention mechanism for fusion; The initial short-term rainfall prediction model is generated.

4. The short-term precipitation prediction method based on the MSPTS-Net network model according to claim 3 is characterized in that: The construction of the spatial coding block to obtain spatial scale features of radar echoes of different scales includes: Construct a dual-branch spatial coding block and use convolution kernels of different sizes to process radar echo images; The self-attention mechanism and CBAM channel space attention mechanism of the Transformer framework are used to extract the global features and local features of the radar echo image respectively. The residual connection (Add) and group normalization (Norm) are embedded after the self-attention mechanism to ensure the stable transmission and training of the features in the network. The data is batch-normalized before the CBAM channel space attention mechanism, and the activation function (ReLU) is used to filter irrelevant information. The global features and local features are adaptively fused through a gating mechanism, and a fused feature map is output to obtain spatial scale features of radar echoes of different scales.

5. A short-term precipitation prediction method based on the MSPTS-Net network model according to claim 4, characterized in that: The dual-branch coding block is two parallel branches, including: The first branch: uses convolution operations to map the data into a high-dimensional latent space and downsamples the radar echo image to half the input size; The second branch uses a convolution kernel of a different size from that of the first branch to perform a greater degree of downsampling on the radar echo image.

6. The short-term precipitation prediction method based on the MSPTS-Net network model according to claim 3 is characterized in that: In parallel, a time multi-scale coding block is introduced to obtain the time multi-scale features of the radar echo, and the long-term and short-term time evolution laws are learned through the time multi-scale features, including: By reshaping spatiotemporal series data to implicitly encode temporal information; A temporal feature extraction unit is introduced, which is mapped into a high-dimensional space through convolution, and convolution kernels of different sizes are used for multi-scale convolution to extract temporal features of different scales. By stacking the time feature extraction units and using the shallow network and deep network of the network model, the short-term and long-term time scale features in the radar echo image are obtained.

7. The short-term precipitation prediction method based on the MSPTS-Net network model according to claim 3 is characterized in that: The step of sending the spatial multi-scale features and the temporal multi-scale features into a self-attention mechanism for fusion includes: The temporal multi-scale feature is used as a query vector, and the spatial multi-scale feature is used as a key vector and a value vector, and fused through a self-attention mechanism; The fused features are then sent to the feedforward network for mapping transformation to further improve the fused features.

8. The short-term precipitation prediction method based on the MSPTS-Net network model according to claim 1 is characterized in that: The method uses a part of the radar echo data set as a training set to train the initial short-term rainfall prediction model, and performs reverse tuning and optimization of network parameters through a combined loss function to obtain a final short-term rainfall prediction model, including: The first n frames of images in the radar echo data set are used as input, and the last m frames of images are used to train the initial short-term rainfall prediction model; Determine the initial number of training rounds, enhance the generalization ability of the model by rotating the training data, set the initial learning rate, and adjust it through the cosine annealing adjustment strategy; The training data is sent to the MSPTS-Net network for forward propagation to obtain the predicted frame; By combining the loss function MSE and Charbonnier Loss, the difference between the predicted image and the real image is calculated to obtain the loss value; Reverse tuning is performed according to the loss value to minimize the difference between the real frame and the predicted frame to train the model to the optimal performance and obtain the final short-term rainfall prediction model.

9. The short-term precipitation prediction method based on the MSPTS-Net network model according to claim 8 is characterized in that: The combined loss function MSE and Charbonnier Loss is specifically as follows: Loss=aMSE(Y pred -AND truth )+βCharb(Y pred -AND truth ) Among them, set α to 2, β to 1, and Y pred refers to the predicted radar echo image, Y truth Refers to the actual radar echo image; Among them, the calculation formula of Charbonnier Loss is: In the Charbonnier Loss formula, Y pred is the predicted radar echo image, Y truth is the real radar echo image, and ε is a constant.

10. A short-term precipitation prediction system based on the MSPTS-Net network model, characterized in that: include: An acquisition module is used to obtain radar echo images and form a radar echo data set; A prediction model building module is used to build a network model based on MSPTS-Net, embed a spatiotemporal feature extraction and fusion mechanism in the framework of the MSPTS-Net, and generate an initial short-term rainfall prediction model; A training module, used to train the initial short-term rainfall prediction model using the radar echo data set, and to reversely tune and optimize network parameters through a combined loss function to obtain a final short-term rainfall prediction model; The output module is used to input the radar echo image to be measured into the final short-term rainfall prediction model to obtain the short-term rainfall prediction result.

Images