Short-term precipitation prediction method and device based on multi-scale spatiotemporal consistency

Abstract

This invention discloses a method and apparatus for predicting short-term rainfall based on multi-scale spatiotemporal consistency. The method includes: encoding and constraining spatial distribution features using obtained rainfall time-series features to obtain spatiotemporal fusion features of rainfall; using multi-scale spatiotemporal consistency supervision, fitting Gaussian distributions to the spatiotemporal fusion feature data of rainfall at different scales, using the differences between features at different scales as supervision loss; using nearest neighbor difference consistency to mutually constrain neighboring rain pixels in the obtained predicted rainfall map to ensure the correlation of local rainfall; calculating loss functions at the feature map level and pixel level respectively, and adding them with the consistency supervision loss and the nearest neighbor difference consistency loss as a global loss, used for optimizing the training of the neural network to obtain the optimal short-term rainfall prediction model for short-term rainfall prediction. The apparatus includes a processor and a memory. This invention achieves effective prediction of short-term rainfall and improves disaster early warning capabilities.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and short-term rainfall prediction, and in particular to a method and apparatus for short-term rainfall prediction based on multi-scale spatiotemporal consistency. Background Technology

[0002] Short-term precipitation forecasting refers to the quantitative forecasting of short-term (0-6 hours) weather phenomena such as rainfall and convection using meteorological observation data such as radar echoes, numerical models, and automatic weather stations. [1] With the intensification of marine climate change, the frequency and intensity of extreme rainfall events are constantly increasing. Extreme heavy rainfall has triggered severe rainstorms and floods in coastal and inland areas, significantly impacting agriculture, transportation, and communications, causing severe economic losses and casualties. Short-term forecasting of localized heavy rainfall, strong winds, and other severe convective weather is of great importance for establishing a disaster early warning mechanism.

[0003] Traditional methods for predicting short-term precipitation typically employ numerical simulation based on actual atmospheric conditions. This involves solving the dynamic and thermodynamic equations governing atmospheric motion through numerical calculations to predict precipitation.

[0004] Because the atmosphere is a highly complex chaotic system, the relevant equations based on dynamic modeling are very complex, computationally intensive, and difficult to predict quickly. Summary of the Invention

[0005] This invention provides a method and apparatus for predicting short-term rainfall based on multi-scale spatiotemporal consistency. This invention solves the problems of insufficient mining of regional rainfall spatiotemporal characteristics, slow prediction speed, low accuracy, and inaccurate prediction in existing technologies, thereby achieving effective prediction of short-term rainfall and improving disaster early warning capabilities. See the description below for details:

[0006] Firstly, a short-term rainfall prediction method based on multi-scale spatiotemporal consistency, the method comprising:

[0007] The obtained rainfall temporal features are used to encode and constrain the spatial distribution features to obtain rainfall spatiotemporal fusion features;

[0008] Using multi-scale spatiotemporal consistency supervision, Gaussian distribution fitting is performed on spatiotemporal fusion feature data of rainfall at different scales, and the differences between features at different scales are used as supervision loss.

[0009] By utilizing the nearest neighbor difference consistency, neighboring rain pixels in the acquired predicted rainfall map are mutually constrained to ensure the correlation of local rainfall.

[0010] Loss functions are calculated at the feature map level and the pixel level respectively, and added to the consistency supervision loss and the nearest neighbor difference consistency loss as the global loss, which is used for the optimization training of the neural network to obtain the best short-term rainfall prediction model for short-term rainfall prediction.

[0011] A second aspect is a short-term rainfall prediction device based on multi-scale spatiotemporal consistency, the device being used to perform the steps of the method described in the first aspect, the device comprising:

[0012] A multi-scale spatiotemporal feature extraction module is used to obtain the temporal and spatial distribution features of rainfall from shallow to deep at different scales;

[0013] The rainfall spatiotemporal fusion module utilizes temporal-constrained spatial coding operations to encode and constrain features with rainfall spatial location information using features containing rainfall temporal information, thereby obtaining rainfall spatiotemporal fusion features.

[0014] The multi-scale spatiotemporal consistency supervision module performs Gaussian distribution fitting on spatiotemporal fusion feature data of rainfall at different scales, and uses the differences between multi-scale features as auxiliary loss for consistency supervision.

[0015] The regional rainfall prediction module transforms the deep abstraction of low-precision spatiotemporal rainfall features into deep abstraction of high-precision features, and then fuses them with shallow high-precision features to generate a rainfall map containing global features for regional rainfall prediction.

[0016] Thirdly, a short-term rainfall prediction device based on multi-scale spatiotemporal consistency, the device comprising: a processor and a memory, the memory storing program instructions, the processor calling the program instructions stored in the memory to cause the device to perform the method steps described in any of the first aspects.

[0017] The beneficial effects of the technical solution provided by this invention are:

[0018] 1. This invention overcomes the shortcomings of traditional numerical simulation, which requires complex modeling and huge computing power, and solves the meteorological chaos problem from a purely data-driven perspective.

[0019] 2. This invention generates multi-scale feature maps from both temporal and spatial perspectives of rainfall, mining detailed and abstract features of rainfall. It utilizes cross-covariance attention to achieve long-distance spatial modeling, effectively overcoming the shortcomings of RNNs (Recurrent Neural Networks) in modeling long-distance spaces and the need for massive computational resources in Transformers (pure attention mechanisms). Furthermore, it employs temporal-constrained spatial encoding operations, using features with rainfall temporal information to encode and constrain features with rainfall spatial location information, thus strengthening spatiotemporal feature fusion. This improves short-term prediction capabilities and allows for rapid deployment in practical applications.

[0020] 3. This invention utilizes multi-scale spatiotemporal consistency supervision, using the differences between multi-scale rainfall spatiotemporal fusion features as auxiliary loss for consistency supervision, which bridges the semantic gap between multi-scale features and improves the representational ability of features.

[0021] Therefore, this invention can fully learn and mine the temporal and spatial information, details and abstract information of rainfall, reduce the required computing resources, improve the accuracy, precision and speed of rainfall prediction, thereby meeting people's needs for weather forecasts, monitoring short-term heavy rainfall, and realizing early warning of natural disasters. Attached Figure Description

[0022] Figure 1 This is a flowchart of a short-term rainfall prediction method based on multi-scale temporal consistency;

[0023] Figure 2 A diagram illustrating the network structure for short-term rainfall prediction with multi-scale temporal consistency.

[0024] Figure 3 This is a diagram of the temporal constraint spatial coding structure.

[0025] Figure 4 This is a schematic diagram of a short-term rainfall prediction device based on multi-scale spatiotemporal consistency. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the embodiments of the present invention will be described in further detail below.

[0027] In recent years, data-driven deep learning has developed rapidly. Simultaneously, observational satellites, radar, and sensor networks provide massive amounts of meteorological data, thus weather forecasting and deep learning have a natural coupling relationship. Deep learning-based short-term precipitation forecasting has significant advantages over traditional numerical forecasting in terms of timeliness and resolution. Furthermore, deep learning can leverage expert knowledge and experience to improve average forecast accuracy and utilize abstract forecasting knowledge that is unavailable in statistical and numerical models. Currently, deep learning-based short-term precipitation forecasting models are mainly divided into three categories: First, time-series forecasting models, which treat short-term precipitation forecasting as a spatiotemporal sequence prediction problem. Commonly used network structures include ConvLSTM (Convolutional Long Short-Term Memory), ConvGRU (Convolutional Gated Recurrent Unit), and TrajGRU (Trajectory Gated Recurrent Unit). [2][3][4] One approach effectively mines the temporal characteristics of rainfall information but struggles to achieve long-range spatial modeling. Another approach is based on UNet (U-shaped network) prediction models, treating short-term precipitation forecasting as a graph-to-graph problem, such as SmaAt-UNet (small attention-based U-shaped network) and AA-TransUNet (attention-supervised TransUNet). [5][6][7]First, it integrates global spatial information, but does not explicitly model temporal relationships and suffers from information loss across multiple scales. Second, it is a prediction model based on Transformer, which can achieve long-distance modeling, but requires a large amount of computing resources.

[0028] Deep learning has promising applications in the meteorological field. By designing suitable deep learning networks based on the temporal and spatial characteristics of existing data, the accuracy, precision, and timeliness of short-term rainfall forecasts can be improved, effectively preventing natural disasters and protecting people's lives and property.

[0029] Example 1

[0030] A short-term rainfall prediction method based on multi-scale spatiotemporal consistency, see [link to relevant documentation]. Figure 1 The short-term rainfall forecasting method includes the following steps:

[0031] 101: Based on the rainfall map generated by radar echo image, the temporal and spatial aspects are modeled separately. By reducing the resolution multiple times and adjusting the number of channels, the temporal and spatial distribution characteristics of rainfall from shallow to deep at different scales are obtained.

[0032] 102: The obtained rainfall temporal features are used to encode and constrain the spatial distribution features to obtain the spatiotemporal fusion features of rainfall;

[0033] 103: By utilizing multi-scale spatiotemporal consistency supervision, Gaussian distribution fitting is performed on the spatiotemporal fusion feature data of rainfall at different scales. The differences between features at different scales are used as supervision loss for consistency supervision, thereby improving the representation ability of multi-scale spatiotemporal fusion features of rainfall.

[0034] 104: The low-resolution spatiotemporal fusion features of rainfall extracted by the deep network of the encoder are restored to high-resolution spatiotemporal fusion features by the decoder, and then fused with the high-resolution spatiotemporal fusion features of rainfall extracted by the shallow network of the encoder to generate a predicted rainfall map containing global features.

[0035] 105: By utilizing the nearest neighbor difference consistency, neighboring rain pixels in the predicted rainfall map are mutually constrained to ensure the correlation of local rainfall;

[0036] 106: Design a feature loss function, calculate the loss at the feature map level and the pixel level respectively, solve the problem of excessive smoothing in the predicted image, and add it with the supervision loss obtained in step 103 and the nearest neighbor difference consistency loss obtained in step 105 as the global loss, which is used for the optimization training of the neural network to obtain the best short-term rainfall prediction model.

[0037] 107: Use the optimal short-term rainfall prediction model obtained in step 106 for rainfall prediction. Input a sequence of T frames of rainfall spatiotemporal map and output a rainfall prediction map t minutes later. Each pixel value represents the amount of precipitation, thus realizing the prediction of short-term rainfall.

[0038] In summary, the embodiments of the present invention, through steps 101-107, overcome the complex modeling of traditional numerical simulation or statistical analysis methods. By encoding spatial distribution constraints using rainfall temporal features, the model's ability to fuse temporal and spatial information is improved. Spatiotemporal consistency supervision bridges the semantic gap between multi-scale feature expressions, thereby improving rainfall prediction capabilities. Nearest neighbor difference consistency constrains neighboring rain pixels to ensure the correlation of local rainfall. By designing new loss functions at both the feature and pixel levels, the accuracy of rainfall prediction is improved.

[0039] Example 2

[0040] The following section uses specific calculation formulas and examples to illustrate this point. Figures 2-3 The solution in Example 1 will be further described below:

[0041] 201: Patch encoding is performed on the preprocessed multi-frame rainfall map to obtain a rainfall feature map of size C×H×W, where C is the number of channels, H is the height of the rainfall feature map, and W is the width of the rainfall feature map;

[0042] 202: Utilize cross-covariance attention to mine long-range spatial relationships, and generate feature maps S at three scales by reducing the resolution of the feature maps and adjusting the number of channels through downsampling operations. i (S 1 ,S 2 ,S 3 ) are respectively A multi-scale representation of the spatial distribution characteristics of rainfall was obtained;

[0043] 203: The input multi-frame rainfall maps are encoded temporally. Based on empirical knowledge, different weights are assigned to the input multi-frame rainfall maps according to their distance from the predicted rainfall map, and these weights are encoded as temporal information. The encoded multi-frame rainfall map data is mapped into multi-scale high-dimensional features through convolutional layers to generate rainfall spatial distribution features S. i Rainfall time series feature map T at the corresponding scale i (T 1 ,T 2 ,T 3 );

[0044] 204: Using temporal-constrained spatial coding operations, features with rainfall temporal information are used to encode and constrain features with rainfall spatial location information to obtain spatiotemporal fusion features of rainfall;

[0045] The temporal constraint spatial coding formula is as follows:

[0046]

[0047]

[0048] Q = C p (C d (N(T i ))),K=C p (C d (N(S i V=C p (C d (N(S i (3)

[0049] Among them, C d (·) represents depthwise convolution, C p (·) represents point convolution, N(·) represents normalization, V represents spatial weighted features, K represents spatial key features, Q represents temporal query features, and α represents a learnable parameter that scales the features.

[0050] 205: By utilizing multi-scale spatiotemporal consistency supervision, Gaussian distribution fitting is performed on spatiotemporal fusion feature data of rainfall at different scales. The differences between features at different scales are used as supervision loss for consistency supervision, thereby improving the representation ability of spatiotemporal features of multi-scale rainfall.

[0051] Gaussian distributions were fitted to spatiotemporal fusion feature data of rainfall at different scales to obtain probability distribution functions denoted as p. i (x)(p1(x),p2(x),p3(x)), when the distributions are more similar, it means that the difference in the feature distributions expressed at different scales is smaller. The formula for measuring the feature difference between scales is:

[0052]

[0053] Where i represents the i-th layer scale and j represents the j-th layer scale;

[0054] The differences between scales are accumulated, and gradient backpropagation is performed using the supervised loss to update the network parameters. The smaller the supervised loss, the smaller the differences between scales, which can ensure the consistency of multi-scale feature information. The supervised loss is:

[0055]

[0056] 206: By utilizing spatiotemporal feature decoding, abstract information about rainfall is further extracted and its resolution is improved. The low-resolution spatiotemporal fusion features of rainfall extracted by the deep network of the encoder are restored to high-resolution spatiotemporal fusion features of rainfall, and the feature map size is restored to [value missing]. Recorded as High-resolution spatiotemporal fusion features of rainfall extracted from the shallow network of the encoder High-resolution spatiotemporal fusion features of rainfall obtained from the decoder Concatenate at the channel dimension A feature fusion mechanism is performed using deep convolution to reduce redundant information;

[0057] 207: The feature map after spatiotemporal decoding is processed by depthwise separable convolution and CBAM (convolutional block attention mechanism) to fuse temporal and spatial information as well as local and global information, generating a 1×288×288 rainfall map;

[0058] 208: By utilizing the nearest neighbor difference consistency, neighboring rain pixels in the predicted rainfall map are mutually constrained to ensure the correlation of local rainfall;

[0059] For each rain pixel p(m, n) in the predicted rainfall map, where m and n represent the rain pixel located in the m-th row and n-th column of the predicted rainfall map, calculate the difference between this rain pixel and its 8 neighboring pixel values, denoted as vector P(p0, p1, p2, p3, p4, p5, p6, p7). Calculate the difference between the rain pixel q(m, n) at the same location in the actual rainfall map and its 8 neighboring pixel values, denoted as vector Q(q0, q1, q2, q3, q4, q5, q6, q7). Calculate the distance between vectors P and Q.

[0060]

[0061] The consistency loss due to proximity difference is:

[0062]

[0063] 209: Design a feature loss function, calculate the loss at the feature map level and the pixel level for the predicted rainfall map containing global features and the actual rainfall map respectively, and add them with the supervision loss and the proximity difference consistency loss as the global loss. Use the loss to optimize the neural network, where each pixel value represents the predicted rainfall.

[0064] A feature loss function is designed to pass both the actual and predicted rainfall maps through the same CNN network, calculating the loss at the feature map level. This addresses the issue of over-smoothing in the predicted image, resulting in a clearer predicted rainfall map. The feature loss function is as follows:

[0065]

[0066] Among them, W k and H k T represents the width and height of the k-th layer feature extracted by the CNN network. x,y G(I) represents the characteristics of actual rainfall. x,y This represents a map showing the characteristics of predicted rainfall.

[0067] The pixel loss function is:

[0068] Loss P =L2[T x,y , G(I) x,y (9)

[0069] Where L2(·) represents the mean squared error loss function.

[0070] The global loss function is shown below:

[0071]

[0072] Where α, β, and γ are the weights of each loss term, respectively.

[0073] In summary, the embodiments of the present invention overcome the shortcomings of traditional numerical simulation, which requires complex modeling and massive computing power, through steps 201-207, and solve the meteorological chaos problem from a purely data-driven perspective. Long-distance spatial modeling effectively compensates for the weaknesses of RNN networks in long-distance spatial modeling and CNN (Convolutional Neural Network) networks in temporal mining. Temporal-constrained spatial encoding operations are used to encode and constrain features with rainfall temporal information, strengthening the fusion of spatiotemporal features. Multi-scale spatiotemporal consistency supervision is used, with the differences between multi-scale spatiotemporal fusion features of rainfall serving as a supervisory loss for consistency supervision, bridging the semantic gap between multi-scale features, improving the representational ability of features, and enabling the prediction of short-term regional rainfall. Nearest neighbor difference consistency is used to mutually constrain neighboring rain pixels, ensuring the correlation of local rainfall. By designing new loss functions at both the feature and pixel levels, the accuracy of rainfall prediction is improved.

[0074] Example 3

[0075] The feasibility of the schemes in Examples 1 and 2 is verified below with specific experiments, as detailed in the following description:

[0076] The dataset comes from precipitation data from the Royal Netherlands Meteorological Institute. After removing noise signals using two C-band Doppler weather radars, the reflectivity factor Z was converted to precipitation intensity R. This dataset contains over 420,000 rain images from the Netherlands and neighboring countries from 2016 to 2019, with a spatial resolution of 1 km and a temporal resolution of 5 minutes. To bias the network towards precipitation prediction, two additional datasets, NL-20 and NL-50, were created based on the minimum percentage of rainy pixels in the target images within the dataset, representing samples where at least 20% or 50% of the pixels contain rain, respectively.

[0077] This experiment was conducted on the NL-50 dataset. 5734 precipitation images from 2016-2018 were used as the training set, 20% of the precipitation images were randomly sampled from the training set as the validation set, and 1557 precipitation images from 2019 were used as the test set. Twelve consecutive frames of precipitation images were used as input, and precipitation images 30 minutes later were used as labels.

[0078] By training a rainfall model and selecting the best model based on evaluation criteria, the rainfall values ​​for various regions in the area can be predicted 30 minutes later by inputting 12 consecutive frames of precipitation maps into the model.

[0079] Example 4

[0080] A short-term rainfall prediction device based on multi-scale spatiotemporal consistency, see [link to documentation]. Figure 4 The device includes:

[0081] A multi-scale spatiotemporal feature extraction module continuously reduces the resolution to obtain rainfall temporal and spatial distribution features from shallow to deep at different scales;

[0082] The rainfall spatiotemporal fusion module utilizes temporal-constrained spatial coding operations to encode and constrain features with rainfall spatial location information using features containing rainfall temporal information, thereby obtaining rainfall spatiotemporal fusion features.

[0083] The multi-scale spatiotemporal consistency supervision module performs Gaussian distribution fitting on spatiotemporal fusion feature data of rainfall at different scales. The differences between multi-scale features are used as auxiliary loss for consistency supervision, thereby improving the representation ability of spatiotemporal features of multi-scale rainfall.

[0084] The regional rainfall prediction module transforms the deep abstraction of low-precision spatiotemporal rainfall features into deep abstraction of high-precision features, and then fuses them with shallow high-precision features to generate a rainfall map containing global features for regional rainfall prediction.

[0085] It should be noted that the device descriptions in the above embodiments correspond to the method descriptions in the embodiments, and the embodiments of the present invention will not be repeated here.

[0086] In summary, the embodiments of the present invention improve the accuracy of short-term rainfall prediction by extracting multi-scale spatiotemporal features of rainfall through the above modules, solving the problems of insufficient accuracy and slow prediction speed in existing technologies, and realizing the forecasting and early warning of rainfall.

[0087] Example 5

[0088] A short-term rainfall prediction device based on multi-scale spatiotemporal consistency, the device comprising: a processor and a memory, wherein the memory stores program instructions, and the processor calls the program instructions stored in the memory to cause the device to execute the following method steps:

[0089] The obtained rainfall temporal features are used to encode and constrain the spatial distribution features to obtain rainfall spatiotemporal fusion features;

[0090] Using multi-scale spatiotemporal consistency supervision, Gaussian distribution fitting is performed on spatiotemporal fusion feature data of rainfall at different scales, and the differences between features at different scales are used as supervision loss.

[0091] By utilizing the nearest neighbor difference consistency, neighboring rain pixels in the acquired predicted rainfall map are mutually constrained to ensure the correlation of local rainfall.

[0092] Loss functions are calculated at the feature map level and the pixel level respectively, and added to the consistency supervision loss and the nearest neighbor difference consistency loss as the global loss, which is used for the optimization training of the neural network to obtain the best short-term rainfall prediction model for short-term rainfall prediction.

[0093] Specifically, the acquisition of the rainfall time sequence characteristics is as follows:

[0094] The input multi-frame rainfall maps are encoded temporally, and different weights are assigned to the input multi-frame rainfall maps according to their distance from the predicted rainfall map. These weights are then encoded as temporal information. A convolutional layer maps the encoded multi-frame rainfall map data into multi-scale high-dimensional features, generating a feature S corresponding to the spatial distribution of rainfall. i Rainfall time series characteristics map at the corresponding scale.

[0095] Furthermore, the acquisition of the spatiotemporal fusion features of rainfall specifically includes:

[0096] By utilizing temporal-constrained spatial coding operations, features with rainfall temporal information are used to encode and constrain features with rainfall spatial location information, thereby obtaining spatiotemporal fusion features of rainfall.

[0097] The difference between features at different scales is used as the supervision loss:

[0098] The formula for measuring feature differences between scales is:

[0099]

[0100] Where i represents the i-th layer scale and j represents the j-th layer scale;

[0101] The differences between scales are accumulated, and gradient backpropagation is performed using the supervised loss to update the network parameters. The supervised loss is:

[0102]

[0103] Where, p i (x) is the probability density function of the i-th layer feature; p j (x) is the probability density function of the j-th layer feature; D is the formula for measuring the feature difference between scales.

[0104] Furthermore, the nearest neighbor difference consistency loss is:

[0105] For each rain pixel in the predicted rainfall map, calculate the difference between that rain pixel and its 8 neighboring pixel values, denoted as vector P. Calculate the difference between the rain pixel at the same location in the actual rainfall map and its 8 neighboring pixel values, denoted as vector Q. Calculate the distance between vectors P and Q.

[0106]

[0107] The consistency loss due to proximity difference is:

[0108]

[0109] The feature loss function is:

[0110]

[0111] Among them, W k and H k T represents the width and height of the k-th layer feature extracted by the CNN network. x,y G(I) represents the characteristics of actual rainfall. x,y This represents a map showing the characteristics of predicted rainfall.

[0112] The pixel loss function is:

[0113] Loss P =L2[T x,y , G(I) x,y ]

[0114] Where L2(·) represents the mean squared error loss function;

[0115] The global loss function is:

[0116]

[0117] Where α, β, and γ are the weights of each loss term, respectively.

[0118] It should be noted that the device descriptions in the above embodiments correspond to the method descriptions in the embodiments, and the embodiments of the present invention will not be repeated here.

[0119] The execution entities of the aforementioned processor and memory can be devices with computing functions such as computers, microcontrollers, and single-chip microcomputers. In specific implementations, the embodiments of the present invention do not limit the execution entities and can select them according to the needs of actual applications.

[0120] Data signals are transmitted between the memory and the processor via a bus, which will not be elaborated upon in this embodiment of the invention.

[0121] Based on the same inventive concept, embodiments of the present invention also provide a computer-readable storage medium, the storage medium including a stored program, which, when the program is running, controls the device where the storage medium is located to execute the method steps in the above embodiments.

[0122] The computer-readable storage medium includes, but is not limited to, flash memory, hard disk, solid-state drive, etc.

[0123] It should be noted that the description of the readable storage medium in the above embodiments corresponds to the description of the method in the embodiments, and the embodiments of the present invention will not be repeated here.

[0124] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated.

[0125] A computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in or transmitted through a computer-readable storage medium. A computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic or semiconductor, etc.

[0126] References:

[0127] [1] Yi Lei. Research on short-term precipitation prediction based on spatiotemporal prediction network and convolutional neural network [D]. Nanjing University of Information Science and Technology, 2022. DOI:10.27248 / d.cnki.gnjqc.2022.001176.

[0128] [2]X.Shi,Z.Chen,H.Wang,D.-Y.Yeung,W.-k.Wong,and W.c.Woo,“Convolutional LSTM Network:A Machine Learning Approach for PrecipitationNowcasting,”in CVPR,jun 2015.

[0129] [3]Nicolas,B.;Li,Y.;Chris,P.;Aaron,C.Delving deeper intoconvolutional networks for learning video representations.arXiv 2015,arXiv:1511.06432.

[0130] [4]Shi,X.;Gao,Z.;Lausen,L.;Wang,H.;Yeung,D.-Y.;Wong,W.-K.;Woo,W.-C.Deep learning for precipitation nowcasting:A benchmark and a newmodel.arXiv 2017,arXiv:1706.03458.

[0131] [5]Han,L.etal.(2021).“Convective precipitation nowcasting using U-NetModel,”IEEE Transactions on Geoscience and Remote Sensing,2021.

[0132] [6]K.Trebing,T.Sta′nczyk,and S.Mehrkanoon,“SmaAt-UNet:Precipitationnowcasting using asmall attention-unet architecture,”Pattern RecognitionLetters,vol.145,pp.178–186,2021.

[0133] [7] Y.Yang and S.Mehrkanoon, "AA-TransUNet: Attention AugmentedTransUNet For Nowcasting Tasks," 2022International Joint Conference on NeuralNetworks(IJCNN), 2022, pp.01-08, doi:10.1109 / IJCNN55064.2022.9892376.

[0134] Unless otherwise specified, the model numbers of the various devices in this embodiment of the invention are not limited, and any device that can perform the above functions is acceptable.

[0135] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0136] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A short-term rainfall prediction method based on multi-scale spatio-temporal consistency, characterized in that, The method comprises: Based on the radar echo map generated rainfall map, respectively, the time series and space are modeled, by reducing the resolution and adjusting the channel number multiple times to obtain different scales from shallow to deep rainfall time series characteristics and spatial distribution characteristics; The obtained rainfall time series characteristics are used to encode and constrain the spatial distribution characteristics, and rainfall spatiotemporal fusion characteristics are obtained; The multi-scale space-time consistency supervision is used to perform Gaussian distribution fitting on the rainfall space-time fusion feature data at different scales, and the difference between the features at different scales is taken as a supervision loss; mutual covariance attention is used to mine long-distance spatial relationships, and a down-sampling operation is used to reduce the resolution of the feature map and adjust the number of channels, thereby generating feature maps at three scales , , , ​ The low-resolution rainfall spatiotemporal fusion characteristics extracted by the deep network of the encoder are restored to high-resolution rainfall spatiotemporal fusion characteristics through the decoder, and then the high-resolution rainfall spatiotemporal fusion characteristics extracted by the shallow network of the encoder are fused to generate a predicted rainfall map containing global features; The obtained predicted rainfall map is mutually constrained by using the near neighbor difference consistency, so as to ensure the correlation of local rainfall; Loss functions are calculated at the feature map level and the pixel level respectively, and are added to the consistency supervision loss and the near neighbor difference consistency loss as a global loss for optimization training of the neural network, so as to obtain an optimal short-term rainfall prediction model for short-term rainfall prediction; The rainfall time series characteristics are obtained in detail as follows: The input multi-frame rainfall map is encoded in time sequence, the distance of the predicted rainfall map is predicted according to the distance, different weights are given to the input multi-frame rainfall map, and the input multi-frame rainfall map is encoded as time sequence information, the encoded multi-frame rainfall map data is mapped to multi-scale high-dimensional features through a convolution layer, and a rainfall spatial distribution feature is generated A rainfall time sequence feature map of a corresponding scale The near neighbor difference consistency loss is as follows: For each rain pixel in the predicted rainfall map, the difference between the rain pixel and its 8-neighborhood pixel values is calculated, denoted as vector P, the difference between the same position rain pixel in the real rainfall map and its 8-neighborhood pixel values is calculated, denoted as vector Q, and the distance between vectors P and Q is calculated: d m，n = ； The near neighbor difference consistency loss is as follows: ； H is the height of the rainfall feature map, W is the width of the rainfall feature map, and m and n represent the mth row and nth column in the predicted rainfall map.

2. The short-term rainfall prediction method based on multi-scale spatiotemporal consistency according to claim 1, characterized in that, The rainfall spatiotemporal fusion characteristics are obtained in detail as follows: The rainfall spatiotemporal fusion characteristics are obtained by using the time series constraint spatial encoding operation to encode and constrain the rainfall spatial position information characteristics with the rainfall time series information characteristics.

3. The short-term rainfall prediction method based on multi-scale spatiotemporal consistency according to claim 1, characterized in that, The difference between the characteristics at different scales is taken as a supervision loss as follows: The scale difference measurement formula is as follows: ； Wherein, i represents the i th layer scale, and j represents the j th layer scale; The difference between the scales is accumulated, the gradient is returned through the supervision loss, and the network parameters are updated, and the supervision loss is as follows: ； wherein, the probability density function of the i-th layer feature; D is the inter-scale feature difference measure formula.

4. The short-term rainfall prediction method based on multi-scale spatiotemporal consistency according to claim 3, characterized in that In the present application, The feature loss function is as follows: ； where W k and H k denote the width and height of the k-th layer feature extracted by the CNN network, T x,y denotes the real rainfall feature map, G(I) x,y denotes the predicted rainfall feature map; The pixel loss function is as follows: ； wherein, (·) represents a mean squared error loss function; + + ； wherein a, b, g, are the weights of the respective losses.

5. A short-term rainfall prediction device based on multi-scale spatiotemporal consistency, characterized by, The device is used to execute the method of claim 1, and the device comprises: A multi-scale spatiotemporal feature extraction module is used to obtain rainfall time series and spatial distribution characteristics from shallow to deep at different scales; A rainfall spatiotemporal fusion module is used to obtain rainfall spatiotemporal fusion characteristics by using a time series constraint spatial encoding operation to encode and constrain rainfall spatial position information characteristics with rainfall time series information characteristics; A multi-scale spatiotemporal consistency supervision module is used to perform Gaussian distribution fitting on rainfall spatiotemporal fusion characteristic data at different scales, and the difference between multi-scale characteristics is taken as an auxiliary loss for consistency supervision; A regional rainfall prediction module is used to change deep abstract low-precision rainfall spatiotemporal fusion characteristics into deep abstract high-precision characteristics, and fuse the high-precision characteristics with shallow high-precision characteristics to generate a rainfall map containing global features, and perform regional rainfall prediction.

6. A short-term rainfall prediction device based on multi-scale spatiotemporal consistency, characterized by, The apparatus comprises a processor and a memory having program instructions stored therein, the processor invoking the program instructions stored in the memory to cause the apparatus to perform the method of any one of claims 1-4.

Images