A short-term precipitation prediction method based on deep attention voxel flow in complex frequency domain

Abstract

The present invention discloses a short-term precipitation prediction method based on complex frequency domain deep attention voxel flow. The method comprises constructing meteorological image data of continuous time series into three-dimensional video stream data; inputting the three-dimensional video stream data into the constructed three-dimensional space-time voxel motion model, wherein the three-dimensional space-time voxel motion model comprises a spatial path, a motion path and a frequency domain path; the spatial path is used to predict the three-dimensional voxel state at a future time point according to the input three-dimensional video stream data; the motion path is used to capture the dynamic relationship between the nodes of the three-dimensional video stream data based on the space-time attribute random walk similarity algorithm; the frequency domain path is used to invert the input three-dimensional video stream data to obtain the prediction information of the time domain; the three-dimensional voxel state, the dynamic relationship between the nodes and the prediction information of the time domain are fused to obtain the final short-term precipitation prediction result. The present invention can timely capture the evolution of cloud maps and the formation process of precipitation, and improve the prediction accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of short-term precipitation prediction methods based on complex frequency domain deep attention voxel flow, and in particular to a short-term precipitation prediction method based on complex frequency domain deep attention voxel flow. Background Art

[0002] Short-term rainfall forecast is an important task in the field of atmospheric science, which aims to accurately forecast local rainfall conditions in a short period of time in the future (usually 0-2 hours). At present, this technology has been widely used in meteorological disaster warning, transportation management, urban construction management, agricultural production and other aspects. Its importance lies not only in facilitating people's lives, but also in helping to prevent and reduce disasters, which has always been a key task in the field of weather forecasting. However, with the acceleration of climate change and urbanization, the atmospheric situation has become more and more complicated, and various meteorological phenomena have occurred frequently. Climate change has brought many adverse effects on people's lives and work, and increased many uncertain risks. Effectively predicting and preventing the above-mentioned meteorological phenomena will help reduce losses.

[0003] In recent years, with the rapid development of computer vision and deep learning technology, video prediction technology has gradually been applied to the field of meteorology. Video prediction technology can predict image changes at future time points by analyzing and modeling continuous time series images. This technology has significant advantages in short-term precipitation prediction, because the precipitation process is essentially a dynamic process of spatiotemporal evolution, and video prediction technology can better capture and simulate this dynamic change. However, the current application of video prediction technology in short-term precipitation prediction still faces some challenges. For example, the complexity and variability of meteorological image data require the model to have strong adaptability and generalization capabilities; at the same time, how to effectively integrate multi-source meteorological data to improve the accuracy and reliability of prediction is also an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a short-term precipitation prediction method based on complex frequency domain deep attention voxel flow in view of the shortcomings of the prior art.

[0005] To achieve the above object, the present invention provides a short-term precipitation prediction method based on complex frequency domain deep attention voxel flow, comprising:

[0006] Constructing continuous time series meteorological image data into three-dimensional video stream data;

[0007] Inputting the three-dimensional video stream data into a constructed three-dimensional space-time voxel motion model, wherein the three-dimensional space-time voxel motion model includes a spatial path, a motion path, and a frequency domain path;

[0008] The spatial path is used to predict the 3D voxel state at a future time point based on the input 3D video stream data. ;

[0009] The motion path is used to capture the dynamic relationship between the three-dimensional video stream data nodes based on the temporal and spatial attribute random walk similarity algorithm ;

[0010] The frequency domain path is used to invert the input 3D video stream data to obtain the prediction information in the time domain. ;

[0011] For the three-dimensional voxel state , dynamic relationships between nodes and time domain prediction information Fusion is performed to obtain the final short-term precipitation forecast result. , concat is a fusion operation.

[0012] Furthermore, the spatial path includes a voxel encoding module, a dual-channel attention selection downsampling network and a voxel decoding module, wherein the voxel encoding module is used to perform feature encoding on the input three-dimensional video stream data, the dual-channel attention selection downsampling network is used to avoid feature loss during feature extraction and downsampling, and the voxel decoding module is used to perform feature decoding to obtain a three-dimensional voxel state. ,in, represents the first time channel, is the height of the meteorological image, is the width of the weather image.

[0013] Furthermore, the dual-channel attention selection downsampling network separates the features output by the voxel encoding module To get the features and Features , as follows:

[0014]

[0015] in, is the convolution operation, For separation operation;

[0016] The dual-channel attention selection downsampling network outputs features of the voxel encoding module Based on global average pooling operation and fully connected operations Get the features respectively and Features , and the characteristics The channel weights are obtained by normalization operation and channel weight , the characteristics and Features They are:

[0017]

[0018] in, , is the pixel index, is the Relu activation function, is batch normalization;

[0019] Features and Features Assign channel weights and obtain features and Features , and in the characteristics and Features Select important feature channels for fusion in the direction, as follows:

[0020]

[0021]

[0022]

[0023] in, Select the features of the downsampled network output for the dual-channel attention, Select an operation for the feature.

[0024] Furthermore, the motion path includes a dynamic graph network, and the dynamic graph network captures the dynamic relationship between the three-dimensional video stream data nodes in the following manner:

[0025] Each picture are transformed into a weighted graph ,in, represents a subset of vertices, represents the edge set, s, v, and p are the state, node, and position respectively;

[0026] Calculate the similarity between two nodes for:

[0027]

[0028] in, , For weighted graph The eigenvectors corresponding to nodes i and j in the P Represents the dimension of the attribute vector;

[0029] Calculate the similarity measure matrix between node i and node j in time and space for:

[0030]

[0031] in, is the similarity between node i and node j in space, is the similarity between node i and node j in time, and are the weighted fusion coefficients of spatial and temporal attributes, is the set number of random walk steps, is the normalization factor, represents the similarity score between node i and node j, represents the set of neighbor nodes of i, represents the set of neighbor nodes of j;

[0032] By introducing a bidirectional GRU module, we ensure that the dynamic graph network fully learns the contextual relationship in the time series data, optimizes the data distribution through a normalization layer, and refines the data features through a multi-layer perceptron linear layer. Then, by introducing a residual connection strategy, we avoid the performance degradation that may occur in the deep network during the learning process, and finally output the dynamic relationship between nodes. , Indicates the second time channel.

[0033] Furthermore, the frequency domain path is converted to the frequency domain by fast Fourier transform, and then filtered, and the frequency is interpolated using a single complex-valued linear layer, and the interpolated frequency data is mapped back to the time domain to obtain the frequency domain path characteristics. , Indicates the third time channel.

[0034] Beneficial effects: The present invention proposes a video prediction method to predict the distribution of future precipitation, which can timely capture the evolution of cloud maps and the formation process of precipitation, and improve the spatial resolution; in the spatial path, the use of pooling layers is avoided, and dual-channel attention selection downsampling is proposed to solve the problems of loss of spatiotemporal information and low prediction accuracy; in the motion path, the dynamic spatiotemporal relationship between node variables is captured and simulated, which helps to improve the model's capture and utilization of key features in the data, thereby improving the accuracy of the prediction; in the frequency domain path, Fourier transform can provide detailed information on the signal in time and frequency, so as to perform time-frequency analysis, which is conducive to identifying problems such as sudden events in the signal, frequency changes or the evolution of frequency components over time, and can help capture the dynamic characteristics and change patterns in the data, thereby improving the accuracy of the prediction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of the structure of a three-dimensional space-time voxel motion model according to an embodiment of the present invention;

[0036] Figure 2 A schematic diagram of the structure of a dual-channel attention selection downsampling network according to an embodiment of the present invention;

[0037] Figure 3 The figure is a schematic diagram of the principle of the frequency domain path according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. The embodiments are implemented based on the technical solution of the present invention. It should be understood that these embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0039] The embodiment of the present invention provides a method for predicting short-term precipitation based on complex frequency domain deep attention voxel flow, including:

[0040] The continuous time series of meteorological image data (such as RGB data) is constructed into three-dimensional video stream data.

[0041] The three-dimensional video stream data is input into the constructed three-dimensional spatiotemporal voxel motion model, see Figure 1 , which illustrates the structure of the three-dimensional space-time voxel motion model of an embodiment of the present invention, wherein the three main paths are the space path, the motion path, and the frequency domain path from left to right.

[0042] The spatial path is used to predict the 3D voxel state at a future time point based on the input 3D video stream data. , that is, the position, movement, density and other information of the voxel. It can accurately model complex fluid dynamics behaviors, including vortices, turbulence and other phenomena. By discretizing the fluid space into voxels (three-dimensional pixels), various details and features of the fluid motion process can be captured more accurately. Including irregular boundaries and complex fluid flow paths, it has great flexibility and practicality.

[0043] Specifically, the spatial path includes a voxel encoding module, a dual-channel attention selection downsampling network (DCAM), and a voxel decoding module. The voxel encoding module is used to encode the input 3D video stream data, the dual-channel attention selection downsampling network is used to avoid feature loss during feature extraction and downsampling, and the voxel decoding module is used to decode the features to obtain the 3D voxel state. ,in, represents the first time channel, is the height of the meteorological image, is the width of the weather image.

[0044] Ordinary downsampling networks are mostly pooling layers and convolution layers with a stride of 2, which can remove redundant information, but are also accompanied by the loss of key features. In order to solve the above problems, the present invention proposes a dual-channel attention selection downsampling network structure, specifically Figure 2 As shown in Figure 2. During downsampling, no maximum pooling layer is used in the (DCAM) network structure. Instead, in order to retain all informative features and reduce the size of image features, the above dual-channel attention selection downsampling network outputs features by separating the voxel encoding module To get the features and Features , as follows:

[0045]

[0046] in, is the convolution operation, For separation operation.

[0047] Features of the output of the voxel encoding module using a dual-channel attention selection downsampling network Based on global average pooling operation and fully connected operations Get the features respectively and Features , and the characteristics The channel weights are obtained by normalization operation and channel weight , the characteristics and Features They are:

[0048]

[0049] in, , is the pixel index, is the Relu activation function, is batch normalization.

[0050] Features and Features Assign channel weights and obtain features and Features , and in the characteristics and Features Select important feature channels for fusion in the direction, as follows:

[0051]

[0052]

[0053]

[0054] in, Select features of the downsampled network output for dual-channel attention, Select an operation for the feature.

[0055] The above motion path is used to capture the dynamic relationship between the nodes of the three-dimensional video stream data based on the random walk similarity algorithm of spatiotemporal attributes. . The motion path includes a dynamic graph network. In the prediction, the time attribute and the space attribute can affect the existence of links in the layer. The relationship between nodes with similar attributes is often stronger than the relationship between nodes with different attributes. The possibility of links in the layer is greater. Therefore, in order to further discover the potential relationship between meteorological variables and enhance the correlation between them, the present invention proposes a temporal and spatial attribute random walk similarity algorithm. The dynamic graph network captures the dynamic relationship between nodes of three-dimensional video stream data in the following way:

[0056] Each picture are transformed into a weighted graph ,in, represents a subset of vertices, represents a set of edges, s, v, and p are states, nodes, and positions respectively; it means that each edge Assign a weight To quantify the similarity of vertex attributes, The vertex attribute similarity information is encoded into the weighted graph as the edge weight The present invention selects the cosine similarity that only considers the validity of the sparse vector of non-zero values ​​to measure the similarity between two nodes, as follows:

[0057]

[0058] in, is the calculated similarity between two nodes, , For weighted graph The eigenvectors corresponding to nodes i and j in the P Indicates the dimension of the attribute vector. Directly using random walks in a single graph is limited, as it can only calculate the spatial attribute scores between nodes. Therefore, the present invention expands random walks into spatiotemporal attribute random walks to generate temporal and spatial attribute scores between nodes. First, the present invention virtually connects nodes at each time step. arrive For each node in the weighted graph, a biased random walk is performed, where a random node is not only in the current Move around and also jump to the next . where nodes select their next neighbors based on edge weights. A biased random walk on a weighted graph is a method for exploring similarities between nodes. Initially, a "walker" is placed on each node in the graph. At each step, the walker randomly jumps from its current node to a Adjacent nodes and a The adjacent nodes of the nodes are selected to calculate the spatiotemporal attribute similarity between nodes at the same time step and different time steps. This jump process is not equiprobable, but guided by a specific probability rule based on the weight relationship between nodes.

[0059] By introducing specially designed transition probabilities, the present invention can construct a node similarity metric based on spatiotemporal attribute random walks, which can effectively reflect the spatiotemporal attribute information between nodes. The weight bias introduced in this process means that the probability of a walker jumping from one node to another is not only random, but also takes into account the weight strength of the connection between nodes. This weight-biased random walk can be represented by a more general probability transfer matrix P. In this matrix, each element Represents the node arrive The transition probability is adjusted according to the connection weight between nodes. Specifically, the similarity measurement matrix between node i and node j in time and space is calculated: The way is as follows:

[0060]

[0061] in, is the similarity between node i and node j in space, is the similarity between node i and node j in time, and are the weighted fusion coefficients of spatial and temporal attributes, is the set number of random walk steps, is the normalization factor, represents the similarity score between node i and node j, represents the set of neighbor nodes of i, Represents the set of neighbor nodes of j. In the similarity matrix, the similarity between nodes is calculated by applying a specific formula based on their weighted spatiotemporal attribute information. The resulting matrix not only reflects the similarity between node spaces, but also reveals the pairs of nodes that are close in the temporal level of the graph. Therefore, this matrix can reflect the weight distribution of the graph and can serve as the basis for defining the weights of each connection in the graph.

[0062] After completing link prediction, we obtain two types of aggregated networks: and In order to fully explore the time series information contained in these two networks, a bidirectional GRU module (Gated Recurrent Unit) is introduced to cleverly balance the model's processing capabilities for long-term and short-term memory, ensuring that the dynamic graph network fully learns the contextual relationships in the time series data. Subsequently, the data distribution is optimized through a normalization layer, which not only stabilizes the numerical range, but also effectively prevents the gradient vanishing problem that may be encountered during training, ensuring the robustness of the model. The data features are refined through a multi-layer perceptron (MLP) linear layer. In order to maintain the excellent performance of the weights in the network, a residual connection strategy is introduced to avoid the performance degradation that may occur in the deep network during the learning process, and finally the dynamic relationship between the output nodes is obtained. , Indicates the second time channel.

[0063] The frequency domain path is used to invert the input 3D video stream data to obtain the prediction information in the time domain For details, please refer to Figure 3 The frequency domain path is converted to the frequency domain through fast Fourier transform (FFT), and then filtered (LPF). LPF effectively eliminates high-frequency components above the specified cutoff frequency, compresses the model representation, and retains important time series information. The frequency is interpolated using a single complex-valued linear layer. During the frequency domain interpolation process, the standard loss function of the time domain is used for supervision. The interpolated frequency data is then mapped back to the time domain through zero padding and IFFT to obtain the frequency domain path features. , Indicates the third time channel.

[0064] For the three-dimensional voxel state , dynamic relationships between nodes and time domain prediction information Fusion is performed to obtain the final short-term precipitation forecast result. , For fusion operation.

[0065] The above is only a preferred embodiment of the present invention. It should be noted that for ordinary technicians in this technical field, other parts not specifically described belong to the prior art or common knowledge. Several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered as the protection scope of the present invention.

Claims

1. A short-term precipitation prediction method based on complex frequency domain deep attention voxel flow, characterized in that: include: Constructing continuous time series meteorological image data into three-dimensional video stream data; Inputting the three-dimensional video stream data into a constructed three-dimensional space-time voxel motion model, wherein the three-dimensional space-time voxel motion model includes a spatial path, a motion path, and a frequency domain path; The spatial path is used to predict the 3D voxel state at a future time point based on the input 3D video stream data. ; The motion path is used to capture the dynamic relationship between the three-dimensional video stream data nodes based on the temporal and spatial attribute random walk similarity algorithm ; The frequency domain path is used to invert the input 3D video stream data to obtain the prediction information in the time domain. ; The three-dimensional voxel state , dynamic relationships between nodes and time domain prediction information Fusion is performed to obtain the final short-term precipitation forecast result. , For fusion operation; The spatial path includes a voxel encoding module, a dual-channel attention selection downsampling network and a voxel decoding module. The voxel encoding module is used to perform feature encoding on the input three-dimensional video stream data, the dual-channel attention selection downsampling network is used to avoid feature loss during feature extraction and downsampling, and the voxel decoding module is used to perform feature decoding to obtain a three-dimensional voxel state. ,in, represents the first time channel, is the height of the meteorological image, is the width of the meteorological image; The dual-channel attention selection downsampling network separates the features output by the voxel encoding module To get the features and Features , as follows: ； in, is the convolution operation, For separation operation; The dual-channel attention selection downsampling network outputs features of the voxel encoding module Based on global average pooling operation and fully connected operations Get the features respectively and Features , and the features The channel weights are obtained by normalization operation and channel weight , the characteristics and Features They are: ； in, , is the pixel index, is the Relu activation function, is batch normalization; Features and Features Assign channel weights and obtain features and Features , and in the characteristics and Features Select important feature channels for fusion in the direction, as follows: ； ； ； in, Select the features of the downsampled network output for the dual-channel attention, is a feature selection operation; The motion path includes a dynamic graph network, and the dynamic graph network captures the dynamic relationship between the three-dimensional video stream data nodes in the following manner: Each picture are transformed into a weighted graph ,in, represents a subset of vertices, represents the edge set, s, v, and p are the state, node, and position respectively; Calculate the similarity between two nodes for: ； in, , For weighted graph The eigenvectors corresponding to nodes i and j in the P Represents the dimension of the attribute vector; Calculate the similarity measure matrix between node i and node j in time and space for: ； in, is the similarity between node i and node j in space, is the similarity between node i and node j in time, and are the weighted fusion coefficients of spatial and temporal attributes, is the set number of random walk steps, is the normalization factor, represents the similarity score between node i and node j, represents the set of neighbor nodes of i, represents the set of neighbor nodes of j; By introducing a bidirectional GRU module, we ensure that the dynamic graph network fully learns the contextual relationship in the time series data, optimizes the data distribution through a normalization layer, and refines the data features through a multi-layer perceptron linear layer. Then, by introducing a residual connection strategy, we avoid the performance degradation of the deep network during the learning process, and finally output the dynamic relationship between nodes. , Indicates the second time channel.

2. The short-term precipitation prediction method based on complex frequency domain deep attention voxel flow according to claim 1 is characterized in that: The frequency domain path is converted to the frequency domain by fast Fourier transform, then filtered, and the frequency is interpolated using a single complex-valued linear layer, and the interpolated frequency data is mapped back to the time domain to obtain the frequency domain path characteristics. , Indicates the third time channel.

Images