A method and apparatus for water level prediction based on multimodal fusion and the dual-stream spatiotemporal network PredFormer

Abstract

This invention discloses a water level prediction method and apparatus based on multimodal fusion and a dual-stream spatiotemporal network, PredFormer. First, taking the time-series signal prediction problem as a starting point, considering the highly nonlinear and dynamic spatiotemporal dependence of water level and meteorological data, a dynamic dual-stream spatiotemporal network PredFormer model framework is constructed to obtain the spatiotemporal features of precipitation and water level data from multiple meteorological stations. Second, an efficient low-rank tensor fusion method is used to fuse the obtained water level time-series features with the spatiotemporal features of precipitation data from multiple stations at the feature level. Finally, a multilayer perceptron (MLP) is used as the output prediction result. This method considers both the nonlinear time-series relationship problem in water level prediction and the dynamic spatiotemporal correlation of water level and precipitation data, while also addressing the cross-site prediction problem of both the individual characteristics and commonalities of water level data from hydrological stations and precipitation data from different meteorological stations. This gives the method strong generalization ability in water level prediction across multiple meteorological stations.

Description

Technical Field

[0001] This invention relates to the field of water level prediction, and more specifically, to a water level prediction method and apparatus based on the multimodal and spatiotemporal full attention mechanism PredFormer. Background Technology

[0002] Water level and hydrological forecasting provides timely predictions of river levels, peak flow rates, and other information, which is crucial for flood control, disaster reduction, production and daily life, and power generation scheduling. Based on these forecasts, industrial and agricultural production plans, domestic water use plans, and hydropower station power generation plans can be rationally arranged to fully utilize water resources and improve the efficiency of hydropower station generation. Water level forecasting can also promptly identify potential natural disasters such as floods and droughts, enabling the implementation of appropriate disaster prevention and relief measures to minimize losses.

[0003] While numerous data-driven water level prediction methods exist, timely and accurate prediction remains a significant challenge for academics and regulatory bodies due to the highly nonlinear and dynamic spatiotemporal dependence of water level data. Furthermore, although some methods utilize neural network models for water level prediction, these methods fail to consider the relationship between water level stations and multiple meteorological stations, as well as the data relationships between and within different meteorological stations. This results in low generalization ability and requires improvement in prediction accuracy. Summary of the Invention

[0004] To address the technical problems of existing technologies, this invention provides a water level prediction method based on multimodal fusion and the PredFormer dual-stream spatiotemporal network. First, starting from the problem of time-series signal prediction, considering the highly nonlinear and dynamic spatiotemporal dependence of water level and meteorological data, a PredFormer dual-stream spatiotemporal network model framework is dynamically constructed to obtain the spatiotemporal features of precipitation and water level data from multiple meteorological stations. Second, the Efficient Low-rank Multimodal Fusion (ELMF) method is used to fuse the obtained water level time-series features with the spatiotemporal features of precipitation data from multiple stations at the feature level. Finally, a Multilayer Perceptron (MLP) is used as the output prediction result. This method considers both the nonlinear time-series relationship in water level prediction and the dynamic spatiotemporal correlation of water level and precipitation data, while also addressing the unique and common cross-site prediction problems of hydrological station water level data and precipitation data from different meteorological stations. This gives the method strong generalization ability in water level prediction across multiple meteorological stations.

[0005] To achieve the above objectives, the first aspect of this invention provides a water level prediction method based on multimodal fusion and the dual-stream spatiotemporal network PredFormer, comprising:

[0006] Determine the corresponding meteorological stations based on the predicted hydrological stations;

[0007] Collect water level monitoring data from the hydrological stations to be predicted, as well as precipitation monitoring data from the corresponding meteorological stations;

[0008] Preprocessing of collected water level monitoring data and precipitation monitoring data;

[0009] A water level prediction model based on the PredFormer spatiotemporal network was constructed, and the model was trained using preprocessed data. The PredFormer-based water level prediction model includes a feature extraction module, a feature fusion module, and a multilayer perceptron. The feature extraction module has a two-branch structure. The first branch uses a PredFormer model with a spatiotemporal full attention mechanism for input embedding, location encoding, and a PredFormer encoder to establish spatial relationships between meteorological stations and temporal relationships within meteorological stations, obtaining spatiotemporal features of precipitation data from multiple meteorological stations. The second branch uses the PredFormer model with a spatiotemporal full attention mechanism for input embedding, location encoding, and a PredFormer encoder to process the temporal relationship of water level data from the hydrological station to be predicted, obtaining temporal features of the water level data. The feature fusion module uses an efficient low-rank tensor fusion method with modality-specific factors to fuse the spatiotemporal features of precipitation data from multiple meteorological stations with the temporal features of water level data at the feature level, obtaining fused features. The multilayer perceptron outputs the water level prediction result based on the fused features.

[0010] A water level prediction model based on the dual-stream spatiotemporal network PredFormer was used to predict the water level of the hydrological station to be predicted.

[0011] In one implementation, determining the corresponding meteorological station based on the hydrological station to be predicted includes:

[0012] Based on the DEM, the river basin area of ​​the hydrological station to be predicted is delineated, and the meteorological station located upstream of the hydrological station and within the river basin area is selected as the corresponding meteorological station.

[0013] In one implementation, the collected water level monitoring data and precipitation monitoring data are preprocessed, including:

[0014] The collected water level monitoring data and precipitation monitoring data were processed for missing values, outliers, and standardization.

[0015] In one implementation, before the first branch establishes the spatial relationships between meteorological stations and the temporal relationships within meteorological stations by inputting the PredFormer model with a spatiotemporal full attention mechanism through embedding, location encoding, and PredFormer encoder, and obtaining the spatiotemporal characteristics of precipitation data from multiple meteorological stations, the method further includes:

[0016] Graph convolutional neural networks are used to model the spatial relationships of precipitation data from multiple stations, construct the correlation information between stations, and obtain multi-station structured graph data.

[0017] In one implementation, the first branch uses a PredFormer model with a spatiotemporal full attention mechanism to input embedding, location encoding, and PredFormer encoder to sequentially establish spatial relationships between meteorological stations and temporal relationships within meteorological stations, thereby obtaining spatiotemporal characteristics of precipitation data from multiple meteorological stations, including:

[0018] The PredFormer model, which uses a spatiotemporal full attention mechanism, embeds the multi-site structured graph data as multiple input sub-channels. The data of each sub-channel is flattened into a one-dimensional vector, and then the one-dimensional vector is projected into the hidden dimension through a linear layer to obtain the corresponding tensor.

[0019] The spatial location and temporal sequence of each station are encoded using absolute location encoding.

[0020] By combining multiple stacked gated transformer blocks of the PredFormer encoder with a multi-head self-attention mechanism, the spatiotemporal characteristics of precipitation data from multiple meteorological stations are obtained based on location encoding.

[0021] In one implementation, the second branch processes the temporal relationship of the water level data of the hydrological station to be predicted through the input embedding, position encoding, and PredFormer encoder of the spatiotemporal full attention mechanism PredFormer model to obtain the temporal features of the water level data, including:

[0022] The input single-site water level data is processed by embedding the PredFormer model using a spatiotemporal full attention mechanism to obtain the corresponding tensor;

[0023] The location coding method uses absolute location coding to encode the water level data of a single station.

[0024] By combining multiple stacked gated transformer blocks of the PredFormer encoder with a multi-head self-attention mechanism, the temporal characteristics of water level data are obtained based on position encoding.

[0025] In one implementation, the output layer of the efficient low-rank tensor fusion method with modality-specific factors employs an MLP regression layer.

[0026] Based on the same inventive concept, a second aspect of the present invention provides a water level prediction device based on the multimodal and spatiotemporal full attention mechanism PredFormer, comprising:

[0027] The meteorological station determination module is used to determine the corresponding meteorological stations based on the hydrological stations to be predicted.

[0028] The data collection module is used to collect water level monitoring data from the hydrological stations to be predicted and precipitation monitoring data from the corresponding meteorological stations.

[0029] The data preprocessing module is used to preprocess the collected water level monitoring data and precipitation monitoring data;

[0030] The model building and training module is used to build a water level prediction model based on the PredFormer spatiotemporal network of a dual-stream system. The model is trained using preprocessed data. The PredFormer-based water level prediction model includes a feature extraction module, a feature fusion module, and a multilayer perceptron. The feature extraction module has a two-branch structure. The first branch uses a PredFormer model input with a spatiotemporal full attention mechanism, location encoding, and a PredFormer encoder to sequentially establish the spatial relationships between meteorological stations and the temporal relationships within meteorological stations, obtaining the spatiotemporal features of precipitation data from multiple meteorological stations. The second branch uses the PredFormer model input with a spatiotemporal full attention mechanism, location encoding, and a PredFormer encoder to process the temporal relationship of the water level data from the hydrological station to be predicted, obtaining the temporal features of the water level data. The feature fusion module uses an efficient low-rank tensor fusion method with modality-specific factors to fuse the spatiotemporal features of precipitation data from multiple meteorological stations with the temporal features of the water level data at the feature level, obtaining the fused features. The multilayer perceptron outputs the water level prediction result based on the fused features.

[0031] The water level prediction module is used to perform water level prediction on hydrological stations using a pre-trained water level prediction model based on the two-stream spatiotemporal network PredFormer.

[0032] Based on the same inventive concept, a third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the water level prediction method based on multimodal fusion and dual-stream spatiotemporal network PredFormer described in the first aspect.

[0033] Based on the same inventive concept, the fourth aspect of the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the water level prediction method based on multimodal fusion and dual-stream spatiotemporal network PredFormer described in the first aspect.

[0034] Compared with the prior art, the advantages and beneficial technical effects of the present invention are as follows:

[0035] This invention provides a water level prediction method based on multimodal fusion and the PredFormer spatiotemporal network. A water level prediction model based on the PredFormer spatiotemporal network is constructed, starting from the problem of time-series signal prediction. Considering the highly nonlinear and dynamic spatiotemporal dependence of water level and meteorological data (precipitation monitoring data), the PredFormer model framework is dynamically constructed. Through the PredFormer model with a spatiotemporal full attention mechanism, the spatiotemporal features of precipitation data from multiple meteorological stations and the temporal features of water level data are dynamically acquired. The Efficient Low-rank Multimodal Fusion (ELMF) method is used to fuse the extracted temporal features of water level data with the spatiotemporal features of precipitation data from multiple stations at the feature level. Finally, a Multilayer Perceptron (MLP) is used as the output prediction result. This method considers the nonlinear temporal relationship in water level prediction, integrates the spatiotemporal dynamic correlation between water level data and precipitation data, and takes into account the cross-site prediction problem of both the individuality and commonality of water level data from hydrological stations and precipitation data from different meteorological stations. This makes the prediction model have a strong generalization ability in water level prediction across multiple meteorological stations, thus improving the accuracy of water level prediction. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a flowchart of a water level prediction method based on multimodal fusion and the dual-stream spatiotemporal network PredFormer, according to an embodiment of the present invention.

[0038] Figure 2 This is a schematic diagram of a weather station in an embodiment of the present invention.

[0039] Figure 3This is a comparison chart of the prediction results of the method in the embodiments of the present invention and the existing methods.

[0040] Figure 4 This is a block diagram of a water level prediction device based on multimodal fusion and dual-stream spatiotemporal network PredFormer, according to an embodiment of the present invention.

[0041] Figure 5 This is a structural diagram of a computer-readable storage medium provided in an embodiment of the present invention.

[0042] Figure 6 This is a structural diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] Example 1

[0045] This embodiment discloses a water level prediction method based on multimodal fusion and the dual-stream spatiotemporal network PredFormer. Please refer to [link to relevant documentation]. Figure 1 ,,include:

[0046] S1: Determine the corresponding meteorological stations based on the hydrological stations to be predicted;

[0047] S2: Collect water level monitoring data from the hydrological stations to be predicted and precipitation monitoring data from the corresponding meteorological stations;

[0048] S3: Preprocess the collected water level monitoring data and precipitation monitoring data;

[0049] S4: A water level prediction model based on the dual-stream spatiotemporal network PredFormer is constructed, and the model is trained using preprocessed data. The PredFormer-based water level prediction model includes a feature extraction module, a feature fusion module, and a multilayer perceptron. The feature extraction module has a two-branch structure. The first branch uses a PredFormer model with a spatiotemporal full attention mechanism for input embedding, location encoding, and a PredFormer encoder to establish spatial relationships between meteorological stations and temporal relationships within meteorological stations, obtaining spatiotemporal features of precipitation data from multiple meteorological stations. The second branch uses the PredFormer model with a spatiotemporal full attention mechanism for input embedding, location encoding, and a PredFormer encoder to process the temporal relationship of water level data from the hydrological station to be predicted, obtaining temporal features of the water level data. The feature fusion module uses an efficient low-rank tensor fusion method with modality-specific factors to fuse the spatiotemporal features of precipitation data from multiple meteorological stations with the temporal features of water level data at the feature level, obtaining fused features. The multilayer perceptron outputs the water level prediction result based on the fused features.

[0050] S5: Use the trained water level prediction model based on the dual-stream spatiotemporal network PredFormer to predict the water level of the hydrological station to be predicted.

[0051] Specifically, S1 involves watershed delineation and meteorological station selection; S2 involves collecting precipitation data from multiple meteorological stations and water level data from hydrological stations; S3 involves data preprocessing; and S4 involves model construction. This invention constructs a water level prediction model based on the dual-stream spatiotemporal network PredFormer. The feature extraction module adopts a dual-branch structure, with each branch employing a spatiotemporal full attention mechanism PredFormer model to extract the spatiotemporal features of precipitation data from multiple meteorological stations and the temporal series features of water level data. The feature fusion module uses an efficient low-rank tensor fusion method with modality-specific factors, thereby strengthening the correlation of multimodal features during model learning and improving prediction accuracy compared to other fusion methods. S5 involves using the constructed model for water level prediction. Because the water level prediction model considers both the nonlinear temporal relationship of water level prediction and the spatiotemporal dynamic correlation of water level and precipitation data, it also addresses the cross-site prediction problem of both the individual characteristics and commonalities of hydrological station water level data and precipitation data from different meteorological stations. This gives the prediction model a strong generalization ability in water level prediction across multiple meteorological stations, thus enabling it to obtain accurate prediction results.

[0052] In one implementation, determining the corresponding meteorological station based on the hydrological station to be predicted includes:

[0053] Based on the DEM, the river basin area of ​​the hydrological station to be predicted is delineated, and the meteorological station located upstream of the hydrological station and within the river basin area is selected as the corresponding meteorological station.

[0054] In practice, based on the hydrological station whose water level is to be predicted, the river basin area of ​​the station is delineated using the DEM (Digital Equation Model). Then, meteorological stations located upstream of the station and within the river basin area are selected as backup meteorological stations. There is no limit to the number of meteorological stations, but generally no fewer than three.

[0055] In one implementation, the collected water level monitoring data and precipitation monitoring data are preprocessed, including:

[0056] The collected water level monitoring data and precipitation monitoring data were processed for missing values, outliers, and standardization.

[0057] In practice, the preprocessing method is as follows:

[0058] First, methods such as mean replacement, linear interpolation, time interpolation, padding interpolation, and nearest neighbor interpolation were used, and the best missing value handling method was selected based on the fitting effect.

[0059] Second, the Laida criterion and Grubbs criterion are used to handle outliers. Assume that the measured variable is measured with equal precision, yielding x1, x2, ..., x... i Calculate its arithmetic mean. and residual error And calculate the standard error σ according to Bessel's formula. If a certain measured value x b The residual error v b (1≤b≤n), satisfying Then we assume x b These are outlier values ​​containing gross errors and are marked accordingly. The Bessel formula is as follows:

[0060]

[0061] n represents the number of variables being measured.

[0062] Third, linear normalization ("max-min" normalization) is used to obtain the difference between the maximum and minimum values ​​in the two datasets, and the data points are normalized to a base.

[0063]

[0064] x′ represents the normalized data, x represents the original data, and max(x) and min(x) represent the maximum and minimum values ​​of the data, respectively.

[0065] In one implementation, before the first branch establishes the spatial relationships between meteorological stations and the temporal relationships within meteorological stations by inputting the PredFormer model with a spatiotemporal full attention mechanism through embedding, location encoding, and PredFormer encoder, and obtaining the spatiotemporal characteristics of precipitation data from multiple meteorological stations, the method further includes:

[0066] Graph convolutional neural networks are used to model the spatial relationships of precipitation data from multiple stations, construct the correlation information between stations, and obtain multi-station structured graph data.

[0067] Specifically, because precipitation data from multiple stations is sparsely distributed, it cannot be directly input into PredFormer for processing. Therefore, this implementation uses a convolutional neural network to perform spatiotemporal encoding on the precipitation data, transforming it into structured graph data before inputting it into the PredFormer model. The GCN captures its spatial dependencies, thus solving the data sparsity problem.

[0068] In one implementation, the first branch uses a PredFormer model with a spatiotemporal full attention mechanism to input embedding, location encoding, and PredFormer encoder to sequentially establish spatial relationships between meteorological stations and temporal relationships within meteorological stations, thereby obtaining spatiotemporal characteristics of precipitation data from multiple meteorological stations, including:

[0069] The PredFormer model, which uses a spatiotemporal full attention mechanism, embeds the multi-site structured graph data as multiple input sub-channels. The data of each sub-channel is flattened into a one-dimensional vector, and then the one-dimensional vector is projected into the hidden dimension through a linear layer to obtain the corresponding tensor.

[0070] The spatial location and temporal sequence of each station are encoded using absolute location encoding.

[0071] By combining multiple stacked gated transformer blocks of the PredFormer encoder with a multi-head self-attention mechanism, the spatiotemporal characteristics of precipitation data from multiple meteorological stations are obtained based on location encoding.

[0072] Specifically, the dual-stream PredFormer spatiotemporal prediction model employs two PredFormer models to extract spatiotemporal features from multi-site precipitation data and target site water level data, respectively. The specific principle of the PredFormer model is as follows:

[0073] First, the PredFormer model input embedding: For precipitation data from multiple stations, the data from multiple stations are used as multiple input sub-channels. The data from each sub-channel is flattened into a one-dimensional (1D) vector, and then these one-dimensional vectors are projected onto the hidden dimension D through a linear layer to obtain a tensor. Where B, T, N, and D represent the batch number, time step, number of stations, and dimension, respectively; for the water level data of a single station, it is first flattened into a one-dimensional (1D) vector, and then projected onto the hidden dimension D through a linear layer to obtain a tensor.

[0074] Second, PredFormer location encoding: It adopts absolute location encoding, and encodes the spatial location and time step sequence of each station through a sine function.

[0075] Specifically, for position k, its position code e k The i-th component is calculated as follows:

[0076]

[0077] Each dimension of the triangular positional encoding is actually a sine function (even-dimensional component) or a cosine function (odd-dimensional component) of position k, which uses sine and cosine functions respectively according to the parity of the dimension.

[0078] Third is the PredFormer encoder: The PredFormer encoder consists of gated transformer blocks stacked in various ways.

[0079] Gated Linear Units (GLUs) are often used to replace simple linear transformations involving the element-wise product of two linear projections, one of which is passed through a sigmoid function. Various GLU variants control the information flow by replacing the sigmoid with other nonlinear functions; for example, SwiGLU replaces the sigmoid with the Swish activation function (SiLU), calculated as follows:

[0080] Swish β (x)=xσ(βx)

[0081]

[0082] Where x is the hidden representation at a specific position in the sequence, β is an additional parameter, typically set to 1; W and V are weight matrices, and b and c are bias vectors. This represents the tensor outer product.

[0083] The Gated Transformer Block (GTB) combines multi-head self-attention (MSA) with a SwiGLU-based feed-forward network (FFN). The GTB is defined as follows:

[0084] Y l=MSA(LN(Z) l ))+Z l

[0085] Z l+1 =SwiGLU(LN(Y l ))+Y l

[0086] Where MSA represents multi-head self-attention, LN is layer normalization, and Z... l Y l Both represent the features of the l-th layer.

[0087] Each element of the MSA input sequence is first mapped to three vectors, typically through a linear transformation with three weight matrices. Specifically, the input sequence X is multiplied by its respective weight matrix to obtain Q, K, and V. MSA is defined as:

[0088] MultiHead(Q,K,V)=Concat(head1,…,head h W O

[0089] head i =Attention(QW i Q ,KW i K VW i V )

[0090] Where MultiHead(Q,K,V) represents the output sequence, head i Let W represent the multi-head attention weight of the i-th element. Q, K, and V represent the three vectors mapped to each element of the input sequence: the query (target to be analyzed), the key (feature to be analyzed), and the value (result to be analyzed) matrix, respectively. i Q W i K W i V Each represents its corresponding weight.

[0091] To convert attention scores into weights, a Softmax function is applied for normalization, resulting in an N×N attention weight matrix representing the influence of each patch on other patches. Softmax ensures that the sum of all output weights is 1, allowing the model to learn the importance of each element pair.

[0092]

[0093] Where Attention(Q,K,V) represents the multi-head attention weights of the output elements, Q and K. T It is the dot product between the query and the keys, representing the degree of attention each query pays to all keys. This is a scaling factor used to stabilize gradient propagation.

[0094] The first stream (first branch) first uses a graph convolutional neural network (GCN) to model the spatial relationships of precipitation data from multiple stations. Then, it uses the PredFormer spatiotemporal prediction model to extract cross-station spatial relationships and intra-station temporal dependencies. Finally, it uses a multilayer perceptron (MLP) to map these relationships to the spatiotemporal features of the multi-station precipitation data. The specific principle is as follows:

[0095] First, the spatial relationships of precipitation data from multiple stations are modeled using GCN to construct the correlation information between stations. Precipitation data is represented as graphical signals over a set of stations. GCN uses the Thulaplace operator The basic calculation method for extracting spatial information is as follows:

[0096]

[0097] Among them, U1 (0) =X1, which is the input multi-station precipitation data; W (l) Let L be the training weight matrix for the Lth layer; For the normalized Laplace matrix, Given an adjacency matrix with self-loops, σ is the degree matrix; σ(·) is the ReLU activation function.

[0098] Subsequently, the PredFormer spatiotemporal prediction model is used to extract cross-site spatial relationships and intra-site temporal dependencies. Let the spatial features output by GCN be... Use it as input to the PredFormer model, model the spatiotemporal relationship of input U1 through the PredFormer model, and output the prediction result:

[0099]

[0100] in The PredFormer output features are then used. Finally, an MLP is applied to perform a nonlinear transformation on the PredFormer output, mapping it to the spatiotemporal features of multi-site precipitation data:

[0101]

[0102] in It is The two-dimensional vectors after flattening the dimensions N and D, σ(·) is the ReLU activation function, ReLU(X)=max(0,X); W1 and W2 are the training weight matrices, and b1 and b2 are the training bias vectors; Spatiotemporal characteristics of precipitation data from multiple stations.

[0103] In one implementation, the second branch processes the temporal relationship of the water level data of the hydrological station to be predicted through the input embedding, position encoding, and PredFormer encoder of the spatiotemporal full attention mechanism PredFormer model to obtain the temporal features of the water level data, including:

[0104] The input single-site water level data is processed by embedding the PredFormer model using a spatiotemporal full attention mechanism to obtain the corresponding tensor;

[0105] The location coding method uses absolute location coding to encode the water level data of a single station.

[0106] By combining multiple stacked gated transformer blocks of the PredFormer encoder with a multi-head self-attention mechanism, the temporal characteristics of water level data are obtained based on position encoding.

[0107] In the specific implementation process, the second stream (second branch) uses the PredFormer spatiotemporal prediction model to extract the temporal features of the water level data of the target hydrological station, and further maps the water level data to obtain the temporal features of the water level data through MLP. The specific principle is as follows:

[0108] Assume the water level data of the target hydrological station is a time series. The PredFormer model learns long-term dependencies in time series data through a self-attention mechanism. Its input-output relationship can be represented as:

[0109]

[0110] in The PredFormer output features are then used. Furthermore, an MLP is used to perform a nonlinear transformation on the PredFormer output, mapping it to time-series features of the water level data.

[0111]

[0112] in This represents the temporal characteristics of water level data.

[0113] In one implementation, the output layer of the efficient low-rank tensor fusion method with modality-specific factors employs an MLP regression layer.

[0114] The efficient low-rank multimodal fusion (ELMF) method is used to fuse the spatiotemporal features of precipitation V1 and the time series features of water level data obtained in step S4, resulting in fused features. The input-output relationship can be expressed as follows:

[0115] H = ELMF(V1, V2)

[0116] in This is a feature of fusion.

[0117] The specific principle of the ELMF method is as follows:

[0118] Multimodal fusion is expressed as a multilinear function f:

[0119] V1, V2, ..., VM→H

[0120] in H is the vector space of the input modes, and H is the vector space of the output modes.

[0121] Given a set of vector representations, It encodes single-peak information of M different modalities. The goal of multimodal fusion is to integrate the single-peak representation into a compact multimodal representation for downstream tasks.

[0122] Input tensor formed by unimodal representation Calculated using the following formula:

[0123]

[0124] in, Let zm represent the tensor outer product over a set of vectors indexed by m, where zm is the input representation with 1s appended.

[0125] Input tensor Then, a vector representation is generated through a linear layer g(·):

[0126]

[0127] Where W is the weight of the layer and b is the bias voltage. Since It is an M-order tensor (where M is the number of input modes), and the weights W are naturally... The tensor is of order (M+1); the additional (M+1) dimension corresponds to the size of the output representation dh. In the tensor dot product... In this context, the weight tensor W can be considered as an M-order tensor of dh.

[0128] For M-order tensors There always exists a form that can be precisely decomposed into vectors:

[0129]

[0130] in It is called the rank R factor of the original tensor.

[0131] Recombine and concatenate these vectors into M mode-specific low-rank factors. Let So for mode m, It is its corresponding low-rank factor, and the low-order weight tensor can be recovered in the following way:

[0132]

[0133] Therefore, h can be calculated using the following formula:

[0134]

[0135] It should be noted that for all The second dimension is the same size. We define their external products as dimensions that only cover and are not shared:

[0136] By introducing a low-rank factor, it is necessary to calculate Further calculations are needed. Using tensors... Naturally decomposed into the original input It is parallel to the mode-specific low-rank factor.

[0137] use Simplifying the equation, we get:

[0138]

[0139] in Representing the element-wise product over a tensor sequence:

[0140] A water level prediction model based on the dual-stream spatiotemporal network PredFormer is used to predict the water level of the hydrological station to be predicted. The fused feature H obtained in step five of the MLP decoding process is used to obtain the future water level prediction result. Similar to the steps in S4 above, the spatiotemporal features of precipitation data and the temporal features of water level data from multiple meteorological stations are extracted by the feature extraction module (dual-stream spatiotemporal network PredFormer); the extracted features are then fused at the feature level using an efficient low-rank tensor fusion method; finally, a multilayer perceptron (MLP) is used as the output prediction result.

[0141] The specific processing procedure of a multilayer perceptron is as follows:

[0142] H f =Flatten(H)

[0143]

[0144] in It is a one-dimensional vector after H is flattened. This is a future water level forecast.

[0145] In one implementation, the method further includes a post-evaluation of the accuracy of the water level prediction model. Based on the water level prediction results from multimodal fusion and the two-stream spatiotemporal network PredFormer, as well as subsequent actual water level monitoring data, the accuracy of the water level prediction model is post-evaluated using root mean square logarithmic error (RMSLE) and mean absolute percentage error (WMAPE) as evaluation metrics.

[0146]

[0147] Where y i and These represent the actual and predicted water levels, respectively.

[0148] This indicator, with the same root mean square error, increases or decreases the penalty for underestimating the predicted value, thereby reducing the risk of flood control impact caused by underestimating short-term water level predictions.

[0149]

[0150] Where y i and These represent the actual and predicted water levels, respectively.

[0151] WMAPE, by weighting the impact of individual data on the value of the relevant population, has smaller error fluctuations caused by extreme values, and its applicability is better than that of Mean Absolute Percentage Error (MAPE).

[0152] The concept and key points of this invention will be explained below.

[0153] Regarding water level changes, water levels themselves possess a certain degree of temporal continuity, and capturing this temporal relationship is fundamental to water level prediction. Secondly, water level values ​​are interrelated with meteorological factors (especially precipitation), making it crucial to effectively and efficiently capture the spatial and temporal dependencies between water level and meteorological data. Many data-driven water level prediction methods exist. However, due to the highly nonlinear and dynamic spatiotemporal dependence of water level data, timely and accurate prediction remains a significant challenge for academia and regulatory authorities. Compared to general spatiotemporal prediction tasks, water level prediction is unique in its multivariate inputs (water level and multiple precipitation data points) and the differences in the spatial characteristics of the data (precipitation has spatiotemporal characteristics, while water level only has temporal characteristics). Therefore, existing spatiotemporal network PredFormer models cannot be directly used for water level prediction.

[0154] Based on the need for water level prediction, this invention makes the following improvements to the spatiotemporal network PredFormer model:

[0155] 1. Adapting to multivariate input: A dual-branch structure is adopted, with PredFormer used to encode water level features and precipitation features separately, and then the two features are fused together for prediction;

[0156] 2. Addressing the sparsity of precipitation data: Graph Convolutional Neural Networks (GCNs) are used to encode multi-site precipitation data, capturing its spatial dependencies and solving the data sparsity problem;

[0157] 3. Optimize feature extraction and compression: Replace PredFormer's Patch Recovery with a fully connected layer to reduce feature dimensionality, reduce computational complexity, and make model learning more efficient.

[0158] Meanwhile, existing technologies face the following technical challenges in processing multi-site precipitation data: 1. Data sparsity: Multi-site precipitation data exhibits a sparse distribution, making it impossible to directly input into PredFormer for processing. Therefore, spatiotemporal encoding of the precipitation data is necessary to transform it into structured graph data before it can be input into the prediction model; 2. Spatial flow relationships: Precipitation data exhibits strong spatial flow between different stations, meaning that precipitation changes have spatial correlations. Therefore, the geographical topological relationships between stations need to be fully considered during the encoding process to effectively capture the spatial dependencies between stations.

[0159] Regarding feature fusion, the efficient low-rank tensor fusion (ELMF) method is commonly used for classification tasks such as multimodal emotion recognition and visual question answering. Unlike classification tasks in existing technologies, this invention applies it to the regression task of water level prediction, requiring the following improvements:

[0160] 1. Output layer modification: Replace the ELMF output layer with an MLP regression layer to adapt to continuous water level prediction;

[0161] 2. Feature Encoding Adjustment: When performing multimodal feature encoding, the spatiotemporal flow of data needs to be considered. Therefore, PredFormer is used for spatiotemporal feature encoding to capture the spatiotemporal dependence of precipitation and water level data. Water level prediction methods based on multimodal fusion and dual-stream spatiotemporal networks using PredFormer, which simultaneously consider the relationship between water level stations and multiple meteorological stations, as well as the spatiotemporal relationships between and within different meteorological stations, are relatively rare.

[0162] In the prior art, CN116151411A proposed a water level prediction method based on AutoFormer multi-feature spatiotemporal fusion, and CN116562416B disclosed a reservoir water level prediction method based on a variable modality and time-delay feature Transformer model. Compared with the AutoFormer and Transformer models in the above methods, the PredFormer model of this invention can process temporal and spatial information simultaneously in one framework, does not require computational complexity that is quadratic of the sequence length, has neither autoregression nor any convolution, is simple and efficient, has fewer parameters, and its performance is significantly better than previous methods. In addition, the model architecture of this method uses two independent branches for feature extraction, adopts the spatiotemporal characteristics of precipitation from multiple stations, and uses multimodal efficient low-rank tensor fusion ELMF. Using this fusion method can strengthen the correlation of multimodal features during model learning, and can improve prediction accuracy compared with other fusion methods.

[0163] The method proposed in this invention will be verified and explained through specific examples below.

[0164] Example: Taking a hydrological station in the lower reaches of the Han River as an example, the water level prediction method based on multimodal fusion and the spatiotemporal full attention mechanism PredFormer is used to predict the water level data of the hydrological station in the next month.

[0165] Step 1: Based on the station where the water level is to be predicted, divide the river basin area where the hydrological station is located based on the DEM, extract the basin area controlled by the hydrological station, and select 3 basic meteorological stations A, B and C located in the basin area.

[0166] Step 2: Collect water level monitoring data (unit: m) for this hydrological station from 2016 to 2020. 3 / s), and monthly precipitation monitoring data (unit: mm) for meteorological stations A, B, and C during the same time period.

[0167] Step 3: Preprocessing of monitoring data from the hydrological monitoring station and three meteorological stations, mainly including missing value handling, outlier handling, and standardization. The descriptive statistical results of the processed precipitation and water level data are as follows:

[0168] Table 1 Descriptive statistics of the data

[0169] Statistical value A B C Hydrological station Sample size 444 444 444 444 mean 61.89 60.52 72.27 21.58 Standard deviation 54.22 53.54 68.22 2.59 Minimum value 0 0 0 17.67 25th percentile 15.94 20.26 24.13 19.55 median 45.34 45.47 52.58 20.83 75th percentile 98.11 89.54 100.52 23.27 Maximum value 338.58 435.1 447.55 30.75

[0170] Step 4: Using the GCN model for spatial location encoding, and the PredFormer model's input, location encoding, and encoder, the spatial relationships between stations and the temporal relationships within stations are established sequentially to obtain the spatiotemporal characteristics of precipitation data from the three meteorological stations. The specific model parameter settings are as follows:

[0171] Table 2 shows the model parameters and specific values ​​used for multi-site precipitation data.

[0172] Model Components parameter Value Input data dimensions (B,T,N,D) (1,12,3,1) Adjacency Matrix A 3×3 normalized adjacency matrix GCN layers L(GCN) 2 GCN Hidden Layer Dimensions D(GCN) 64 GCN activation function / ReLU PredFormer Input Dimensions (B,T,N,D) (1,12,3,64) PredFormer position code Way Absolute position encoding PredFormer layer number L(PredFormer) 4 Number of attention heads h(MSA) 8 PredFormer hidden layer dimensions D(PredFormer) 64 PredFormer activation function / SwiGLU MLP Dimensions D(MLP) 64 MLP activation function / ReLU Output data dimensions (B,T,D) (1,12,64)

[0173] Step 5: Using the PredFormer model input, location encoding, and encoder, process the temporal relationship of the water level data of the target hydrological station to obtain the temporal characteristics of the water level data.

[0174] Table 3 shows the model parameters and specific values ​​used for the target site water level data.

[0175]

[0176]

[0177] Step Six: Using the efficient low-rank tensor fusion (ELMF) method with modality-specific factors, the temporal features of the previously obtained water level data are fused with the spatiotemporal features of multi-station precipitation data at the feature level. This improves the correlation between features and enhances the expressive power of the model, resulting in fused features. The fused features are then used to output the 2020 water level prediction results through the MLP.

[0178] Table 4 ELMF Parameter Information and Specific Values

[0179] Model Components parameter Value Input data dimensions (B,T,N,D) (1,12,2,64) Low-rank decomposition dimensionality r 32 Output data dimensions (B,T,D) (1,12,128) MLP Dimensions D(MLP) 64 MLP activation function / ReLU Output data dimensions (B,T,D) (1,12,1)

[0180] Step 7: Select data samples from 2016 to 2019 as the training set and 2020 data as the test set to verify the model's effectiveness. Output the 2020 water level prediction results based on a Multilayer Perceptron (MLP); simultaneously, use traditional MLP, Long Short-Term Memory (LSTM) networks, Gated Recurrent Units (GRUs), and the PredFormer spatiotemporal prediction network to predict the hydrological station's 2020 forecast results.

[0181] Step 8: Based on the model prediction results, the root mean square logarithmic error (RMSLE) and mean absolute percentage error (WMAPE) are used as evaluation indicators to rate the performance of the multimodal fusion and PredFormer spatiotemporal network-based water level prediction method compared to traditional prediction models. Calculations show that the RMSLE and WMAPE of the prediction method of this invention are 6.34% and 21.02%, respectively, indicating that the model's prediction performance is significantly better than that of traditional prediction methods.

[0182] Table 5 Comparison of water level prediction results from different prediction models

[0183] Model RMSELE (%) WMAPE (%) MLP 9.94 35.01 LSTM 9.52 33.71 GRU 8.41 29.21 PredFormer 7.81 27.04 Multimodal fusion + PredFormer 6.34 21.02

[0184] This embodiment presents a water level prediction method based on multimodal fusion and the PredFormer dual-stream spatiotemporal network. By acquiring data samples from hydrological and meteorological stations, and preprocessing these two types of time series data, a PredFormer dual-stream spatiotemporal network prediction model framework is constructed. Then, based on efficient low-rank multimodal fusion, the data is fused, and the water level prediction result is output using a multilayer perceptron (MLP). The model demonstrates excellent prediction accuracy and strong generalization performance.

[0185] Example 2

[0186] Based on the same inventive concept, this embodiment discloses a water level prediction device based on the multimodal and spatiotemporal full attention mechanism PredFormer. Please refer to [link to relevant documentation]. Figure 2 ,include:

[0187] Meteorological station determination module 201 is used to determine the corresponding meteorological station based on the hydrological station to be predicted;

[0188] The data collection module 202 is used to collect water level monitoring data of the hydrological station to be predicted and precipitation monitoring data of the corresponding meteorological station;

[0189] Data preprocessing module 203 is used to preprocess the collected water level monitoring data and precipitation monitoring data;

[0190] The model building and training module 204 is used to build a water level prediction model based on the dual-stream spatiotemporal network PredFormer and train the model using preprocessed data. The PredFormer-based water level prediction model includes a feature extraction module, a feature fusion module, and a multilayer perceptron. The feature extraction module has a two-branch structure. The first branch uses a PredFormer model input with a spatiotemporal full attention mechanism, location encoding, and a PredFormer encoder to sequentially establish the spatial relationships between meteorological stations and the temporal relationships within meteorological stations, obtaining the spatiotemporal features of precipitation data from multiple meteorological stations. The second branch uses the PredFormer model input with a spatiotemporal full attention mechanism, location encoding, and a PredFormer encoder to process the temporal relationship of the water level data from the hydrological station to be predicted, obtaining the temporal features of the water level data. The feature fusion module uses an efficient low-rank tensor fusion method with modality-specific factors to fuse the spatiotemporal features of precipitation data from multiple meteorological stations with the temporal features of the water level data at the feature level, obtaining the fused features. The multilayer perceptron outputs the water level prediction result based on the fused features.

[0191] The water level prediction module 205 is used to predict the water level of the hydrological station to be predicted using a pre-trained water level prediction model based on the dual-stream spatiotemporal network PredFormer.

[0192] Since the apparatus described in Embodiment 2 of this invention is the same apparatus used to implement the water level prediction method based on multimodal fusion and the dual-stream spatiotemporal network PredFormer in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this apparatus based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All apparatuses used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.

[0193] Example 3

[0194] Based on the same inventive concept, please refer to Figure 5 The present invention also provides a computer-readable storage medium 300 having a computer program 311 stored thereon, which, when executed by a processor, implements the method described in Embodiment 1.

[0195] Since the computer-readable storage medium described in Embodiment 3 of this invention is the same computer-readable storage medium used in implementing the water level prediction method based on multimodal fusion and the dual-stream spatiotemporal network PredFormer in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this computer-readable storage medium based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All computer-readable storage media used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.

[0196] Example 4

[0197] The present invention also provides a computer device, see below. Figure 6 It includes a memory 401, a processor 402, and a computer program 403 stored in the memory and executable on the processor. When the processor executes the program, it implements the method described in Embodiment 1.

[0198] Since the computer device described in Embodiment 4 of this invention is the same computer device used to implement the water level prediction method based on multimodal fusion and dual-stream spatiotemporal network PredFormer in Embodiment 1 of this invention, those skilled in the art can understand the specific structure and variations of this computer device based on the method described in Embodiment 1 of this invention, and therefore will not be repeated here. All computer devices used in the method of Embodiment 1 of this invention fall within the scope of protection of this invention.

[0199] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0200] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0201] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various modifications and variations to the embodiments of the invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the embodiments of the invention fall within the scope of the claims of the invention and their equivalents, the invention also intends to include these modifications and variations.

Claims

1. A water level prediction method based on multimodal fusion and the dual-stream spatiotemporal network PredFormer, characterized in that, include: Determine the corresponding meteorological stations based on the predicted hydrological stations; Collect water level monitoring data from the hydrological stations to be predicted, as well as precipitation monitoring data from the corresponding meteorological stations; Preprocessing of collected water level monitoring data and precipitation monitoring data; A water level prediction model based on the dual-stream spatiotemporal network PredFormer was constructed, and preprocessed data was used to train the model. The PredFormer-based water level prediction model includes a feature extraction module, a feature fusion module, and a multilayer perceptron. The feature extraction module has a two-branch structure. The first branch uses a PredFormer model with a spatiotemporal full attention mechanism for input embedding, location encoding, and a PredFormer encoder to sequentially establish the spatial relationships between meteorological stations and the temporal relationships within meteorological stations, obtaining the spatiotemporal features of precipitation data from multiple meteorological stations. The second branch uses the PredFormer model with a spatiotemporal full attention mechanism for input embedding, location encoding, and a PredFormer encoder to process the temporal relationship of the water level data from the hydrological station to be predicted, obtaining the temporal features of the water level data. The fusion module uses an efficient low-rank tensor fusion method with modality-specific factors to perform feature-level fusion of the spatiotemporal features of precipitation data from multiple meteorological stations with the temporal features of water level data to obtain fused features; the multilayer perceptron outputs water level prediction results based on the fused features. A pre-trained water level prediction model based on the dual-stream spatiotemporal network PredFormer was used to predict the water level of the hydrological station to be predicted.

2. The water level prediction method based on the multimodal and spatiotemporal full attention mechanism PredFormer as described in claim 1, characterized in that, The corresponding meteorological stations are determined based on the predicted hydrological stations, including: Based on the DEM, the river basin area of ​​the hydrological station to be predicted is delineated, and the meteorological station located upstream of the hydrological station and within the river basin area is selected as the corresponding meteorological station.

3. The water level prediction method based on the multimodal and spatiotemporal full attention mechanism PredFormer as described in claim 1, characterized in that, The collected water level monitoring data and precipitation monitoring data are preprocessed, including: The collected water level monitoring data and precipitation monitoring data were processed for missing values, outliers, and standardization.

4. The water level prediction method based on the multimodal and spatiotemporal full attention mechanism PredFormer as described in claim 1, characterized in that, Before the first branch establishes the spatial relationships between meteorological stations and the temporal relationships within meteorological stations by using the PredFormer model with a spatiotemporal full attention mechanism for input embedding, location encoding, and PredFormer encoder, and obtaining the spatiotemporal features of precipitation data from multiple meteorological stations, the method further includes: Graph convolutional neural networks are used to model the spatial relationships of precipitation data from multiple stations, construct the correlation information between stations, and obtain multi-station structured graph data.

5. The water level prediction method based on the multimodal and spatiotemporal full attention mechanism PredFormer as described in claim 4, characterized in that, The first branch uses a PredFormer model with a spatiotemporal full attention mechanism to input embedding, location encoding, and PredFormer encoder, sequentially establishing spatial relationships between meteorological stations and temporal relationships within meteorological stations, obtaining spatiotemporal features of precipitation data from multiple meteorological stations, including: The PredFormer model, which uses a spatiotemporal full attention mechanism, embeds the multi-site structured graph data as multiple input sub-channels. The data of each sub-channel is flattened into a one-dimensional vector, and then the one-dimensional vector is projected into the hidden dimension through a linear layer to obtain the corresponding tensor. The spatial location and temporal sequence of each station are encoded using absolute location encoding. By combining multiple stacked gated transformer blocks of the PredFormer encoder with a multi-head self-attention mechanism, the spatiotemporal characteristics of precipitation data from multiple meteorological stations are obtained based on location encoding.

6. The water level prediction method based on the multimodal and spatiotemporal full attention mechanism PredFormer as described in claim 1, characterized in that, The second branch processes the temporal relationship of the water level data of the hydrological station to be predicted through the input embedding, position encoding, and PredFormer encoder of the spatiotemporal full attention mechanism PredFormer model, obtaining the temporal features of the water level data, including: The input single-site water level data is processed by embedding the PredFormer model using a spatiotemporal full attention mechanism to obtain the corresponding tensor; The location coding method uses absolute location coding to encode the water level data of a single station. By combining multiple stacked gated transformer blocks of the PredFormer encoder with a multi-head self-attention mechanism, the temporal characteristics of water level data are obtained based on position encoding.

7. The water level prediction method based on the multimodal and spatiotemporal full attention mechanism PredFormer as described in claim 1, characterized in that, The output layer of the efficient low-rank tensor fusion method with modality-specific factors adopts an MLP regression layer.

8. A water level prediction device based on the multimodal and spatiotemporal full attention mechanism PredFormer, characterized in that, include: The meteorological station determination module is used to determine the corresponding meteorological stations based on the hydrological stations to be predicted. The data collection module is used to collect water level monitoring data from the hydrological stations to be predicted and precipitation monitoring data from the corresponding meteorological stations. The data preprocessing module is used to preprocess the collected water level monitoring data and precipitation monitoring data; The model building and training module is used to build a water level prediction model based on the PredFormer spatiotemporal network (a dual-stream network), and to train the model using preprocessed data. The PredFormer-based water level prediction model includes a feature extraction module, a feature fusion module, and a multilayer perceptron. The feature extraction module has a two-branch structure. The first branch uses a PredFormer model input with a spatiotemporal full attention mechanism, location encoding, and a PredFormer encoder to sequentially establish the spatial relationships between meteorological stations and the temporal relationships within meteorological stations, obtaining the spatiotemporal features of precipitation data from multiple meteorological stations. The second branch uses a PredFormer model input with a spatiotemporal full attention mechanism, location encoding, and a PredFormer encoder to process the temporal relationship of the water level data from the hydrological station to be predicted, obtaining the temporal features of the water level data. The fusion module uses an efficient low-rank tensor fusion method with modality-specific factors to perform feature-level fusion of the spatiotemporal features of precipitation data from multiple meteorological stations with the temporal features of water level data to obtain fused features; the multilayer perceptron outputs water level prediction results based on the fused features. The water level prediction module is used to predict the water level of the hydrological station to be predicted using a pre-trained water level prediction model based on the dual-flow spatiotemporal network PredFormer.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements the water level prediction method based on the multimodal and spatiotemporal full attention mechanism PredFormer as described in any one of claims 1 to 7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the water level prediction method based on the multimodal and spatiotemporal full attention mechanism PredFormer as described in any one of claims 1 to 7.

Images