Multi-element water quality monitoring data prediction method based on cross attention fusion
By employing a cross-attention fusion method, the problem of low-quality water quality monitoring data was solved, the robustness and accuracy of the prediction model were improved, and stable prediction and rapid pollution source location were achieved under low-quality data.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HOHAI UNIV
- Filing Date
- 2025-05-23
- Publication Date
- 2026-06-02
Smart Images
Figure CN120598101B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of water quality monitoring data prediction technology, and relates to a multivariate water quality monitoring data prediction method based on cross-attention fusion, which is used to improve the prediction accuracy and robustness of water quality monitoring data, especially when dealing with low-quality multivariate water quality monitoring data. Background Technology
[0002] Real-world spatiotemporal data generally faces the severe challenge of low quality, mainly manifested in data gaps, noise interference, and asynchronous sampling. Taking multivariate water quality monitoring data from a single monitoring station as an example, water quality indicators (such as dissolved oxygen, chlorophyll concentration, and temperature) may exhibit overall periodic or trend-like changes, but the dynamic evolution patterns in local time periods may fluctuate significantly due to external interference. For instance, under the influence of sudden pollution events or changes in hydrological conditions (such as heavy rainfall or algal blooms), different water quality indicators at the same station may exhibit asynchronous and abnormal fluctuations: dissolved oxygen concentrations show increased diurnal differences due to eutrophication, while chlorophyll concentrations show a sudden increase due to algal proliferation, reflecting the complex characteristics of the dynamic correlation between parameters. However, low-quality data severely distorts this temporal evolution pattern: data gaps (such as equipment power outages or uneven sampling intervals) disrupt the continuity of the time dimension, making it difficult to accurately identify key information such as the start time and duration of pollution events; noise interference (such as sensor drift errors) may obscure the true water quality indicator change curves, leading to distorted temporal fluctuation patterns. This data deficiency makes it difficult for traditional time-series forecasting models to capture the true dynamic coupling relationship between parameters (such as the nonlinear synergistic change between dissolved oxygen and chlorophyll concentration), which in turn leads to a significant reduction in the reliability and interpretability of the forecast results.
[0003] Early spatiotemporal prediction research primarily focused on high-quality data scenarios or localized optimization for single types of data defects, neglecting the complexity and diversity of low-quality data. For example, some models capture dynamic spatial correlations through time-aware graph structure learning methods (such as time-aware graph convolutional recurrent networks), but their regularization constraints are easily affected by noise, causing the graph structure to deviate from the true spatial relationships. Another type of model (such as relational temporal graph convolutional networks) introduces a Gaussian distribution to represent the hidden states and suppresses abnormal nodes through variance attention mechanisms, but its assumption that node states follow a static distribution makes it difficult to adapt to dynamic mutations caused by sudden natural disasters leading to pollutants entering rivers in water quality monitoring data.
[0004] More importantly, existing methods have significant limitations in the fusion of spatiotemporal features. Most methods employ linear splicing or weighted averaging strategies to fuse features from the temporal and spatial dimensions, neglecting the deep coupling between them. For example, long-term trends in the temporal dimension may imply multi-parameter co-evolutionary patterns (such as the inverse correlation between continuously rising water temperature and dissolved oxygen concentration at water quality monitoring stations), and simple fusion methods struggle to explicitly model such cross-dimensional interactions. Furthermore, the uneven distribution of noise in spatiotemporal features in low-quality data (such as dense spatial noise in a certain time period) can lead to excessive propagation of noise features by traditional fusion strategies, further amplifying prediction bias.
[0005] To address the aforementioned issues, while existing research has proposed some improvement schemes, it has not yet overcome three major bottlenecks: First, the lack of collaborative processing of multiple defects. In real-world water quality monitoring data, problems such as noise, missing data, and asynchronicity often coexist, but existing methods are only designed for single defects and lack a unified framework. Second, the imbalance between dynamic modeling and noise suppression. Most water quality prediction models sacrifice noise robustness to improve dynamic adaptability, or simplify the complexity of dynamic interactions to suppress noise. Third, the superficiality of spatiotemporal interaction modeling. The simple fusion of temporal and spatial features is insufficient to reveal potential cross-dimensional correlations, resulting in limited expressive power of water quality prediction models for complex dynamic systems. Summary of the Invention
[0006] Purpose of the invention: To address the problems and shortcomings of existing technologies, this invention provides a multivariate water quality monitoring data prediction method based on cross-attention fusion that offers stable prediction results and good practicality.
[0007] Technical solution: A multivariate water quality monitoring data prediction method based on cross-attention fusion, comprising the following steps:
[0008] Step 1: Collect water quality monitoring data from monitoring stations and generate input data;
[0009] Step 2: Construct a multivariate water quality monitoring data prediction model;
[0010] Step 3: Train a multivariate water quality monitoring data prediction model;
[0011] Step 4: Calculate the accuracy of the model prediction. If the accuracy exceeds the preset threshold, proceed to step 5; otherwise, return to step 3.
[0012] Step 5: Input the generated data into the trained prediction model to obtain the predicted values of water quality monitoring data for the monitoring stations.
[0013] Preferably, in step 1, the water quality monitoring data of the monitoring station is: all historical water quality monitoring data of a set monitoring station, including six water quality indicators: temperature, conductivity, pH value, chlorophyll concentration, turbidity and dissolved oxygen, which is multi-dimensional water quality monitoring data.
[0014] Preferably, in step 2, the multivariate water quality monitoring data prediction model includes a time-dimensional modeling module, a spatial-dimensional modeling module, a time-guided cross-attention fusion module, and an output layer. The prediction model takes water quality monitoring data from monitoring stations as input, and through hierarchical feature extraction and deep fusion, ultimately outputs a high-precision prediction result. The time-dimensional modeling module is responsible for capturing the local periodicity and long-term trend of the time series of multivariate water quality monitoring data; the spatial-dimensional modeling module dynamically mines the spatial dependencies between water quality indicators; the cross-attention fusion module explicitly associates spatiotemporal features, avoiding information loss in traditional fusion methods; and the output layer combines a multilayer perceptron and loss function optimization to achieve end-to-end prediction tasks.
[0015] More preferably, for the time-dimensional modeling module: the input of this module is historical water quality monitoring data from monitoring stations containing noise and missing values. The specific implementation steps of the time-dimensional modeling module are as follows: First, Gaussian noise (mean 0, standard deviation 0.1) is injected into the raw data to simulate sensor noise or transmission interference, enhancing adaptability to data disturbances; Second, transformable patching technology is used to align asynchronously sampled time series into fixed-length subsequences (e.g., 6 hours per patch) to solve the problem of sampling frequency differences among monitoring stations; Third, dynamic convolutional networks (TCN) and self-attention mechanisms are further combined to extract local periodic patterns (e.g., hourly pH fluctuations caused by industrial emissions) through adaptive filters, while the self-attention mechanism is responsible for mining long-term dependencies across patches (e.g., the cumulative impact of continuous rainfall on eutrophication). The final output time features contain both short-term details and global trends, providing multi-level time-series information for subsequent modeling.
[0016] More preferably, for the spatial dimension modeling module: the input of this module is a dynamic graph structure constructed from the collected water quality monitoring data, where the nodes of the dynamic graph are water quality indicators and the edges are the correlation weights between the water quality indicators. The specific implementation steps of the spatial dimension modeling module are as follows: First, the module dynamically generates an adjacency matrix through adaptive graph structure learning, and then calculates the correlation weights between water quality indicators based on the learnable node embeddings, and marks missing values (such as data missing due to equipment failure at monitoring stations) using a mask matrix; Second, a gated graph attention network (Gated GAT) is introduced to optimize feature extraction, and the gating mechanism generates feature weights through the Sigmoid function to filter instantaneous noise from the sensor (such as abnormal pH jumps); Third, the graph attention network calculates the attention coefficients between nodes based on the dynamic adjacency matrix, strengthening the contribution of key water quality indicators (such as turbidity and dissolved oxygen in pollution events) dynamically identified based on correlation weights, gating weights, and attention coefficients, and weakening the influence of irrelevant or high-noise nodes. The spatial dimension modeling module focuses on capturing the dynamic correlation between different water quality indicators within the monitoring station and suppressing the interference of low-quality data. Finally, it outputs the denoised dynamic spatial dependency feature representation as spatial features.
[0017] The key water quality indicators are determined as follows:
[0018] ① In the first step of the spatial dimension modeling module, the correlation weights A between water quality indicators are calculated. p,ij , which represents the correlation strength between water quality indicators i and j within the time window p. If A p,ij If the value is significantly higher than the weights of other water quality indicator pairs (such as the strong coupling between turbidity and dissolved oxygen in pollution events), then water quality indicator i or j is considered the key water quality indicator for the current window; ② The gating weight gf represents the degree of contribution of a certain water quality indicator to the model results. If the gating weight gf of a certain water quality indicator is consistently higher than that of other indicators, it indicates that its original characteristics are more reliable and it is regarded as a key water quality indicator by the model;
[0019] ③ Calculate the attention coefficient e between nodes using a multi-head attention mechanism. ij Its value reflects the degree of dependence of water quality index i on water quality index j. If the attention coefficient e of water quality index i... ij A water quality indicator is identified as a key water quality indicator if it is highly valued by multiple attention heads or has a strong influence on multiple neighboring nodes (water quality indicators).
[0020] More preferably, for the time-guided cross-attention fusion module: the input to this module is temporal features (local periodicity and long-term trend) and spatial features (correlation of dynamic water quality indicators). The specific implementation steps of the cross-attention fusion module are as follows: First, using temporal features as the query and spatial features as the key and value, multi-head attention is used to calculate cross-dimensional interaction weights; second, residual connections are introduced during the fusion process, adding the cross-attention output to the original temporal features, preserving key temporal information while correcting for spatial noise interference. This module explicitly associates the semantic patterns of time and space dimensions through the cross-attention mechanism, ultimately outputting a deeply fused spatiotemporal semantic feature representation, providing a high-information-density input for the prediction task.
[0021] More preferably, for the output layer: the input of this module is the spatiotemporal semantic feature representation of water quality monitoring data and the query conditions for the prediction time step. The specific implementation steps of the output layer are as follows: First, the spatiotemporal semantic features are mapped to the prediction space through a multilayer perceptron (MLP); Second, an iterative prediction strategy is adopted to gradually generate multi-step results, that is, the influence weight of historical periods on the current prediction step is evaluated through a self-attention mechanism (such as the reference value of the pollution peak of the past 3 days for the prediction of the next 6 hours); Third, the predicted values are generated by combining a fully connected network, and finally the predicted values of water quality monitoring data for the future time step (such as turbidity and dissolved oxygen concentration for the next 12 hours) are output.
[0022] More preferably, step 3 specifically comprises:
[0023] The overall model is trained, and during training, the model is fine-tuned using the backpropagation algorithm based on the results of the loss function.
[0024] More preferably, the loss function is divided into a loss function L in the time dimension modeling module. MSE and the loss function L in the output layer pre The former L MSE Specifically:
[0025]
[0026] Where i represents the index of the historical time step, L n Let n be the number of observed values for the nth water quality indicator. and These correspond to the actual observed value of the nth water quality index at the ith time step and its reconstructed value after being processed by a transformable patch, respectively.
[0027] The latter L pre Specifically:
[0028]
[0029] Where j represents the index of the future time step, Q n This represents the number of query values for the nth water quality indicator. and These correspond to the actual observed value and the model prediction value of the nth water quality index at the jth time step, respectively.
[0030] By minimizing the mean squared error of all variables and queries, and combining this with a loss function modeled over the time dimension, a model capable of accurately predicting spatiotemporal data is trained. The specific loss function of the entire model is as follows:
[0031] L = L MSE +L pre .
[0032] A multivariate water quality monitoring data prediction system based on cross-attention fusion includes:
[0033] Data acquisition unit: Collects water quality monitoring data from monitoring stations and generates input data;
[0034] Prediction model building unit: Constructing a prediction model for multivariate water quality monitoring data;
[0035] Training unit: Training a multivariate water quality monitoring data prediction model;
[0036] Evaluation unit: Calculate the accuracy of the model prediction. If the accuracy exceeds the preset threshold, proceed to step 5; otherwise, return to step 3.
[0037] Prediction Unit: Inputs the generated data into the trained prediction model to obtain predicted values of water quality monitoring data at the monitoring stations;
[0038] The multivariate water quality monitoring data prediction model includes a time-dimensional modeling module, a spatial-dimensional modeling module, a time-guided cross-attention fusion module, and an output layer. The prediction model takes water quality monitoring data from monitoring stations as input, extracts features layer by layer, and performs deep fusion to output high-precision prediction results. The time-dimensional modeling module is responsible for capturing the local periodicity and long-term trend of the time series of multivariate water quality monitoring data. The spatial-dimensional modeling module dynamically mines the spatial dependencies between water quality indicators. The cross-attention fusion module explicitly associates spatiotemporal features. The output layer combines a multilayer perceptron and loss function optimization to achieve end-to-end prediction.
[0039] The system implementation process and method are the same and will not be described again.
[0040] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the multivariate water quality monitoring data prediction method based on cross-attention fusion as described above.
[0041] A computer-readable storage medium storing a computer program that performs the multivariate water quality monitoring data prediction method based on cross-attention fusion as described above.
[0042] Compared with the prior art, the present invention has the following beneficial effects:
[0043] I. Stable Prediction Results: In real-world scenarios where data is missing or of insufficient quality, the water quality monitoring data prediction method in this invention employs a dual-protection mechanism built upon dynamic graph structure learning and transformable patching technology. This mechanism automatically completes the spatiotemporal dependencies across indicators when some data is missing, while accurately capturing periodic patterns even under data noise interference. This collaborative mechanism significantly improves the feature utilization rate of low-quality data, ensuring the stability of prediction results even when monitoring system data is incomplete.
[0044] Second, it has high practicality: The water quality monitoring data prediction method in this invention can directly predict all water quality monitoring data from monitoring stations, with fast data processing speed, wide coverage, and high practicality. This enables decision-makers to quickly locate pollution sources and assess the effectiveness of emergency measures in the event of a sudden pollution incident, providing a scientific basis for environmental governance. Attached Figure Description
[0045] Figure 1 This is a flowchart of the water quality monitoring data prediction method in this invention.
[0046] Figure 2 This is a diagram illustrating the overall structure of the multivariate water quality monitoring data prediction method based on cross-attention fusion in this embodiment of the invention. Detailed Implementation
[0047] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.
[0048] The multivariate water quality monitoring data prediction method based on cross-attention fusion aims to address the problems of high missing rates, noise interference, and complex spatiotemporal dynamic correlations in real-world water quality monitoring data. It improves the robustness and accuracy of the prediction model by deeply fusing feature representations of both temporal and spatial dimensions, and is further optimized to suit the characteristics of water quality monitoring data. A specific implementation method is provided below:
[0049] A multivariate water quality monitoring data prediction method based on cross-attention fusion, the process of which is as follows: Figure 1 As shown, it includes:
[0050] Step 1: Generate input data;
[0051] Water quality monitoring data is collected from pre-set monitoring stations, and the observation data from these stations is represented as a multivariate time series. in, Let i represent a binary array containing information about the observed values of the nth water quality index, where i represents the index of the historical time step. and L represents the timestamp and value of the observation at the i-th time step for the n-th water quality indicator, respectively. n This indicates the number of observations included in the nth water quality indicator. Water quality monitoring data specifically refers to all historical water quality monitoring data from a particular monitoring station, including six water quality indicators: temperature, conductivity, pH, chlorophyll concentration, turbidity, and dissolved oxygen. The historical water quality monitoring data is divided into time steps at 10-minute intervals.
[0052] Step 2: Construct a multivariate water quality monitoring data prediction model;
[0053] The specific structure of the model is as follows Figure 2 As shown, the core architecture of this model consists of time-dimensional modeling, spatial-dimensional modeling, spatiotemporal cross-attention fusion, and an output layer, achieving high-precision prediction through multi-level feature extraction and interaction.
[0054] Temporal modeling aims to capture local periodic patterns and long-term trends in water quality monitoring data, while enhancing robustness to missing values and noise. It first performs noise injection and patch alignment, randomly adding Gaussian noise with a mean of 0 and a standard deviation of 0.1 to the input data X at a preset noise rate v, generating noisy data. Subsequently, a transformable patching technique was used to divide the time series into P equal-length patch blocks, ensuring data alignment in temporal resolution. The timestamp of each patch block was embedded into a multi-dimensional vector using a Time2Vec encoder, containing linear terms (capturing non-periodic changes) and sinusoidal terms (capturing periodic features), with the specific formula as follows:
[0055]
[0056] Where p is the patch index, t represents the timestamp, φ(·) is a piecewise function, and d p D represents the embedding dimension of the p-th patch. t It is the total embedding dimension, i.e., d p The maximum value of ω 0p α 0p ω dp and α dp These are all learnable parameters that determine the frequency and phase of the sine wave, thus enabling the encoding of different periodic patterns.
[0057] Subsequently, multi-scale temporal feature encoding is performed, employing a joint architecture of adaptive weighted TCN (Temporal Convolutional Network) and Transformer to extract temporal features. The adaptive weighted TCN captures local periodic patterns by dynamically generating filters that match the input length. The filter calculation formula is shown below:
[0058]
[0059] Where d is the index of the feature map, It is the filter derived from the d-th feature map, D in L represents the dimension of the input. p For patch length, F d z represents a meta-filter instantiated from a learnable neural network. i and z j Let be the features within the patch block at time steps i and j. Using the filter calculation formula to derive d-1 filters, applying TCN yields the latent embeddings for each patch. The specific calculation formula is as follows:
[0060]
[0061] in, This is the convolution output of the p-th patch. Let D be the feature matrix of the patch block, and D be the latent representation dimension (i.e., feature dimension) of the input data in the feature space. Subsequently, to reduce the occurrence of patches with no observations, the patch mask (identifying missing data states) is incorporated into the latent embedding of each transformable patch. The specific operation is performed according to the following formula:
[0062]
[0063] Where p is the index of the patch, m p Let m be the mask for the p-th patch, || denotes the concatenation operation, and m is the mask when there are observations in the transformable patch. p =1, otherwise, m p =0. Combining the embedding representations of the P patch blocks, the latent representation of each time series output by the adaptive weighted TCN module is: Finally, the TCN output is concatenated with the patch mask and input into the Transformer. Long-term dependencies are captured through a multi-head self-attention mechanism, the specific formula of which is as follows:
[0064]
[0065] in, This represents the output of the adaptive weighted TCN module for water quality index n, where h represents the index of the attention head, H represents the number of attention heads, || represents concatenating the outputs of all attention heads, and Softmax is a non-linear activation function. and Let the query matrix, key matrix, and value matrix represent the query matrix, key matrix, and value matrix for water quality index n, respectively. Their calculation formulas are shown below:
[0066]
[0067] in, It is a learnable weight matrix. This represents the input for water quality index n, which incorporates the latent representation and location encoding.
[0068] Combining the output representations of N time series, the final output of the Transformer (i.e., the latent representation in the time dimension) is: Where P is the number of patches and D is the feature dimension. For the latent representation of all variables H in the same patch p, the output representation can be integrated as follows:
[0069] Spatial dimension modeling aims to reveal the dynamic correlations among different water quality indicators in the water quality monitoring data of monitoring stations, specifically including two stages: adaptive graph structure learning and gated graph attention. In the adaptive graph structure learning stage, the generated multivariate water quality time series is first divided into P transformable patches according to a time window. Then, for each patch, a two-dimensional mask matrix is dynamically calculated. Where T p The patch time step is defined, and if data for a certain indicator is missing at a specific time, the corresponding position in the mask matrix is set to 0; otherwise, it is set to 1. Then, the learnable source node embedding E1 and target node embedding E2 are initialized, and the dynamic adjacency matrix between different water quality indicators within the patch is calculated. The specific calculation formula is shown below:
[0070]
[0071] Where p is the index of the patch, E p,1 For the source node embedding of patch p, E p,2 For the target node embedding of patch p, they are all node embedding matrices with learnable parameters, and different nodes correspond to different water quality indicators, M p The mask matrix is denoted by ⊙, where ⊙ represents element-wise multiplication. Softmax and ReLU are both non-linear activation functions. The final result A is obtained. p It can reveal the dynamic interaction weights of different water quality indicators within the current time window.
[0072] In the gating graph attention phase, to suppress noise interference, the initial features of the nodes are... The Sigmoid activation function is applied to generate a feature-gated gate function (gf), which is then weighted to obtain the denoised features. Their specific formulas are as follows:
[0073]
[0074] Where σ is the Sigmoid activation function, i is the node index, and W f and b f Let be the learnable parameter matrix and bias, and ⊙ denotes element-wise multiplication.
[0075] Based on the dynamic adjacency matrix A p The attention coefficients between nodes are calculated using a graph attention network (GAT), and the specific formula is shown below:
[0076]
[0077] Where i and j are the indices of the nodes, e ij Represents node v i and v j The attention coefficients between them, LeakyReLU is a non-linear activation function, and 'a' is a learnable attention mechanism parameter. It is a learnable weight matrix used to transform node features. and Representing node v respectively i and v j The eigenvectors of A, || denote the vector concatenation operation, A p,ij Represents node v i and v j The dynamic adjacency matrix between them.
[0078] Then the Softmax function is used on node v. i Normalize all neighboring nodes to obtain the attention coefficient 'a' between neighboring nodes. ij Finally, spatial features are output by aggregating neighbor information through multi-head attention. Where P is the number of patches and D is the feature dimension.
[0079] In the time-guided cross-attention fusion part, the model uses time features H T To query Q, using spatial features H S Given key K and value V, feature interaction is achieved through multi-head cross-attention, and the specific formula is shown below:
[0080] Q = W Q H TK = W k H S V = W V H S ,
[0081]
[0082]
[0083] Among them, W Q W k , It is a learnable weight matrix, where D is the feature dimension, i is the index of the attention head, and Head is the number of heads. i H represents the output of the i-th attention head. c For the number of attention heads, D h =D / H c For a single-head dimension, Softmax is a non-linear activation function, MultiHead(Q,K,V) is the fused output representation of multi-head cross-attention, and Concat is the connection function.
[0084] Subsequently, the fusion result is added to the temporal features through residual connections to generate a spatiotemporal joint representation H. ST The specific formula is as follows:
[0085] H ST =W O MultiHead(Q,K,V)+H T
[0086] in, This is the weight matrix of the linear layer, where P is the number of patches and D is the feature dimension. The final generated spatiotemporal fusion feature representation... By integrating information from both time and space dimensions, it provides richer and more multi-layered input data for subsequent prediction tasks.
[0087] At the output layer, MLP is further used to implement spatiotemporal prediction tasks to obtain predicted values from water quality monitoring data. Its spatiotemporal fusion feature representation H for a given nth water quality index n ∈H ST And a set of predictive queries An MLP projection layer is used to generate the predictions for these queries, where j represents the index at a future time step. This represents the query for the nth water quality indicator at the j-th time step, Q. n This represents the number of queries for the nth water quality indicator. Specifically, for each query... Compare it with the corresponding feature representation H nThese are combined and then processed through an MLP function. The specific formula for this process is shown below:
[0088]
[0089] Where φ(·) is a non-linear function used to transform the query, H n Let be the spatiotemporal fusion feature representation of the nth water quality indicator, where || denotes the vector concatenation operation, and j represents the index of the future time step. For query The query indicates that...
[0090] Step 3: Train the prediction model built in Step 2;
[0091] For the constructed model, training set, validation set and test set are built. The model is initialized and trained using the training set data. The model parameters are adjusted using the validation set and evaluation metrics. The model performance is tested using the test set. It is worth mentioning that we train the model once for each water quality indicator in the water quality monitoring data and make predictions.
[0092] In this embodiment, the proportions of the training dataset, validation dataset, and test dataset are 70%, 10%, and 20%, respectively.
[0093] The model is trained using a loss function and then fine-tuned to complete the training. Finally, the predicted water quality monitoring data for the monitoring station in the next T time steps are output.
[0094] Loss functions are divided into loss functions L in the time dimension modeling module. MSE and the loss function L in the output layer pre The former L MSE Specifically:
[0095]
[0096] Where i represents the index of the historical time step, L n Let n be the number of observed values for the nth water quality indicator. and These correspond to the actual observed value of the nth water quality index at the ith time step and its reconstructed value after being processed by a transformable patch, respectively.
[0097] The latter L pre Specifically:
[0098]
[0099] Where j represents the index of the future time step, Q n This represents the number of query values for the nth water quality indicator. and These correspond to the actual observed value and the model predicted value of the nth water quality index at the jth time step, respectively.
[0100] By minimizing the mean squared error of all variables and queries, and combining this with a loss function modeled over the time dimension, a model capable of accurately predicting spatiotemporal data is trained. The specific loss function of the entire model is as follows:
[0101] L = L MSE +L pre
[0102] Step 4: Calculate the accuracy of the model prediction. If the accuracy exceeds the preset threshold, proceed to step 5; otherwise, return to step 3.
[0103] Step 5: Predetermine one of the prediction targets, input the water quality monitoring data of the monitoring station into the trained prediction model, and obtain the predicted value of the water quality monitoring data of the monitoring station in the next T time steps.
[0104] Obviously, those skilled in the art should understand that the steps of the multivariate water quality monitoring data prediction method based on cross-attention fusion described in the above embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by the computing device. Furthermore, in some cases, the steps shown or described can be performed in a different order than presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.
Claims
1. A multivariate water quality monitoring data prediction method based on cross-attention fusion, characterized in that, Includes the following steps: Step 1: Collect water quality monitoring data from monitoring stations and generate input data; Step 2: Construct a multivariate water quality monitoring data prediction model; Step 3: Train a multivariate water quality monitoring data prediction model; Step 4: Calculate the accuracy of the model prediction. If the accuracy exceeds the preset threshold, proceed to step 5; otherwise, return to step 3. Step 5: Input the generated data into the trained prediction model to obtain the predicted values of water quality monitoring data for the monitoring stations; In step 2, the multivariate water quality monitoring data prediction model includes a time dimension modeling module, a spatial dimension modeling module, a time-guided cross-attention fusion module, and an output layer. The time dimension modeling module is responsible for capturing the local periodicity and long-term trend of the time series of multivariate water quality monitoring data: First, Gaussian noise is injected into historical water quality monitoring data to simulate sensor noise or transmission interference. Second, based on the results of the first step, the time series of asynchronously sampled water quality monitoring data is aligned into fixed-length subsequences using a transformable patching technique. Third, based on the results of the second step, a dynamic convolutional network and a self-attention mechanism are combined to extract local periodic patterns through an adaptive filter, while the self-attention mechanism is responsible for mining long-term dependencies across patches. The spatial dimension modeling module dynamically mines the spatial dependencies between water quality indicators: the input of this module is a dynamic graph structure constructed from the collected water quality monitoring data, where the nodes of the dynamic graph are water quality indicators and the edges are the association weights between the water quality indicators. The specific implementation steps of the spatial dimension modeling module are as follows: First, the module learns and dynamically generates an adjacency matrix through an adaptive graph structure, then calculates the association weights between water quality indicators based on the learnable node embeddings, and marks missing values with a mask matrix. Second, a gated graph attention network is introduced to optimize feature extraction, while the gating mechanism generates feature weights through the Sigmoid function to filter instantaneous noise from the sensor. Third, the graph attention network calculates the attention coefficients between nodes based on the dynamic adjacency matrix. The spatial dimension modeling module focuses on capturing the dynamic correlation between different water quality indicators within the monitoring station and suppressing the interference of low-quality data. Finally, it outputs the denoised dynamic spatial dependency feature representation as spatial features. The cross-attention fusion module explicitly associates spatiotemporal features; the output layer combines multilayer perceptron and loss function optimization to achieve end-to-end prediction tasks. The cross-attention fusion module takes as input time features containing local periodicity and long-term trends, and spatial features associated with dynamic water quality indicators. The specific implementation steps are as follows: First, using time features as queries and spatial features as keys and values, multi-head attention is used to calculate cross-dimensional interaction weights; second, residual connections are introduced during the fusion process, adding the cross-attention output to the original time features, preserving key temporal information while correcting for spatial noise interference. This module explicitly associates the semantic patterns of time and space dimensions through the cross-attention mechanism, ultimately outputting a deeply fused spatiotemporal semantic feature representation.
2. The multivariate water quality monitoring data prediction method based on cross-attention fusion according to claim 1, characterized in that, In step 1, the water quality monitoring data of the monitoring station is: all historical water quality monitoring data of a set monitoring station, including six water quality indicators: temperature, conductivity, pH value, chlorophyll concentration, turbidity and dissolved oxygen, which are multi-dimensional water quality monitoring data.
3. The multivariate water quality monitoring data prediction method based on cross-attention fusion according to claim 1, characterized in that, For the output layer: the input of this module is the spatiotemporal semantic feature representation of water quality monitoring data and the query conditions for the prediction time step; the specific implementation steps of the output layer are as follows: First, the spatiotemporal semantic features are mapped to the prediction space through a multilayer perceptron; Second, an iterative prediction strategy is adopted to generate multi-step results step by step, that is, the influence weight of historical periods on the current prediction step is evaluated through a self-attention mechanism; Third, the prediction value is generated by combining a fully connected network, and finally the predicted value of water quality monitoring data for the future time step is output.
4. The multivariate water quality monitoring data prediction method based on cross-attention fusion according to claim 1, characterized in that, Step 3 specifically includes: The overall model is trained, and during training, the model is fine-tuned using the backpropagation algorithm based on the results of the loss function; the loss function is divided into the loss function in the time dimension modeling module. and the loss function in the output layer ;former Specifically: in, An index representing a historical time step. For the first The number of observed values for each water quality indicator. and Corresponding to the first The water quality indicators are in the first The actual observations at each time step and their reconstructed values after being processed by transformable patches; the latter Specifically: in, Indicates an index for a future time step. Indicates the first The number of query values for each water quality indicator. and Corresponding to the first The water quality indicators are in the first The actual observations and model predictions at each time step; By minimizing the mean squared error of all variables and queries, and combining this with a loss function modeled over the time dimension, a model capable of accurately predicting spatiotemporal data is trained. The specific loss function of the entire model is as follows: 。 5. A system for predicting multivariate water quality monitoring data based on cross-attention fusion as described in claim 1, characterized in that, include: Data acquisition unit: Collects water quality monitoring data from monitoring stations and generates input data; Prediction model building unit: Constructing a prediction model for multivariate water quality monitoring data; Training unit: Training a multivariate water quality monitoring data prediction model; Evaluation unit: Calculate the accuracy of the model prediction. If the accuracy exceeds the preset threshold, proceed to step 5; otherwise, return to step 3. Prediction Unit: Inputs the generated data into the trained prediction model to obtain predicted values of water quality monitoring data at the monitoring stations; The multivariate water quality monitoring data prediction model includes a time-dimensional modeling module, a spatial-dimensional modeling module, a time-guided cross-attention fusion module, and an output layer. The prediction model takes water quality monitoring data from monitoring stations as input, extracts features layer by layer, and performs deep fusion to output high-precision prediction results. The time-dimensional modeling module is responsible for capturing the local periodicity and long-term trend of the time series of multivariate water quality monitoring data. The spatial-dimensional modeling module dynamically mines the spatial dependencies between water quality indicators. The cross-attention fusion module explicitly associates spatiotemporal features. The output layer combines a multilayer perceptron and loss function optimization to achieve end-to-end prediction.
6. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the multivariate water quality monitoring data prediction method based on cross-attention fusion as described in any one of claims 1-4.
7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that performs the multivariate water quality monitoring data prediction method based on cross-attention fusion as described in any one of claims 1-4.