Multi-site water quality prediction method based on AGFormer-GPT2

Abstract

The invention relates to a multi-site water quality prediction method based on AGFormer-GPT2, which is applied to prediction of multi-site water quality monitoring time sequence data. The method comprises the following steps: firstly, preprocessing and windowing a multi-site water quality sequence, constructing a pseudo vocabulary library based on GPT-2 word vector dimensionality reduction, and executing dynamic matching and fusion between pseudo vocabularies and a time sequence Patch by adopting a Top-k sparse attention mechanism to form a semantic enhanced time slice representation; the method comprises the following steps of: introducing an anchor graph Transform (AGFormer) to carry out modeling on multi-site spatial dependence, and superimposing geographic offset description proximity based on longitude and latitude, so as to improve structured expression; modeling is carried out on the context in combination with GPT2, long-term dependence and global dynamics are captured, and a double-branch space-time fusion framework of space-time / time-space is formed with AGFormer; and finally, a cascade deep forest (DFEC) is used as a posterior residual learning and output module, error compensation is carried out on the main prediction, and multi-step prediction is generated. According to the method, complex interaction between water quality variables and sites can be described more accurately, trend prediction precision is improved, and the method has good adaptability, robustness and popularization and application value.

Description

Technical Field

[0001] This invention belongs to the field of water quality monitoring technology, and relates to artificial intelligence technology and water eutrophication detection technology. Specifically, it relates to a multi-site water quality prediction method based on AGFormer-GPT2. Background Technology

[0002] In recent years, water pollution has become a major environmental problem of widespread global concern, with eutrophication being particularly prominent. Eutrophication is characterized by excessively high concentrations of nutrients such as nitrogen and phosphorus, disrupting the balance of aquatic ecosystems, inducing abnormal proliferation of algae and some aquatic organisms, and ultimately leading to algal blooms, seriously threatening drinking water safety and human production and lives. According to statistics from the United Nations Environment Programme, the area threatened by eutrophication in global rivers and lakes is expanding year by year. Due to the difficulty and high cost of remediation, timely prediction and proactive intervention in eutrophication have become crucial aspects of current water environment management. Eutrophication is driven by a combination of multiple factors, mainly involving the concentrations of nutrients such as nitrogen and phosphorus, chemical indicators such as dissolved oxygen and permanganate index, and environmental factors such as water temperature. Human activities bring exogenous nutrient input, stimulating rapid algal proliferation and creating a vicious cycle through self-reinforcing mechanisms; simultaneously, seasonal and climate changes cause significant time-series fluctuations in water quality indicators. Therefore, it is urgent to construct a long-term, stable, and operational water quality monitoring and prediction system to support scientific regulation and ecological security. There are currently two main technical approaches to predictive modeling of eutrophication: mechanism-driven models and data-driven models.

[0003] Mechanism-driven models, based on ecological-physical-chemical processes, characterize algal growth and material transformation from nutrient loading to ecological dynamics. They can explain internal and external regulatory mechanisms and key processes, and possess a certain degree of interpretability and scenario analysis capability. However, their application typically relies on a large number of field parameters and boundary conditions, resulting in high calibration and validation costs and limited portability to different water bodies. In complex, dynamic, and cross-regional scenarios, the model structure and parameters often require targeted reconstruction and parameter tuning, limiting their generalization ability.

[0004] Data-driven models, centered on historical observation data, uncover the potential relationships between multiple factors. They encompass both mathematical and statistical methods such as correlation analysis and regression, as well as machine learning and deep learning methods capable of handling strong nonlinearity and high-dimensional coupling. Compared to mechanistic models, data-driven models have weaker prior dependence, higher modeling and iteration efficiency, and are suitable for rapid deployment to multi-site, multi-indicator real-world scenarios. However, when faced with real-world conditions such as long-term time-series dependencies, spatial heterogeneity, and the coexistence of missing data and anomalies, these models are prone to problems such as insufficient characterization of spatiotemporal relationships, limited cross-site generalization, and error accumulation.

[0005] Currently, water quality prediction based on deep learning faces the following main problems:

[0006] (1) In current deep learning-based methods for predicting eutrophication of water bodies, an increasing number of studies are attempting to introduce language models for semantic enhancement to improve the model's ability to understand complex environmental factors. However, existing methods generally employ static or simple attention mechanisms to fuse semantic and temporal data, failing to effectively achieve dynamic selection and matching of information. This fusion strategy has many limitations in practical applications: on the one hand, semantic information often contains a large amount of redundant content, and direct fusion will interfere with the model's temporal learning; on the other hand, the lack of a mechanism to dynamically select the most relevant prompt words to the current prediction target leads to unstable semantic enhancement effects and low fusion accuracy. Therefore, it is necessary to construct a selective and adaptive fusion strategy to achieve deep synergy between high-quality semantic information and temporal features, providing more accurate support for eutrophication prediction.

[0007] (2) Current eutrophication prediction methods often rely on simple topology or treat inter-site relationships as static prior inputs when processing multi-site water quality data, making it difficult to uncover the potential complex and deep interactions between sites. This strategy remains at the surface level of "embedded topology," lacking information propagation and coupling modeling at the time-slice granularity, and failing to capture the mutual influence and transmission chains between cross-site time slices. Due to the failure to explicitly characterize co-evolution and multi-scale dependencies, the model's prediction accuracy, robustness, and generalization ability are limited in complex watershed scenarios. Therefore, it is necessary to introduce a spatial modeling framework with deep site structure perception and adaptive capabilities to achieve dynamic fusion and interactive learning of cross-site information at fine-grained time slices. Summary of the Invention

[0008] This invention addresses key issues in existing water quality prediction methods when processing complex time-series data from multiple sites, proposing a multi-site water quality prediction method based on AGFormer-GPT2. It solves the problem of weak modeling ability for complex spatiotemporal patterns in existing prediction methods, achieving accurate modeling of complex relationships and long-term dependencies between sites in multi-site time-series data, further improving the prediction accuracy and practicality of the prediction model.

[0009] This invention provides a multi-site water quality prediction method based on AGFormer-GPT2, comprising the following steps:

[0010] Step 1: Data Preprocessing and Windowing

[0011] This step first aligns the time-series water quality data from multiple stations by time, and then uses a multilayer perceptron (MLP) machine learning method to fill in missing values. Next, it slides slices through a fixed window and maps them to time slice vectors, preserving station information such as latitude and longitude. Finally, each station is divided into training, validation, and test sets according to time order.

[0012] Step 2: Lexical-temporal feature alignment based on sparse attention

[0013] This step establishes a sparse attention-based vocabulary and temporal alignment module. A controlled-size vocabulary is constructed based on word vectors from the pre-trained language model GPT2. The relevance of each temporal patch to the dimensionality-reduced vocabulary is calculated, retaining only the most relevant words and performing weighted fusion to obtain a semantically enhanced patch representation. The aim is to establish an alignment channel between "numerical time periods" and "semantic information," enabling subsequent spatial modeling and spatiotemporal fusion to obtain a clearer, structured representation at the input level, facilitating the identification of key changes and potential patterns.

[0014] Step 3: Construct the AGFormer spatial modeling module

[0015] This step uses the Anchor Graph Transformer (AGFormer) model to complete spatial modeling. First, an anchor set is constructed based on the stations. Within each anchor, global spatial information is aggregated through self-attention. Then, spatial attention from anchor to nodes is used to propagate anchor information down to each station's time slice. An offset related to connectivity based on station geographical distance is introduced to constrain the attention distribution to conform to actual spatial relationships. Through multi-layer stacking, spatial representations at different scales are obtained.

[0016] Step 4: Construct a dual-branch spatiotemporal fusion framework

[0017] This step establishes a dual-branch spatiotemporal framework. Branch A uses the spatial augmentation features of AGFormer as input, models long- and short-term temporal dependencies using GPT2, and outputs the main temporal representation, reflecting the "space-driven time" path. Branch B first extracts temporal dynamic features from GPT2, then injects them back into AGFormer for spatial augmentation, reflecting the "time-guided space" path. This achieves bidirectional coupling and complementary fusion of spatial and temporal information. The outputs of the two branches are fused using learnable weights to provide the main prediction sequence.

[0018] Step 5: Construct the cascaded deep forest error compensation module

[0019] This step constructs a cascaded deep forest error compensation module (DFEC). Using historical window features, site spatial features, and the initial predictions from the two-branch master model as comprehensive inputs, a cascaded structure composed of multiple layers of random forests is trained, learning the residual patterns of the master predictions layer by layer. During the inference phase, the compensation amount is output and superimposed with the initial predictions to obtain the final result.

[0020] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0021] (1) The method of the present invention first performs data preprocessing, unifies time alignment and uses MLP to complete missing value completion; then performs windowing processing, divides the long sequence into fixed Patch input, and retains station priors such as latitude and longitude.

[0022] (2) In the lexical-temporal alignment stage, the method of this invention introduces sparse attention and directly utilizes the word vectors of the pre-trained language model GPT2 as semantic priors. Without relying on the prompt text, it retains only a small number of semantic units most relevant to the time slice for fusion. Compared with dense attention or purely numerical end-to-end solutions, this mechanism significantly reduces irrelevant noise and computational overhead, making the input more focused on key changes and potential patterns, promoting convergence and improving generalization stability in complex environments.

[0023] (3) In the spatial modeling stage, the method of this invention adopts AGFormer, which uses a two-level structure of "anchor point self-attention - anchor point to node attention" and superimposes geographical bias to explicitly encode the global and local dependencies between irregular sites. Unlike other models, this method divides the entire time period into time slices (Patch), enabling spatial information to interact and propagate with the local time context at a finer granular level, thereby more effectively strengthening local information exchange and capturing key changes.

[0024] (4) The method of this invention constructs a dual-branch framework in the spatiotemporal fusion stage, while retaining two modeling orders: "space-driven time" and "time-guided space," and adaptively fuses them through learnable weights. Compared with a single path, this structure avoids one-sided information dominance and gradient bottlenecks. Single channels often have an inherent bias towards information, either prioritizing space over time, emphasizing spatial constraints, or prioritizing time over space, emphasizing temporal dynamics. The dual-branch approach retains both orders and adaptively fuses them with learnable weights, allowing the biases on both sides to cancel each other out, reducing the accumulation of biases, and obtaining a more balanced and stable result.

[0025] (5) The method of this invention introduces a cascaded deep forest in the error compensation stage to perform posterior learning and online updating of the systematic residuals predicted by the main network. The original end-to-end model is prone to systematic residuals in complex time series at multiple sites. After introducing the cascaded deep forest error compensation, the residuals are learned hierarchically by the main prediction and related features, and the compensation amount is superimposed on the main prediction, thereby compressing the bias and accumulated error and improving the prediction accuracy without changing the backbone network. Attached Figure Description

[0026] Figure 1 This is a flowchart of the multi-site water quality prediction method based on AGFormer-GPT2 of the present invention;

[0027] Figure 2 This is a complete structural diagram of the multi-site water quality prediction based on AGFormer-GPT2 in this invention;

[0028] Figure 3 This is a schematic diagram of the structure of the present invention based on sparse attention for aligning words with temporal data;

[0029] Figure 4 This is a structural diagram of the spatial feature extraction layer AGFormer of the present invention;

[0030] Figure 5 This is a model structure diagram of the pre-trained language model GPT2 of this invention;

[0031] Figure 6 This is a diagram of the dual-branch spatiotemporal feature fusion structure of the present invention;

[0032] Figure 7 This is a structural diagram of the cascaded deep forest error compensation module of the present invention;

[0033] Figure 8 This invention provides the predicted results and ablation experiments. Detailed Implementation

[0034] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0035] like Figure 1 As shown, this invention proposes a multi-site water quality prediction method based on AGFormer-GPT2. The invention follows a bottom-up process of "preprocessing—alignment—spatial—spatiotemporal—correction." Figure 2The diagram shows the overall structure of this model. First, missing data is repaired in the original sequences of multiple sites and indicators, and the sequences are sliced ​​into continuous time slices according to fixed time windows. Second, a sparse attention-based vocabulary-temporal alignment module is constructed. Word vectors from the large-scale pre-trained language model GPT2 are used as vocabulary priors. Each time slice is matched with the vocabulary for relevance matching, retaining only the top-K most important semantic units and weighted fusion to obtain a more discriminative time slice representation. Next, an AGFormer spatial modeling module centered on anchor points is built. A global spatial summary is obtained through self-attention (AASA) between anchor points, and then the spatial prior is injected into each time slice through anchor-node attention (ANSA). RBF geographical bias constraint weights are introduced to enable the model to reflect proximity when learning spatial dependencies. Subsequently, a dual-branch spatiotemporal fusion framework is constructed: Branch A drives GPT2 modeling of long-short-term dependencies using AGFormer output, while Branch B first extracts temporal dynamics from GPT2 and then re-injects AGFormer for secondary spatial enhancement. The two branches are automatically balanced and fused with learnable weights. Finally, through cascaded deep forest error compensation (DFEC), the residual distribution is learned layer by layer using station latitude and longitude, historical features, and dual-branch predictions as inputs. The compensation term output during the inference stage corrects the master prediction, thereby significantly reducing systematic bias and improving the stability and accuracy of multi-index multi-step prediction. The implementation steps of this embodiment are described in detail below, including steps one to five.

[0036] Step 1: Data Preprocessing and Windowing

[0037] First, the raw water quality series from multiple stations and multiple indicators were preprocessed. Specifically, this included aligning the data from each station by time and imputing missing values ​​using machine learning-based MLP.

[0038] After generating high-quality sequences, windowing is performed, i.e., training the sliding window with a fixed length and stride. Continuous observations are divided into local time slices, and each time slice is mapped to a fixed-length vector as a patch. In addition to retaining the original indicators, each time slice also saves station metadata, such as latitude and longitude. This information will provide prior knowledge for subsequent spatial modeling of AGFormer and residual learning of DFEC. Finally, training, validation, and testing are performed in chronological order.

[0039] Step 2: Lexical-temporal feature alignment based on sparse attention

[0040] To systematically model the potential relationship between local dynamic changes and the lexical space in multivariate time series, a lexical temporal alignment mechanism is proposed. This mechanism does not rely on pre-set text for natural language prompts but directly utilizes the word embedding matrix from the GPT2 model. To reduce training costs and improve efficiency, dimensionality reduction is achieved through a fully connected layer, compressing the embedding matrix into a low-dimensional set of lexical token vectors. The time series data is divided into multiple time-segment patches using a sliding window, and each patch is linearly mapped to obtain d. p The feature vectors are used to form the Patch feature sequence.

[0041] like Figure 3 The diagram shows the temporal semantic alignment structure of sparse attention. To enhance its semantic information, pre-trained word embedding vectors from GPT2 are introduced. The GPT2 model contains a fixed word embedding matrix. Where V vocab It is the size of the vocabulary, d vocab This represents the dimension of each word, with each row being a pre-trained word vector. To reduce computational complexity and compress the vocabulary space to a more tractable scale, a linear mapping layer is designed to reduce the dimensionality of the original vocabulary to a candidate set containing fewer word token vectors:

[0042]

[0043] in, The weight matrix for dimensionality reduction mapping yields A. vocab The token embedding matrix is ​​a word-token matrix, with each token having dimension d. vocab This process can be viewed as projecting a high-dimensional vocabulary onto a dense semantic representation subspace, forming a compressed semantic space that can be used for alignment.

[0044] Sparse multi-head attention is used to perform cross-modal fusion of patch vectors and tokens in the GPT2 vocabulary space. Attention scores are calculated for each patch vector and 1000 pseudo-token vectors. For any attention head h, its dimension is d. h .

[0045] First, the attention score is obtained by comparing the attention score with that of each pseudo-token:

[0046]

[0047] in Let Q represent the p-th patch of the n-th variable of the b-th sample at the h-th attention head. Let K be the representation of the Mth semantic token at the hth head.

[0048] To reduce redundant computation and alignment noise, for each patch-head pair, only the top-k most relevant semantic tokens are retained. The attention scores of the remaining tokens are set to -∞ and then normalized to 0 after softmax, resulting in a sparse attention weight matrix. Subsequently, based on the sparse attention, a weighted fusion is performed on the value representations of the semantic tokens to obtain the alignment features under this attention head.

[0049]

[0050] in This represents the V representation of the k-th semantic token in the h-th attention head. The final output is the result of concatenating all attention heads.

[0051]

[0052] Step 3: Construct the AGFormer spatial modeling module

[0053] In the dual-branch spatiotemporal feature extraction framework, the AGFormer (Anchor Graph Transformer) module is the core structure for realizing spatial relationship modeling and feature enhancement. Its main function is to explicitly encode the spatial dependencies between water quality monitoring stations and inject spatial topological information into the time-series features, thereby obtaining a geographically consistent spatial feature representation. The overall structure of AGFormer is as follows: Figure 4 As shown, the module consists of two core sub-layers: Anchor-to-Anchor Self-Attention (AASA) and Anchor-to-Node Spatial Attention (ANSA). The two are alternately connected through a Feed Forward Network (FFN) and layer normalization (Add & Norm) to form a stable and efficient spatial feature modeling unit.

[0054] (1) Anchor Self-Attention Layer (AASA)

[0055] In this stage, the model first constructs a learnable anchor vector for each monitoring station. The anchors are obtained by weighted aggregation of the node features of each station through an attention mechanism. The set of anchors is denoted as:

[0056] A = {a1, a2, ... a} A},a i ∈R A (5)

[0057] Where A represents the number of anchor points, and each anchor point has a iFor a specific site, this represents the global spatial representation of that site, used to capture the overall spatial distribution pattern. Through end-to-end training, the model can adaptively update each anchor vector based on historical water quality data, enabling it to characterize the long-term spatial dependence features of that site in the feature space.

[0058] The core of the AASA layer is to achieve self-organized aggregation of global spatial patterns by interacting with information within the anchor set through a self-attention mechanism. Let the anchor matrix be A∈R. A×d Then self-attention is calculated as follows:

[0059]

[0060] AASA output is stably updated through residual structure and layer normalization:

[0061] A″=LayerNorm(A′+FFN(A′)) (8)

[0062] FFN consists of two layers of linear mapping and a nonlinear activation function (GELU) to enhance the expressive power of nonlinear features. The result of AASA can be regarded as a global interactive model of all monitoring stations, and the resulting A″ is the high-dimensional abstract feature of each station in the global space. Figure 4 The left half shows the overall structure of the mechanism, where each anchor exchanges information with other anchors through a fully connected attention structure, thereby forming a globally consistent spatial feature representation.

[0063] (2) Anchor-Node Spatial Attention Layer (ANSA)

[0064] After obtaining the global anchor feature A″, the model uses an ANSA layer to transfer the anchor feature to time segment nodes, injecting spatial dependency information into the temporal dimension. It is important to note that in this study, "node" does not refer to a monitoring site, but rather to a time segment (token) within a time series window divided in the temporal dimension for a single site. For each site, its time series window of length W is extracted using Patch Embedding to obtain the node sequence:

[0065] X = [x1, x2, ..., x s ],x i ∈R S (9)

[0066] Where S is the number of time slices (Nodes), X t This represents the local temporal features within a time window. ANSA uses anchor points as a global information source to guide the fusion of spatial context information at each time slice node, achieving information mapping from "site-level" to "time-level". Its calculation process is as follows:

[0067]

[0068] Where α ij This represents the attention weight of the i-th time node to the j-th site anchor point. Through this mechanism, the model enables each time node to consider the global information of all sites simultaneously when updating features, achieving spatial feature sharing and dynamic fusion among multiple sites.

[0069] (3) Anchor Point-Node Spatial Attention Layer (ANSA)

[0070] To make the spatial attention distribution more consistent with real geographical patterns, AGFormer introduces a geographical bias term to explicitly model the spatial correlation between sites and time nodes. Let G be the latitude and longitude corresponding to time node i. i The latitude and longitude of anchor point (station) j are G i Geographic similarity is then calculated using the radial basis function (RBF):

[0071]

[0072] Where σ is the distance decay parameter. The attention score after introducing the bias is:

[0073]

[0074] Here, β is the geographic bias weight coefficient, used to adjust the influence of spatial distance on the attention weight. This design ensures that the model maintains geographic continuity when capturing spatial dependencies, making the features between adjacent sites more smooth, thus conforming to the physical laws of spatial diffusion of water quality.

[0075] (4) Feature update and spatial output

[0076] After weighted fusion through the ANSA layer, the time node features are updated as follows:

[0077] X′=layerNorm(X+Dropout(H)) (15)

[0078] X S =LayerNorm(X′+FFN(X′)) (16)

[0079] Among them, X S This represents the temporal feature output that incorporates spatial information. Through multi-layer AGFormer stacking, the model can achieve layer-by-layer abstraction at different spatial scales, enabling the output features to possess both global spatial correlation and local temporal dependence.

[0080] Step 4: Construct a dual-branch spatiotemporal fusion framework

[0081] After spatial feature extraction is completed, the model enters the spatiotemporal fusion stage. This study designs a two-branch spatiotemporal fusion framework, which bidirectionally couples the spatial structure modeling module AGFormer with the time series modeling module GPT2, such as... Figure 5 The diagram shown illustrates the structure of the GPT2 model, designed to enable collaborative prediction of water quality data from multiple sites and indicators. This part of the structure is as follows: Figure 6 As shown, the model consists of two information interaction branches: AGFormer→GPT2 and GPT2→AGFormer, which jointly model spatiotemporal dependencies from two directions: "space-driven temporal modeling" and "temporally guided spatial enhancement," respectively. Through a symmetrical dual-path structure, the model can simultaneously capture the dynamic trends of water quality temporal changes and the spatial coupling characteristics between multiple sites.

[0082] (1) Branch A: AGFormer → GPT2 (space-driven temporal modeling)

[0083] This branch takes the spatial augmentation features output by AGFormer as input and further extracts temporal dynamic features using the sequence modeling capabilities of the GPT2 model. Let the spatial features after the AGFormer module be X. S ∈R B×S×d Where B is the batch size, S is the number of time slices, and d is the feature dimension.

[0084] First, the model will X S Linear projection onto the embedding space of the GPT2 model:

[0085]

[0086] Where d g =768 represents the latent space dimension of the GPT2 model. Subsequently, the sequence Z is fed into the GPT2 model for temporal modeling as the input token sequence:

[0087] H t =GPT2(Z) (18)

[0088] GPT2 captures long-short-term dependencies between different time slices through a multi-head self-attention mechanism and a positional encoding mechanism. The final spatiotemporal features output by the model are represented as follows:

[0089] Y A =LayerNorm(Hlast (19)

[0090] Where H last This indicates the final hidden state.

[0091] The core idea of ​​this branch is to first use AGFormer to fuse spatial features, and then use GPT2 to perform deep sequence modeling in the temporal dimension. This enables time prediction to have global spatial awareness and achieves the capture of "space-driven temporal evolution" features.

[0092] (2) Branch B: GPT2 → AGFormer (Time-guided Space Augmentation)

[0093] To achieve bidirectional spatiotemporal modeling, the second branch employs a reverse information flow structure. The model first extracts temporal features from the input time series window, and then uses AGFormer to map these temporal features onto the spatial structure for enhancement.

[0094] Let the original PatchEmbedding output be X. P ∈R B×S×d The time modeling process is as follows:

[0095] H t =GPT2(X P W in ) t (20)

[0096] Mapping the output of GPT2 back to the AGFormer space representation dimension:

[0097]

[0098] The temporal feature is then input into the AGFormer module to further extract features using the spatial topological relationships between anchor points:

[0099] Y B =AGformer(Z′,E,G) (22)

[0100] Where E represents the constructed graph connection relationship, and G is the geographic coordinate matrix.

[0101] The purpose of this branch is to map the dynamic change patterns learned from the time series back to the spatial domain, enabling the model to adaptively adjust the spatial weight distribution based on the temporal characteristics.

[0102] (3) Dual-branch fusion mechanism

[0103] After the two paths respectively complete the extraction of complementary information—"space-driven temporal modeling" and "time-guided spatial modeling"—the model performs a comprehensive prediction through weighted fusion. Let the outputs of the two branches be Y...A With Y B The fused output is:

[0104] Y F =σ(α)⊙Y A +(1-σ(α))⊙Y B (twenty three)

[0105] Where α∈R d The weight parameters are learnable, σ(·) represents the Sigmoid activation function, and ⊙ represents the element-wise multiplication operation. Through this design, the model can automatically learn the importance ratio of the two branches during training, thereby achieving dynamic information balance in different task scenarios.

[0106] The fused output Y F As the final spatiotemporal joint feature representation, it retains the spatial topological information of AGFormer and integrates the temporal dynamic information of GPT2, providing high-quality input for subsequent error compensation and prediction modules.

[0107] Step 5: Construct a cascaded deep forest error compensation module

[0108] Although the two-branch AGFormer–GPT2 model can effectively characterize the spatiotemporal dependencies of water quality data from multiple sites, it may still be affected by factors such as observation noise, data imbalance, and extreme sample perturbations in actual predictions, leading to systematic biases in some prediction results. To further improve the model's prediction accuracy and generalization performance, this study designed a Deep Forest Error Compensation (DFEC) module in the main model output stage. The structure of this module is as follows: Figure 7 As shown, it is used to correct nonlinear errors in the initial prediction results, thereby enabling the model to "learn twice" in both the spatial and feature dimensions.

[0109] (1) Definition of error compensation problem

[0110] Let the prediction result of the main model be If the true label is y, then the residual is defined as:

[0111]

[0112] The goal of the error compensation model is to construct a function f comp (X comp ), so that:

[0113]

[0114] Where X compIt is a comprehensive set of input features, including: historical time series window features; geographic information of monitoring sites; and multi-branch prediction results of the main model (AGFormer–GPT2).

[0115] By combining the original input with the main model output as compensation features, DFEC can fully capture the high-dimensional nonlinear relationship between the residual and the input space, achieving more refined error modeling.

[0116] (2) Cascaded deep forest structure

[0117] The error compensation module employs an improved Deep Forest structure. Unlike traditional shallow random forests, deep forests construct multiple forest layers through hierarchical cascading, enabling the model to automatically extract higher-level feature patterns from the data.

[0118] Suppose the module contains L hierarchical forests, each level consisting of several random forests, and each forest contains multiple decision trees. The input to the l-th level is the concatenated output of the previous level:

[0119]

[0120] in H represents the set of random forests at layer l; (l) represents the feature response output of the l-th layer; [·] denotes the vector concatenation operation.

[0121] Through layer-by-layer cascading, deep forests can gradually extract higher-order features and residual patterns, thereby achieving layer-by-layer refined error modeling.

[0122] Each forest layer consists of multiple random forests, and each random forest independently models the error of different prediction indicators, as shown in the following formula:

[0123] Δy j =RF j (X comp ),j=1,2,…,n (27)

[0124] The final error prediction matrix is:

[0125]

[0126] (3) Random Forest Learner

[0127] In each layer, the Random Forest Regressor serves as the basic learner, fitting the nonlinear error distribution by ensembling multiple decision trees. For each prediction metric, the Random Forest output is the average of the predictions from multiple decision trees.

[0128]

[0129] Where T represents the number of decision trees in the forest. Let be the prediction function for the residual of the j-th index from the t-th tree.

[0130] Each tree primarily uses bootstrap sampling to generate a subset of samples, and the splitting features and thresholds are determined by minimizing the in-node squared error.

[0131]

[0132] Random forests achieve ensemble decorrelation through feature subsampling and sample randomization, thereby effectively reducing model variance and improving robustness. This design enables deep forests to achieve powerful nonlinear error modeling capabilities with low computational cost when dealing with complex nonstationary water quality sequence data.

[0133] (4) Error Compensation Feature Construction and Training Process

[0134] During the training phase, the prediction residuals of the master model are first calculated. Then construct the compensation feature matrix:

[0135]

[0136] The input features of each sample include time window flattening features, spatial latitude and longitude information, and prediction results of each branch, which can fully reflect the spatiotemporal dependence of the error.

[0137] This feature is then input into a deep forest for learning, and the objective function is optimized as follows:

[0138] min||ef comp (X comp )|| 2 (32)

[0139] After training, the model calculates an error compensation term based on the new input samples during the prediction phase. And make final prediction revisions:

[0140]

[0141] This effectively corrects systematic biases and improves overall prediction accuracy without changing the main model structure.

[0142] Example 1:

[0143] This experiment conducted a water quality prediction study based on multi-site monitoring data. Observational data from six monitoring stations in the target water area were selected as the research object, covering the period from 00:00 on January 1, 2022 to 20:00 on December 31, 2022, resulting in 2090 sets of continuous time-series data. Regarding the selection of indicators, considering the physicochemical properties and nutrient status of the water body, nine key water quality indicators were collected from each station: water temperature (WT), pH, permanganate index (CODMn), ammonia nitrogen (NH3-N), total phosphorus (TP), total nitrogen (TN), conductivity (EC), turbidity (Turbine), and dissolved oxygen (DO). The experimental data are shown in Table 1 below.

[0144] Table 1 Experimental Data

[0145]

[0146] The method of this invention is as follows:

[0147] Step 1: Data Preprocessing and Windowing

[0148] First, the multivariate water quality sequences collected from six monitoring stations underwent unified preprocessing. The observation data from all stations were aligned according to timestamps to ensure the correspondence of indicators within the same time slice. For missing items in the sequences, a multilayer perceptron (MLP) model was used for imputation to maintain data continuity. Then, the complete sequence was divided into multiple time slices (patches) by sliding a fixed time window along the time sequence, and each time slice was encoded as a fixed-length vector input to the model. After processing, the data was sequentially divided into training, validation, and test sets according to time sequence for modeling and evaluation in this embodiment.

[0149] Step 2: Lexical-temporal feature alignment based on sparse attention

[0150] After generating the time-slice input, this embodiment further enhances the numerical time slices using semantic information. First, word embedding vectors are extracted from the pre-trained GPT2 model, and linear dimensionality reduction is used to construct a semantic vector library consisting of a small number of pseudo-tokens. Then, the time-slice vectors and the semantic library are input into a sparse attention module, which retains only the semantic vectors most relevant to the current time slice for weighted fusion, thereby obtaining a semantically enhanced time-slice representation. This provides a more expressive input for subsequent spatial modeling and temporal prediction.

[0151] Step 3: Construct the AGFormer spatial modeling module

[0152] After obtaining the semantically enhanced time-slice representation, this embodiment uses the AGFormer model to model the spatial relationships between the six monitoring stations. First, learnable anchor vectors are set for each station, and global spatial features are extracted through anchor self-attention. Then, through an anchor-node attention mechanism, global spatial information is transmitted to the corresponding time slices of each station, enabling the time slices to reflect the spatial relationships of multiple stations simultaneously during updates. Simultaneously, combined with latitude-longitude-based geographic bias, the model learns a dependency structure that conforms to spatial proximity. After processing by this module, spatially enhanced features suitable for spatiotemporal fusion can be obtained.

[0153] Step 4: Construct a dual-branch spatiotemporal fusion framework

[0154] After obtaining spatial features, this embodiment constructs a two-branch spatiotemporal fusion framework. One branch inputs the spatial features output by AGFormer into GPT2 for further modeling of the time series, forming a space-driven temporal modeling path; the other branch first uses GPT2 to extract temporal dynamic features, and then inputs them into AGFormer for spatial augmentation, forming a time-driven spatial modeling path. The outputs of the two branches are fused through learnable gating weights to obtain a comprehensive initial prediction of spatiotemporal features.

[0155] Step 5: Construct a cascaded deep forest error compensation module

[0156] After obtaining the prediction results from the main network, this embodiment further introduces a cascaded deep forest error compensation module (DFEC) for error correction. Targeting the residual between the main predicted value and the true value, a compensation input is constructed, consisting of historical time slice features, station latitude and longitude, and dual-branch fusion features. This compensation input is then fed into a cascaded random forest model for learning. The model fits the residual variation pattern layer by layer in the multi-level forest, outputting the corresponding compensation amount during inference. This compensation is added to the main model's predicted value to obtain the final prediction result, thereby improving the overall prediction accuracy and stability.

[0157] To verify the impact of each key module in the method of this invention on the overall prediction performance, this embodiment uses the dissolved oxygen (DO) index of a monitoring station as the test object and conducts an ablation experiment. Four model configurations were constructed respectively: branch A (AGFormer→GPT2), branch B (GPT2→AGFormer), dual-branch fusion, and dual-branch + error compensation (DFEC). The improvement effects of different modeling paths and compensation modules on the prediction performance were compared and analyzed. The prediction curves for each configuration are shown below. Figure 8 As shown in Table 2, the quantitative evaluation indicators are presented.

[0158] The results show that the dual-branch fusion model retains both "space-driven time" and "time-guided space" modeling paths, balancing trends and details, resulting in a closer fit between predicted curves and actual values. All metrics are significantly better than those of the single-branch model. Furthermore, by adding cascaded deep forest error compensation (DFEC) to the dual-branch model, the model can perform secondary corrections on residuals that are difficult to fit by the main network, making the predicted curve almost coincide with the actual observations. This leads to significant improvements in MSE, MAE, MAPE, and R... 2 It achieves optimal results on evaluation indicators, demonstrating significant error suppression capability and global stability.

[0159]

[0160] Among them, y i Represents the i-th true value. Represents the i-th predicted value. The average value represents the true value, and n is the number of data samples.

[0161] Table 2 Evaluation Indicators for Ablation Experiments

[0162]

[0163] Table 2 shows that the MSE of branch A is 0.0869, branch B decreases to 0.0554, and the dual-branch model further decreases to 0.0312, indicating that dual-path fusion can indeed effectively improve spatiotemporal modeling capabilities. After adding DFEC, the MSE further decreases to 0.0101, the MAPE decreases to 0.70%, and R... 2 The improvement to 0.9898 indicates that the error compensation module can significantly reduce accumulated errors and improve the accuracy and reliability of predictions.

[0164] In conclusion, the ablation experiment fully demonstrates that:

[0165] (1) Branch A and Branch B focus on different features and are complementary;

[0166] (2) Dual-branch fusion can comprehensively improve the model's ability to model both dissolved oxygen change trends and local fluctuations simultaneously;

[0167] (3) The cascaded deep forest error compensation module further significantly improves the prediction accuracy and is an important component of performance improvement.

[0168] Therefore, the complete framework of "semantic enhancement + dual-branch AGFormer-GPT2 spatiotemporal fusion + DFEC compensation" proposed in this invention shows excellent predictive advantages in multi-site water quality prediction.

[0169] Except for the technical features described in the specification, all other technologies are known to those skilled in the art. Descriptions of well-known components and technologies are omitted in this invention to avoid redundancy and unnecessary limitation. The embodiments described above do not represent all embodiments consistent with this application. Various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of this invention are still within the protection scope of this invention.

Claims

1. A multi-site water quality prediction method based on AGFormer-GPT2, characterized in that, Includes the following steps: Step 1: Data preprocessing and windowing to obtain the complete dataset and prior input. The water quality time series of multiple stations are aligned by time, and a multilayer perceptron (MLP) is used to fill in the missing values. Time slices are obtained by sliding slices with a fixed-length time window, and each time slice is mapped to a fixed-length vector to form a time slice token, while retaining the station's latitude and longitude and other metadata. Step 2: Dimensionality reduction of the vocabulary is performed, and cross-attention fusion is carried out with the temporal patch features through sparsity filtering to obtain semantically enhanced water quality temporal patch features; The vocabulary fusion and temporal patch feature process includes: Step 2.1: Based on the vocabulary obtained by dimensionality reduction of GPT2 word vectors, calculate the relevance score between each temporal patch and each item in the semantic library, select the top-k most relevant words according to the set sparsity parameter, and do not participate in the fusion of other words. Step 2.2: Under the sparsity constraint, cross-attention is calculated using the temporal Patch representation as the query and the Top-k vocabulary set as the key and value. The attention weights of the valid items are normalized and the value vectors are weighted and converged to obtain the semantic alignment vector. This vector is then fused with the original Patch representation according to the cross-attention mechanism to form a semantically enhanced temporal Patch feature. Step 3: Construct the AGFormer spatial modeling module and introduce geographical correlation to integrate the spatial relationships of stations into the multi-site time series data; An anchor graph Transformer (AGFormer) is used to model the spatial dependency of stations. First, the set of anchor points is initialized, and global spatial information is aggregated within the anchor points using anchor self-attention (AASA). Then, the anchor point features are passed down to the time slices of each station through anchor-node attention (ANSA). A distance bias based on latitude and longitude is added to the attention scoring to make the weights reasonably adjusted with spatial distance. Several layers are stacked according to "AASA→ANSA" to output multi-scale spatial augmentation features. Step 4: Construct a dual-branch spatiotemporal fusion framework (space-driven time & time-guided space). Construct branches A (AGFormer→GPT-2, spatially driven temporally) and B (GPT-2→AGFormer, temporally guided spatially): Branch A uses the spatial augmentation features output by AGFormer as input, and obtains temporal features or multi-step prediction representations through GPT-2 encoding; Branch B encodes time slices with GPT-2 to obtain temporal dynamics, then performs a linear mapping back to the spatial dimension of AGFormer and performs another layer of spatial encoding to obtain a spatial reconstruction representation. Learnable weights are introduced for weighted fusion to obtain the initial prediction results. Step 5: Construct a cascaded deep forest error compensation module (DFEC) to correct the prediction residuals of the main model. Using historical window features, site spatial information, and initial predictions as comprehensive inputs, a cascaded deep forest consisting of multiple layers of random forests is used to learn the residual pattern layer by layer. The output compensation amount is then superimposed with the master prediction to obtain the final prediction result.

2. The method according to claim 1, characterized in that, In step 3, the distance weights calculated using the latitude and longitude of the station are used to form a geographical bias. This bias is then embedded into the scoring process of anchor point self-attention and anchor point-node attention, so that the attention score is affected by spatial information.

3. The method according to claim 1, characterized in that, In step 4, a dual-branch spatiotemporal fusion model is constructed. Branch A is set to extract features in the order of "space → time", and branch B is set to extract features in the order of "time → space". The two branches run in parallel and the outputs are fused in a learnable manner.

Images