Water quality prediction method based on graph convolution and time-space interlaced attention mechanism

Abstract

The invention is suitable for the field of water quality prediction, and particularly provides a water quality prediction method based on graph convolution and a time-space interlaced attention mechanism, and the method comprises the steps: constructing a prediction model containing a four-layer Transform architecture based on the time-space interlaced attention mechanism; the first layer is embedded in an embedding manner; the second layer is a position coding module; a third layer is N identical Transform encoder modules with a space-time interlaced attention mechanism, and in the space-time interlaced attention mechanism, a time attention module and a space attention module are stacked by integrating graph convolution into a Transform frame to form a Transform encoder module; the fourth layer is an output layer; and training the constructed prediction model, wherein the trained model is used for predicting the target water quality. The time dimension and the space dimension of the data can be processed at the same time, the dependency relationship under different time-space scales is captured, and the expression ability and the prediction performance of the model are improved.

Description

Technical Field

[0001] The present invention belongs to the field of spatiotemporal data prediction technology, and in particular to a water quality prediction method based on graph convolution and spatiotemporal interleaved attention mechanism. Background Art

[0002] Sensor technology is widely used, which can monitor water pH, dissolved oxygen, concentration of various pollutants and other multi-indicator data in real time, and realize rapid data collection. Internet of Things technology helps to build a data transmission network, so that the collected data can be remotely and efficiently transmitted to the processing end. Big data technology can store, manage and integrate massive and complex water quality data. These technologies have generated a large amount of spatiotemporal data in the field of water quality.

[0003] Compared with traditional spatiotemporal data, water quality data involves multi-indicator coupling and highly dynamic mutation characteristics. Its collection process needs to deal with the problem of multi-source heterogeneous signal fusion under complex environmental interference. Data processing needs to combine physical driving models with spatiotemporal heterogeneity analysis to resolve nonlinear causal relationships and achieve accurate predictions.

[0004] At present, with the development of machine learning technology, deep learning methods are commonly used in water quality prediction. Deep learning can handle complex nonlinear relationships. For example, neural networks can accurately capture the complex correlations between various factors affecting agricultural water quality and improve prediction accuracy. It can handle high-dimensional data, automatically screen key features, and has strong generalization capabilities. It can also make good predictions under new data. It can also automatically learn features and tap into deep value.

[0005] However, existing water quality prediction methods based on deep learning still have the following shortcomings: First, most existing technologies do not fully integrate the spatiotemporal information of water quality data. Traditional methods mostly use static analysis of a single time or space dimension. Some traditional models use convolutional neural networks to extract spatial features alone or realize spatiotemporal interactions through simple concatenation. It is difficult to capture the spatial heterogeneity between monitoring points and the dynamic evolution of water quality parameters. It cannot simultaneously reflect the spatial differences in water quality at different monitoring points and the dynamic changes over time, and the ability to utilize and dynamically adjust real-time data is weak, making it difficult to adapt to rapid changes in water quality, resulting in poor prediction results; Second, existing water quality prediction methods often use static statistical models or shallow machine learning algorithms. The main defect of these models is that they separate time and space for analysis. Specifically, existing systems usually treat spatial topology and time series in isolation and lack the ability to adapt to the dynamic characteristics of data. These methods may not accurately capture complex water quality change patterns, resulting in insufficient or inaccurate feature extraction.

[0006] Third, the generalization ability of traditional water quality prediction systems mainly depends on the comprehensiveness of training data and the complexity of the model. However, in practice, it is difficult to collect water quality data that covers all possible situations. When such water quality prediction systems encounter data outside the distribution range of training data, they are often difficult to process effectively, resulting in poor overall performance of the system and failure to achieve the expected prediction effect. Summary of the invention

[0007] The purpose of the embodiment of the present invention is to provide a water quality prediction method based on graph convolution and spatiotemporal attention mechanism, aiming to solve the technical problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions.

[0009] The water quality prediction method based on graph convolution and time-space staggered attention mechanism provided by the embodiment of the present invention includes the following steps: A prediction model with a four-layer Transformer architecture is built based on the spatiotemporal staggered attention mechanism; The first layer is embedding, which is used to transform spatiotemporal sequences; The second layer is the position encoding module, which is used to encode the corresponding position information in the water quality data; The third layer consists of N identical Transformer encoder modules with a spatiotemporal attention mechanism, which is used to deeply fuse the spatial and temporal information of water quality data. In the spatiotemporal attention mechanism, the temporal attention module and the spatial attention module are constructed by integrating graph convolution into the Transformer framework, and the temporal attention module and the spatial attention module are stacked to form a Transformer encoder module. The fourth layer is the output layer, which is a fully connected layer used to map the output features of the encoder to the predicted target dimension; The constructed prediction model is trained, the gradient of the loss function is calculated through the back propagation algorithm, and the parameters of the model are updated using the optimizer. The trained model is used to predict the target water quality.

[0010] Furthermore, in the embedding of the first layer of the prediction model, it is used to convert the original water quality data into a feature representation suitable for model processing; wherein, the spatiotemporal sequence of water quality data input as [B, T, C, H, W] is converted into a vector of [B, T, N, D], where B represents the batch size, T represents the time step, N is the number of spatial nodes, and D is the hidden dimension.

[0011] Furthermore, in the position encoding module of the second layer of the prediction model, the sine-cosine absolute position encoding method is used to provide position information for the Transformer model, which is expressed as: When i is an odd number: ; When i is an even number: ; Among them, d represents the dimension of the embedded vector in the water quality data, P represents the position information, ps represents the index of the position information, and i represents the index of the embedded vector dimension.

[0012] Furthermore, in the Transformer encoder module of the third layer of the prediction model, the encoder module includes a time block TB, a space block SB and a time-space interleaved encoder architecture. The time block TB is used to calculate the time attention; the space block SB is used to calculate the space attention; Each encoder is formed by stacking time blocks TB and space blocks SB; Each sub-layer is followed by a residual connection; Temporal Block TB integrates multi-head attention mechanism and Bilinear GLU-based feedforward neural network; The multi-head attention mechanism performs a weighted summation of the input by calculating the relationship between the query vector Q, the key vector K, and the value vector V, expressed as: ; Where Q = xW q , K = xW k , V = xW v ; W q , W k , W v They represent the weight matrices of the corresponding vectors; x represents the input information; T represents the transposition operation of the vector; D k represents the dimension of the key vector; As part of the feedforward neural network, Bilinear GLU divides the input x into two parts, x1 and x2, and interacts with two linear transformations and matrix multiplications to capture the feature combination of the input information, expressed as: ; Among them, σ represents the Sigmoid activation function, x1 and x2 are segmentation vectors, represents element-wise product; U1 and U2 represent linear transformation matrices; T represents transpose operation; W1 represents weight matrix; b1 and b2 represent bias vectors.

[0013] Furthermore, in the calculation process of the time block TB, the input I lNormalize it and then calculate it through the multi-head attention mechanism MHA, the calculation formula is: ; Next, for the desired T l After layer normalization and Bilinear GLU-based feedforward neural network processing, I is calculated. l+1 , the calculation formula is: ; In the formula, LN() represents normalization processing; The spatial block SB is used to calculate the attention in space. The spatial block SB integrates graph convolution and attention mechanism, and updates the node features through the graph convolution formula. For node i, its updated feature vector Calculate using the following formula: ; Where W (l) is a trainable weight matrix, N(i) is the set of neighbor nodes of node i, b (l) is the bias term, σ is the ReLU activation function; Then the attention score is calculated based on the graph convolution. The calculation formula of the attention score is: ; Among them, i and j are neighbor nodes, represents the attention score between neighbor nodes i and j, is the query vector of node i, k j is the key vector of node j; Next, the attention score is normalized and the attention weight α is calculated. ij , perform weighted aggregation on the node's neighbor features and update the node's feature representation. The calculation formula is: ; In the formula, h i represents the feature representation of node i; h j represents the feature representation of node j; W represents the weight matrix; N(i) represents the set of neighbor nodes of node i; σ represents the activation function; Through calculation, the characteristics of water quality data are extracted and integrated.

[0014] Furthermore, a spatiotemporal interleaved encoder architecture is constructed, which is composed of a time block and a space block, and an encoder layer with a spatiotemporal interleaved attention mechanism is formed based on the order of the time block TB to the space block SB; In the encoder layer, the input [B, T, N, D] is reshaped into [B, N, T, D], and then the attention is calculated on the time series through the time block TB. After the calculation is completed, the data is reshaped into a suitable dimension as the input of the spatial block SB, and the attention is calculated on the spatial dimension using graph convolution.

[0015] Furthermore, in the process of training the constructed prediction model, the mean square error loss function MSE is used for training. The mean square error loss function MSE is expressed as: ;

[0016] Among them, y i is the true value, is the predicted value; n is the number of samples, and N represents the number of samples involved in the calculation.

[0017] Compared with the prior art, the water quality prediction method based on graph convolution and time-space staggered attention mechanism of the present invention has the following beneficial effects: First, by integrating graph convolution into the spatiotemporal attention mechanism, the model can simultaneously consider the spatial distribution and temporal changes of water quality, so that the model can capture more complex patterns and relationships, and improve the model's expression ability and prediction performance; integrating graph convolution into the encoder can effectively capture spatial dependencies and improve the generalization ability of the model prediction; Third, in the construction of the Transformer model with the spatiotemporal staggered attention mechanism of the present invention, two attention calculation modules are designed. The first module performs attention calculation on time, and the second module performs attention calculation on space. The two modules are connected in a stacking manner, which improves the adaptability and timeliness of model prediction.

[0018] In summary, the present invention fully integrates the spatiotemporal information of water quality data, and can well reflect the spatial differences and dynamic changes of water quality over time at different monitoring points at the same time; combined with graph convolution, it can extract features more accurately; because the present invention can comprehensively consider spatiotemporal information, it has strong anti-interference ability for some sudden interference factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention.

[0020] Figure 1 It is a schematic diagram of the system architecture of the water quality prediction method based on graph convolution and time-space staggered attention mechanism of the present invention; Figure 2This is a flow chart of the water quality prediction method based on graph convolution and spatiotemporal attention mechanism of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0022] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0023] In one embodiment of the present invention, a water quality prediction method based on graph convolution and spatiotemporal staggered attention mechanism is provided. The water quality prediction method first preprocesses the water quality data. Specifically, the present invention is based on collecting and integrating water quality data from multiple sources, taking the acquired water quality data as a data set, and performing data cleaning, data standardization, feature engineering and data partitioning on the acquired data set; In one implementation, during the data preprocessing process, various water quality related indicator data are collected from water quality monitoring stations, and environmental data that may affect water quality are collected, integrated according to time and space dimensions to construct a unified data set; and missing values ​​in the data set are filled with statistical values; Then the data is normalized, including: The Z-score standardization method was used to transform the data into a standard normal distribution with a mean of 0 and a standard deviation of 1; For each data point, its normalized value X norm The calculation formula is expressed as: ; In the formula, μ is the mean of the data and σ is the standard deviation of the data.

[0024] Please refer to Figure 1 and Figure 2 The water quality prediction method based on graph convolution and time-space staggered attention mechanism provided by the embodiment of the present invention also includes the following steps: S1. Construct a prediction model with a four-layer Transformer architecture based on the spatiotemporal attention mechanism. S2, the first layer is Embedding, which is used to transform spatiotemporal sequences; S3, the second layer is a position encoding module, which is used to encode the corresponding position information in the water quality data; S4, the third layer is N identical Transformer encoder modules with a spatiotemporal attention mechanism, which is used to deeply fuse the spatial and temporal information of water quality data. In the spatiotemporal attention mechanism, the temporal attention module and the spatial attention module are constructed by integrating graph convolution into the Transformer framework, and the temporal attention module and the spatial attention module are stacked to form a Transformer encoder module; S5, the fourth layer is the output layer, which is a fully connected layer used to map the output features of the encoder to the predicted target dimension; Specifically, in the embedding of the first layer of the prediction model, it is used to convert the original water quality data into a feature representation suitable for model processing; the spatiotemporal sequence of water quality data input as [B, T, C, H, W] is converted into a vector of [B, T, N, D], where B represents the batch size, T represents the time step, N is the number of spatial nodes, and D is the hidden dimension; this processing can convert the complex original water quality data into a format that is easy for the model to process, which is convenient for subsequent feature extraction; Specifically, in the position encoding module of the second layer of the prediction model, water quality data has temporal and spatial sequence characteristics. The position encoding module can provide position information for the Transformer model, thereby improving the model's ability to capture water quality changes. Among them, the position encoding module uses sine and cosine absolute position encoding to provide position information for the Transformer model. Specifically, for position ps (counting from 0) and dimension i (counting from 0), the calculation formula of sine and cosine encoding is expressed as follows: When i is an odd number: ; When i is an even number: ; Among them, d represents the dimension of the embedded vector in the water quality data, P represents the position information, ps represents the index of the position information, and i represents the index of the embedded vector dimension.

[0025] Furthermore, in the construction of the Transformer model with the time-space interleaved attention mechanism of the present invention, two attention calculation modules are designed, the first module performs attention calculation on time, and the second module performs attention calculation on space, and the two modules are connected in a stacking manner, thereby improving the adaptability and timeliness of model prediction; specifically, in the Transformer encoder module of the third layer of the prediction model, the encoder module includes a time block TB, a space block SB and a time-space interleaved encoder architecture, and the time block TB and the space block SB are the core components in the encoder module; each encoder is formed by stacking the time block TB and the space block SB; each sublayer is followed by a residual connection; the time block TB is used to perform time attention calculation; the space block SB is used to calculate the space attention; The time block TB of the embodiment of the present invention integrates the multi-head attention mechanism and the feedforward neural network based on Bilinear GLU; wherein the multi-head attention mechanism performs weighted summation on the input by calculating the relationship between the query vector Q, the key vector K and the value vector V, which is expressed as: ; Where Q = xW q , K = xW k , V = xW v ; W q , W k , W v They represent the weight matrices of the corresponding vectors; x represents the input information; T represents the transposition operation of the vector; D k represents the dimension of the key vector; Furthermore, as part of the feedforward neural network, Bilinear GLU divides the input x into two parts, x1 and x2, and captures the feature combination of the input information through two linear transformations and matrix multiplication interactions, expressed as: ; Among them, σ represents the Sigmoid activation function, x1 and x2 are segmentation vectors, represents element-wise product; U1 and U2 represent linear transformation matrices; T represents transpose operation; W1 represents weight matrix; b1 and b2 represent bias vectors.

[0026] Furthermore, in the calculation process of the time block TB, the input I l Normalize it and then calculate it through the multi-head attention mechanism MHA, the calculation formula is: ; Next, for the desired T lAfter layer normalization and Bilinear GLU-based feedforward neural network processing, I is calculated. l+1 , the calculation formula is: ; In the formula, LN() represents normalization processing; Furthermore, the spatial block SB is used to calculate the attention in space. The spatial block SB integrates graph convolution and attention mechanism, and updates the node features through the graph convolution formula. For node i, its updated feature vector It can be calculated by the following formula: ; Where W (l) is a trainable weight matrix, N(i) is the set of neighbor nodes of node i, b (l) is the bias term, σ is the ReLU activation function; Then the attention score is calculated based on the graph convolution. The calculation formula of the attention score is: ; Among them, i and j are neighbor nodes, represents the attention score between neighbor nodes i and j, is the query vector of node i, k j is the key vector of node j; Next, the attention score is normalized and the attention weight α is calculated. ij , perform weighted aggregation on the node's neighbor features and update the node's feature representation. The calculation formula is: ; In the formula, h i represents the feature representation of node i; h j represents the feature representation of node j; W represents the weight matrix; N(i) represents the set of neighbor nodes of node i; σ represents the activation function; Through calculation, the characteristics of water quality data can be further extracted and integrated.

[0027] Furthermore, a spatiotemporal interleaved encoder architecture is constructed, which is composed of a time block and a space block, and an encoder layer with a spatiotemporal interleaved attention mechanism is formed based on the order of the time block TB to the space block SB; In the encoder layer, the input [B, T, N, D] is reshaped into [B, N, T, D], and then the attention is calculated on the time series through the time block TB. After the calculation is completed, the data is reshaped into a suitable dimension as the input of the spatial block SB, and the attention is calculated on the spatial dimension using graph convolution.

[0028] By integrating graph convolution into the spatiotemporal attention mechanism, the model can simultaneously consider the spatial distribution and temporal changes of water quality, allowing the model to capture more complex patterns and relationships, thereby improving the model's expressive power and predictive performance. Integrating graph convolution into the encoder can effectively capture spatial dependencies and improve the generalization ability of the model's predictions.

[0029] For further information, please refer to Figure 1 and Figure 2 The water quality prediction method based on graph convolution and time-space staggered attention mechanism provided by the present invention also includes the following steps: S6. Train the constructed prediction model, calculate the gradient of the loss function through the back propagation algorithm, and use the optimizer to update the parameters of the model. The trained model is used to predict the target water quality; Specifically, in the process of training the constructed prediction model, the mean square error loss function MSE is used for training. The mean square error loss function MSE is expressed as: ;

[0030] Among them, y i is the true value, is the predicted value; n is the number of samples, and N represents the number of samples involved in the calculation.

[0031] Finally, the present invention inputs the divided test set into the spatiotemporal sequence prediction model after training, performs testing and outputs the prediction result.

[0032] In summary, the present invention fully integrates the spatiotemporal information of water quality data, and can well reflect the spatial differences and dynamic changes of water quality over time at different monitoring points; in addition, the present invention can extract features more accurately in combination with graph convolution; the present invention has strong anti-interference ability for some sudden interference factors because it can comprehensively consider spatiotemporal information.

[0033] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. It can be fully applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized. Therefore, without departing from the general concept defined by the claims and equivalent scope, the present invention is not limited to the specific details and the illustrations shown and described here.

Claims

1. A water quality prediction method based on graph convolution and spatiotemporal attention mechanism, characterized by: The water quality prediction method includes the following steps: A prediction model with a four-layer Transformer architecture is built based on the spatiotemporal staggered attention mechanism; The first layer is embedding, which is used to transform spatiotemporal sequences; The second layer is the position encoding module, which is used to encode the corresponding position information in the water quality data; The third layer consists of N identical Transformer encoder modules with a spatiotemporal attention mechanism, which is used to deeply fuse the spatial and temporal information of water quality data. In the spatiotemporal attention mechanism, the temporal attention module and the spatial attention module are constructed by integrating graph convolution into the Transformer framework, and the temporal attention module and the spatial attention module are stacked to form a Transformer encoder module. The fourth layer is the output layer, which is a fully connected layer used to map the output features of the encoder to the predicted target dimension; The constructed prediction model is trained, the gradient of the loss function is calculated through the back propagation algorithm, and the parameters of the model are updated using the optimizer. The trained model is used to predict the target water quality.

2. The water quality prediction method based on graph convolution and temporal and spatial staggered attention mechanism according to claim 1 is characterized in that: In the embedding of the first layer of the prediction model, it is used to convert the original water quality data into a feature representation suitable for model processing; the spatiotemporal sequence of water quality data input as [B, T, C, H, W] is converted into a vector of [B, T, N, D], where B represents the batch size, T represents the time step, N is the number of spatial nodes, and D is the hidden dimension.

3. The water quality prediction method based on graph convolution and time-space staggered attention mechanism according to claim 2 is characterized in that: In the position encoding module of the second layer of the prediction model, the sine and cosine absolute position encoding method is used to provide position information for the Transformer model, which is expressed as: When i is an odd number: ； When i is an even number: ； Among them, d represents the dimension of the embedded vector in the water quality data, P represents the position information, ps represents the index of the position information, and i represents the index of the embedded vector dimension.

4. The water quality prediction method based on graph convolution and temporal and spatial staggered attention mechanism according to claim 3 is characterized in that: In the Transformer encoder module of the third layer of the prediction model, the encoder module includes a time block TB, a space block SB and an encoder architecture that constructs a time-space interleaved structure. The time block TB is used to calculate the time attention; the space block SB is used to calculate the space attention. Each encoder is formed by stacking time blocks TB and space blocks SB; Each sub-layer is followed by a residual connection; Temporal Block TB integrates multi-head attention mechanism and Bilinear GLU-based feedforward neural network; The multi-head attention mechanism performs a weighted summation of the input by calculating the relationship between the query vector Q, the key vector K, and the value vector V, expressed as: ； Where Q = xW q , K = xW k , V = xW v ; W q , W k , W v They represent the weight matrices of the corresponding vectors; x represents the input information; T represents the transposition operation of the vector; D k represents the dimension of the key vector; As part of the feedforward neural network, Bilinear GLU divides the input x into two parts, x1 and x2, and interacts with two linear transformations and matrix multiplications to capture the feature combination of the input information, expressed as: ； Among them, σ represents the Sigmoid activation function, x1 and x2 are segmentation vectors, represents element-wise product; U1 and U2 represent linear transformation matrices; T represents transpose operation; W1 represents weight matrix; b1 and b2 represent bias vectors.

5. The water quality prediction method based on graph convolution and time-space staggered attention mechanism according to claim 4 is characterized in that: In the calculation process of the time block TB, first the input I l Normalize it and then calculate it through the multi-head attention mechanism MHA, the calculation formula is: ； Next, for the desired T l After layer normalization and Bilinear GLU-based feedforward neural network processing, I is calculated. l+1 , the calculation formula is: ； In the formula, LN() represents normalization processing; The spatial block SB is used to calculate the attention in space. The spatial block SB integrates graph convolution and attention mechanism, and updates the node features through the graph convolution formula. For node i, its updated feature vector Calculate using the following formula: ； Where W (l) is a trainable weight matrix, N(i) is the set of neighbor nodes of node i, b (l) is the bias term, σ is the ReLU activation function; Then the attention score is calculated based on the graph convolution. The calculation formula of the attention score is: ； Among them, i and j are neighbor nodes, represents the attention score between neighbor nodes i and j, is the query vector of node i, k j is the key vector of node j; Next, the attention score is normalized and the attention weight α is calculated. ij , perform weighted aggregation on the node's neighbor features and update the node's feature representation. The calculation formula is: ； In the formula, h i represents the feature representation of node i; h j represents the feature representation of node j; W represents the weight matrix; N(i) represents the set of neighbor nodes of node i; σ represents the activation function; Through calculation, the characteristics of water quality data are extracted and integrated.

6. The water quality prediction method based on graph convolution and temporal and spatial staggered attention mechanism according to claim 5 is characterized in that: In constructing a spatiotemporal interleaved encoder architecture, the architecture consists of a temporal block and a spatial block, and an encoder layer with a spatiotemporal interleaved attention mechanism is formed based on the order of the temporal block TB to the spatial block SB; In the encoder layer, the input [B, T, N, D] is reshaped into [B, N, T, D], and then the attention is calculated on the time series through the time block TB. After the calculation is completed, the data is reshaped according to the required dimension and used as the input of the spatial block SB. The attention is calculated on the spatial dimension using graph convolution.

7. The water quality prediction method based on graph convolution and temporal and spatial staggered attention mechanism according to claim 6 is characterized in that: In the process of training the constructed prediction model, the mean square error loss function MSE is used for training. The mean square error loss function MSE is expressed as: ； Among them, y i is the true value, is the predicted value, n is the number of samples, and N represents the number of samples involved in the calculation.

Images