Multi-water quality factor prediction method based on feature compression and interpretable feature fusion

Abstract

The invention discloses a multi-water quality factor prediction method based on feature compression and interpretable feature fusion. The method comprises the following steps: acquiring multi-sensor data, and calculating to obtain a physical derivative variable based on the multi-sensor data; multi-sensor data and physical derivative variables serve as input variables to train a CS-CNN-Transform prediction model, local features of the input variables are extracted through CNN, and a pooling layer in the CNN is replaced by a compressed sensing layer; introducing a multi-layer attention mechanism to carry out adaptive feature fusion on the input local features to obtain fusion features; inputting the fusion feature into a Transform, and outputting a predicted value; key features influencing water quality changes are dynamically recognized in the model training process through a multi-layer attention mechanism; the contribution degree of the input variables and the interaction between the input variables are calculated through the SHAP algorithm, and input variable screening is carried out. The prediction precision and the interpretability of the model can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of water quality prediction, and particularly to a multi-water quality factor prediction method based on feature compression and interpretable feature fusion. Background Art

[0002] In view of the low degree of automation in traditional aquaculture workshops, which mainly rely on manual operation of key facilities such as oxygen equipment and water pumps. In addition, the scope of water quality monitoring is limited, making it difficult to comprehensively evaluate the health status of the aquatic environment. Such limitations can pose serious risks, including losses of seafood products and aquaculture failures, as managers may not be able to detect equipment failures in a timely manner or respond promptly to emergencies. Therefore, there is an urgent need for an intelligent aquaculture system that combines historical data from water quality sensors with real-time data streams to accurately predict water quality parameters, helping managers quickly grasp changes in the water environment and efficiently respond to emergencies.

[0003] To solve the water quality prediction problem in aquaculture, researchers around the world have developed various water quality prediction models. Currently, water quality prediction technologies are mainly divided into two categories: mechanism-based models and non-mechanism-based models. Mechanism-based models predict water quality changes by analyzing the relationships between biological, physical, and chemical factors in water bodies. However, due to the highly dynamic nature of the water quality environment in aquaculture, although such models have strong pertinence, they lack accuracy in modeling and predicting multi-factor coupling, and often lack sufficient adaptability and prediction stability in practical applications. In contrast, non-mechanism-based models are usually simpler, relying on statistical analysis and empirical learning to extract features from historical data and then construct models with strong prediction capabilities. Common non-mechanism-based models include time series methods, Markov methods, and function model methods, etc. These models perform well in short-term water quality prediction, but due to the highly non-linear and multi-factor coupling characteristics of the water quality environment in practical applications, it is difficult for them to effectively capture the interaction relationships between multi-factors, and at the same time, it is difficult to support multi-step prediction, resulting in the problem of rapid error accumulation in the prediction process.

[0004] With the continuous development of data acquisition technologies and algorithms, new water quality prediction models driven by computational intelligence have gradually become a research hotspot. Some scholars have proposed using traditional machine learning methods to construct water quality prediction models, such as Support Vector Regression (SVR), Artificial Neural Network (ANN), etc. These methods have certain advantages in dealing with complex non-linear relationships. Singh et al. used ANN to predict the water environment time series indicators of the Gomti River in India, and the results showed that ANN could effectively fit the non-linear relationships in the time series. However, traditional machine learning methods often have difficulty fully mining the complex features of water quality data with long-term dependencies, resulting in limited prediction accuracy. In contrast, deep learning methods can extract more advanced features through multi-level non-linear transformations and show stronger performance in time series modeling. Deep learning models such as Recurrent Neural Network (RNN), Long Short-Term Memory Neural Network (LSTM), and Gated Recurrent Unit (GRU) can effectively capture the time series features of water quality data. Chen Honglei et al. used LSTM to predict water environment data and introduced an attention mechanism to enhance the contribution of key features to the prediction results. However, these models still face problems such as vanishing and exploding gradients, sequential time processing, and insufficient long-term memory capabilities. Especially in the multi-water quality factor prediction task, they usually can only predict short-term trends and are difficult to effectively reflect the long-term evolution law of water quality, resulting in the need to improve the performance of the models in long-time series data prediction. To address the above challenges, the Transformer model has received attention due to its excellent long-term memory ability. Its self-attention mechanism and position encoding can accurately model the dependencies and key features of time series, and improve computational efficiency through parallel computing and reduce memory occupancy. However, due to the perspective limitation of a single model architecture, when directly applying the Transformer to process the original water quality sequence, its global attention mechanism will over-smooth the local spike features caused by sudden events, resulting in prediction lag. This limitation also indicates that a single model is difficult to simultaneously consider global trends and local features, affecting the accuracy and timeliness of prediction.

[0005] In the field of aquaculture, the water environment is usually a complex ecosystem, and traditional single-factor prediction models have many limitations. For example, the incompleteness of data can only provide certain specific data in water quality, making it difficult to comprehensively capture the dynamic changes of the water environment. In response to the deficiencies of traditional single-factor prediction models, many researchers have proposed multi-factor related water quality prediction models. Multi-sensor data fusion technology is one of the earliest multi-modal fusion forms applied in the field of aquaculture. In the data-level fusion method, multiple sensor data are usually directly spliced together or combined into different channels to form fused data, such as direct cascading, weighted summation, etc. Wang et al. proposed an integrated prediction model that fuses multi-source data, combining multi-source data fusion with a hybrid modeling strategy, significantly improving the prediction accuracy and making it highly applicable in different river basins, providing a reliable tool for precise water quality management. Li et al. proposed an adaptive weighted data fusion method that effectively fuses multi-sensor data such as pH value, temperature, and dissolved oxygen, significantly improving the accuracy and robustness of water quality data. In recent years, with the development of deep learning, researchers have further combined multi-sensor data fusion with deep learning algorithms to improve the accuracy of water quality prediction. Tang et al. constructed a multi-source hybrid dataset and used a deep learning model to predict harmful algal bloom events in Lake Okeechobee to monitor and forecast water quality. However, simple splicing or weighted summation will increase the model's computational complexity and lead to information loss, and different water quality parameters contribute differently to the prediction target, so fixed weights may not be able to adapt to dynamic changes. In the feature-level fusion method, the signal features of multiple sensors are mainly extracted through neural networks, and then methods such as feature combination, attention mechanism, convolutional neural network, and graph neural network are used for feature fusion. The feature-level fusion method can extract the key features of sensors and remove interference components to improve the robustness of the fused features. Liu et al. proposed a feature fusion prediction method based on a two-channel convolutional neural network and a long short-term memory attention mechanism. This method parallelly extracts multi-dimensional spatial features of water quality data through a two-channel convolutional neural network, integrates them in the feature fusion layer and then inputs them into the long short-term memory network for training, and introduces an attention mechanism to dynamically strengthen key features, effectively solving the problem of representing complex features of water quality data and providing a new idea for intelligent water quality prediction. However, these current methods still have certain drawbacks. On the one hand, there may be a high degree of correlation between different water quality parameters currently, and the fusion strategy often ignores the physical coupling relationship between sensor features; on the other hand, only features of a certain depth can be utilized, making it difficult to capture other deep information, resulting in the finally fused features not being able to fully reflect important information.Therefore, the current fusion methods face the following challenges: First, the existing fusion methods ignore the physical coupling relationships between sensor features; Second, they are unable to fuse other deep information while adaptively adjusting the importance of different sensor features, resulting in insufficient information fusion; Finally, the processing and analysis of a large amount of data increase the computational complexity and reduce the real-time performance and usability of the model.

[0006] Although existing deep learning models have shown good performance in aquaculture water quality prediction, they generally operate as "black box models" and lack a transparent revelation of the internal decision-making process. The problem of insufficient interpretability limits the credibility and usability of the model in practical applications. Although some studies have introduced post-hoc interpretability techniques such as SHAP and LIME to improve the interpretability of model outputs by quantifying the importance of each water quality factor to the prediction results, these methods still face many limitations. On the one hand, these methods mainly focus on the importance assessment of single variables and are difficult to capture the high-order interactions and non-linear coupling mechanisms between variables; on the other hand, in the face of highly complex and dynamically evolving water quality time series data, existing interpretability methods are difficult to reveal the core variables focused by the model at different prediction stages, resulting in fuzzy identification of key factors and non-traceable behavior patterns of the model. To address the above problems, there is an urgent need to construct an interpretability method that combines the analysis of the internal attention mechanism of the model and post-hoc interpretability evaluation, which can not only dynamically perceive the dependence relationship between features during the model training process, but also further quantify the contributions of each variable and its interactions to the prediction results after the model prediction output, so as to improve the adaptability of the model to complex environmental changes and enhance the transparency and interpretability of its decision-making process.

[0007] In summary, the current water quality prediction methods have limitations in many aspects, mainly reflected in problems such as single model architecture, low accuracy of multi-factor time series modeling, feature extraction and fusion methods, poor interpretability of the model, and disconnection from physical mechanisms. Summary of the Invention

[0008] To solve the above technical problems, the present invention proposes a multi-water quality factor prediction method based on feature compression and interpretable feature fusion, and a combined prediction framework of CNN and Transformer based on efficient feature compression and interpretable feature fusion, which can improve the processing ability and prediction accuracy of multi-dimensional water quality time series data and the interpretability of the model.

[0009] To achieve the above object, the technical solution of the present invention is as follows:

[0010] A multi-water quality factor prediction method based on feature compression and interpretable feature fusion includes the following steps:

[0011] Obtain multi-sensor data collected by several different types of sensors, and calculate physical derivative variables based on the multi-sensor data;

[0012] The multi-sensor data and physically derived variables are used as input variables to train the CS-CNN-Transformer prediction model. Among them, the local features of the input variables are extracted by the CNN module, and the CNN module includes a 2-layer 1D-CNN. The output of each layer of convolution is used as the input of the next layer of convolution after passing through the ReLU activation function and the compressive sensing layer; a multi-layer attention mechanism is introduced to perform adaptive feature fusion on the input local features to obtain fused features; the fused features are input into the Transformer module to output predicted values.

[0013] Determine the evaluation metrics and evaluate the CS-CNN-Transformer prediction model based on the evaluation metrics.

[0014] Through the multi-layer attention mechanism, key features affecting water quality changes are dynamically identified during model training; the contribution degree of input variables and the interaction between input variables are calculated by the SHAP algorithm, and the input variables of the model are screened based on the contribution degree of input variables and the interaction between input variables.

[0015] Preferably, it further includes the following steps: preprocessing the multi-sensor data and physically derived variables.

[0016] Preferably, the physically derived variables include dissolved oxygen saturation, total dissolved solid concentration, and free ammonia concentration.

[0017] Preferably, the compressive sensing layer performs a pooling operation, including the following steps:

[0018] Reshape the feature map output by the convolutional layer and then perform frequency domain conversion using the fast Fourier transform to obtain frequency domain features.

[0019] Randomly sample the frequency domain features using a Bernoulli measurement matrix to generate a compressed signal; extract the real part of the compressed signal and reshape it to obtain compressive sensing features.

[0020] Preferably, the step of introducing a multi-layer attention mechanism to perform adaptive feature fusion on the input local features to obtain fused features specifically includes the following steps:

[0021] Calculate the input local features through the sensor-level attention mechanism, time-level attention mechanism, and space-level attention mechanism respectively to obtain sensor-level features, time features, and space features.

[0022] Concatenate and fuse the sensor-level features, time features, and space features to obtain fused features.

[0023] Preferably, the contribution degree of the input variables is calculated, and the formula is as follows:

[0024]

[0025] Among them, the SHAP value i represents the absolute value of the SHAP value of the i-th input variable, n represents the total number of input variables, and k represents the number of variables after screening.

[0026] Preferably, the interaction between input variables is calculated, and the formula is as follows:

[0027]

[0028] Among them, φ i,j is the SHAP interaction value, F is the set of all features, F\(i,j) represents removing the variables x i and x j from the full set of features F, and the remaining set of variables, S is the subset of all variables that do not include x i and x j , f(S) represents the prediction result using only the variable subset S, f(S∪{i}) represents the prediction result by adding the variable x i to S, f(S∪{j}) represents the prediction result by adding the variable x j to S, and f(S∪{i,j}) represents the prediction result by adding the variables x i and x j to S simultaneously.

[0029] Based on the above technical solutions, the beneficial effects of the present invention are as follows:

[0030] 1) The present invention can improve the modeling ability and prediction accuracy of water quality time series data collected by multiple sensors. In response to the challenges of high data dimension, complex correlations between variables, and strong time dependence in aquaculture water quality prediction, the present invention uses a CNN network to extract local spatial features from data from different sensors, effectively capturing local change patterns and fine-grained features of various water quality factors; at the same time, a Transformer module is introduced to globally model the fused features, using its powerful attention mechanism to assign different weights to information at different times, thereby realizing effective modeling of long-range dependence relationships in the time series. Through the synergistic effect of CNN and Transformer, the present invention can fully explore the non-linear coupling relationships between multiple water quality factors, improve the model's perception ability of the complex dynamic evolution process of multiple variables, and finally achieve joint high-precision prediction of multiple key water quality factors. The present invention not only improves the model's representation ability for complex time series features but also enhances the reference value and scientific nature of the prediction results in actual aquaculture water quality management.

[0031] 2) By introducing compressive sensing technology to replace the pooling operation in traditional convolutional neural networks, the present invention realizes efficient compression and dimensionality reduction of multi-dimensional feature maps and precise encoding, thereby significantly improving the computational efficiency and generalization ability of the model while maintaining the integrity of key feature information. Based on the compressive sensing pooling strategy with a binary observation matrix, the original feature map is encoded in a random measurement manner during the feature extraction process, breaking through the linear dimensionality reduction mechanism of traditional pooling operations and achieving high-fidelity data compression. It can not only effectively reduce the dimensionality of the feature map and reduce the subsequent model calculation burden, but also maintain a more complete information transmission, which helps to improve the prediction accuracy and stability of the overall model. In addition, the measurement matrix in compressive sensing is adjustable, and the present invention also proposes to flexibly select different types of uncorrelated random measurement matrices according to the target characteristics and data distribution of specific prediction tasks to adapt to the data sparsity and structural features in different scenarios, thereby further enhancing the adaptability and generality of the model.

[0032] 3) The present invention proposes a multi-sensor feature fusion method based on multi-level adaptive weight learning and physical relationship modeling, aiming to improve the modeling accuracy and feature fusion effect of multi-source heterogeneous information in multi-factor water quality prediction tasks. First, by fully utilizing the original observation information of multi-sensor data and introducing the physical mechanism knowledge in the aquaculture water body process, derivative variables with clear physical meanings are constructed to enrich the structural and interpretive nature of feature expressions. Secondly, a feature-level fusion framework based on a multi-layer attention mechanism is constructed to realize the dynamic adaptive adjustment of the importance of features at different scales and of different types. And the multi-layer attention mechanism can not only identify the key features that contribute significantly to the prediction task, but also strengthen the correlation modeling between features by introducing physical coupling relationship constraints, avoiding the weakening or masking of key variable information during the feature fusion process. Finally, the present invention effectively solves the problems of feature redundancy, rough fusion, and lack of physical coupling relationship modeling existing in existing multi-sensor feature fusion methods, and significantly improves the accuracy and robustness of the model for multi-factor water quality prediction.

[0033] 4) In order to improve the interpretability analysis and model decision transparency in multi-factor water quality prediction and solve the problem of insufficient credibility of existing "black box models" in practical applications, the present invention first dynamically identifies key influencing factors at different levels during the model training process through a multi-layer attention mechanism, clearly reveals the focus of attention of the model during the prediction process, realizes the transparency and traceability of the internal decision-making path of the model, and thus enhances the interpretability and scientificity of the model decision-making process. In addition, the present invention further introduces the Shapley Additive Explanation algorithm (SHAP) to quantitatively evaluate the impact of input variables on the prediction results, and then screens out key variables and eliminates input factors with low or even redundant contribution degrees. This can not only effectively improve the operation efficiency and generalization ability of the model, but also minimize the construction and operation costs of the model. At the same time, in order to characterize the complex non-linear interaction relationships between multiple water quality factors, SHAP interaction values are used to reveal the possible coupling effects between input variables, so that the model not only has the ability to explain the impact of single variables, but also can insight into the cooperative or restrictive relationships between variables, ensuring that the model can follow reasonable physical or chemical laws during the learning and prediction processes and improving the credibility and stability of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a schematic structural diagram of a multi-water quality factor prediction method based on feature compression and interpretable feature fusion in an embodiment;

[0035] Figure 2 is a block diagram of the calculation principle of multi-sensor data input and physical derivative variables in a multi-water quality factor prediction method based on feature compression and interpretable feature fusion in an embodiment;

[0036] Figure 3 is a block diagram of the principle of realizing adaptive feature fusion by a multi-layer attention mechanism in a multi-water quality factor prediction method based on feature compression and interpretable feature fusion in an embodiment;

[0037] Figure 4 is an overall structural diagram of a CS-CNN-Transformer model in a multi-water quality factor prediction method based on feature compression and interpretable feature fusion in an embodiment;

[0038] Figure 5 is a diagram showing the difference between a black box model and an interpretable artificial intelligence model in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0039] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0040] See Figure 1, this embodiment provides a multi-water quality factor prediction method based on feature compression and interpretable feature fusion, including the following steps:

[0041] Obtain multi-sensor data collected by several different types of sensors, and calculate physical derivative variables based on the multi-sensor data;

[0042] Use the multi-sensor data and physical derivative variables as input variables to train the CS-CNN-Transformer prediction model. Among them, the local features of the input variables are extracted through the CNN module. The CNN module includes a 2-layer 1D-CNN. The output of each layer of convolution is used as the input of the next layer of convolution after passing through the ReLU activation function and the compressive sensing layer; introduce a multi-layer attention mechanism to perform adaptive feature fusion on the input local features to obtain fused features; input the fused features into the Transformer module and output the predicted value;

[0043] Determine the evaluation index, and evaluate the CS-CNN-Transformer prediction model based on the evaluation index;

[0044] Through the multi-layer attention mechanism, dynamically identify the key features affecting water quality changes during the model training process; calculate the contribution degree of the input variables and the interaction between the input variables through the SHAP algorithm, and screen the input variables of the model based on the contribution degree of the input variables and the interaction between the input variables.

[0045] In this embodiment, based on multi-level adaptive weight learning and physical relationship modeling, by introducing physical derivative variables such as dissolved oxygen saturation and total dissolved solid concentration and jointly inputting them with the original multi-sensor data, local features are extracted in the CNN module, and on this basis, multi-level dynamic feature fusion is realized through sensor-level, time-level, and space-level attention mechanisms respectively, so as to improve the global consistency and robustness of the representation; Secondly, to achieve efficient and compressed feature and long-term dependence modeling, a CNN-Transformer combined prediction framework based on compressive sensing is constructed. CNN is used for preliminary feature extraction, and multi-sensor data is dimensionally reduced and encoded by compressive sensing. After multi-level dynamic feature fusion, it is connected to the Transformer for long-term dependence modeling and prediction, taking into account both the model efficiency and the temporal expression ability; Finally, to improve the transparency and credibility of the model, interpretable artificial intelligence is integrated, the SHAP algorithm is used to evaluate the contribution of different input variables in the prediction, reveal the feature contribution structure inside the model, and verify it with the physical mechanism, analyze the interaction between variables, and reveal the hidden variable coupling relationship, so as to achieve accurate prediction and mechanism explanation of the dissolved oxygen change trend. At the same time, a multi-layer attention mechanism is introduced in the model training stage to realize the dynamic identification of key influencing factors, which can automatically capture the focus of attention of the model at each prediction stage at different levels, so as to clearly present the decision logic and feature dependence path inside the model, improve the interpretability of the model behavior, and ensure that the constructed prediction framework not only has high-precision prediction ability, but also has good physical consistency and interpretable scientificity. The detailed technical solution is as follows:

[0046] 1. Multi-sensor feature fusion based on multi-level adaptive weight learning and physical relationship modeling

[0047] (1) Multi-sensor data input and physical derivative variable calculation

[0048] In the task of aquaculture water quality prediction, multi-sensor data contains multiple water quality parameters, such as temperature, dissolved oxygen, pH value, etc. However, since the water environment is a highly dynamic and complex system, the parameters are not independent of each other, but interact through physical, chemical, and biological processes, forming complex non-linear relationships. For example, an increase in temperature will reduce the dissolved oxygen content in water. Facing these challenges, relying solely on the raw data measured by sensors may not be sufficient to build a high-precision prediction model. On the one hand, sensor data often contains noise, missing values, or measurement errors, and preprocessing operations such as data cleaning, standardization, and alignment are required to ensure the consistency and reliability of the data. On the other hand, relying solely on data-driven models cannot fully utilize the existing knowledge of water body dynamics and environmental science. Therefore, it is necessary to calculate derivative variables by combining physical laws to enhance the model's understanding of the complex dynamic changes of the water body. For example, the dissolved oxygen saturation value is calculated using the non-linear relationship between temperature and dissolved oxygen. Physical derivative variables can not only supplement the information gap in sensor data but also be input into the model as additional features, making it more in line with actual physical laws when learning water quality change patterns and improving the accuracy and interpretability of predictions. Therefore, before feature fusion, it is first necessary to preprocess the multi-sensor data to ensure data quality, and calculate derivative variables by combining physical constraints, enabling the model to fully utilize the advantages of combining data-driven and physical priors, laying a solid foundation for subsequent feature fusion and modeling.

[0049] First of all, multi-sensor data usually comes from different types of sensors, with different measurement frequencies, units, and scales. If not preprocessed reasonably, directly using this data for analysis or modeling is likely to cause the model to learn incorrect patterns and affect the prediction accuracy. Therefore, a series of preprocessing operations on multi-sensor data are required, including missing value filling, outlier detection, standardization, and data alignment, etc., to improve the consistency and usability of the data.

[0050] Secondly, since there are physical laws between different water quality parameters (such as temperature and dissolved oxygen showing a non-linear negative correlation), before fusing the raw data measured by sensors, a physical constraint layer is introduced to calculate physical derivative variables by combining domain knowledge. These physical derivative variables will be input into the model as additional features to enhance the understanding of different sensor data, enabling the model to better learn the internal mechanism of water quality changes. The calculation of physical derivative variables usually involves relevant formulas such as dissolved oxygen saturation estimation, total dissolved solid concentration conversion, and free ammonia concentration calculation. The following are several typical calculation methods for physical derivative variables:

[0051] The saturation of dissolved oxygen depends on the maximum dissolved oxygen capacity of the water body. When the temperature rises, the dissolved oxygen capacity of water decreases, resulting in a decrease in the saturated dissolved oxygen value; an increase in salinity also reduces the dissolved oxygen capacity of water. Calculating the dissolved oxygen saturation can provide a basis for whether the water body is in a supersaturated state (prone to cause fish bubble disease) or a hypoxic state (prone to cause the death of aquatic animals due to lack of oxygen). The calculation of dissolved oxygen saturation is shown in Formula 1:

[0052]

[0053] where T is the temperature, Salinity is the salinity, and C0, C1, C2 are empirical constants.

[0054] Conductivity (EC) reflects the concentration of dissolved ions in the water body, such as sodium, potassium, calcium, magnesium and other ions. Through conductivity, the total dissolved solids (TDS) content of the water body can be estimated to judge the degree of water body mineralization. A higher TDS value may lead to excessive mineralization of the water body, affecting the osmoregulation ability of aquatic organisms and thus their growth. A low TDS value may mean insufficient mineral content in the water body, which is not conducive to the growth of organisms. The calculation of total dissolved solids is shown in Formula 2:

[0055] TDS = k EC ×NE (2)

[0056] where TDS is the total dissolved solids, k EC is the empirical conversion coefficient, and EC is the conductivity.

[0057] Total ammonia nitrogen (TAN) includes free ammonia (NH3) and ammonium ions Among them, free ammonia is toxic to aquatic organisms, while ammonium ions are relatively harmless. The existence form of TAN is affected by pH value, temperature and ionic strength, among which the pH value is the most critical factor. When the pH value increases, the proportion of free ammonia increases, thus increasing the toxicity of the water body. Therefore, it is necessary to calculate the concentration of NH3. The calculation of the concentration of free ammonia is shown in Formula 3:

[0058]

[0059] where pKa is the acid-base equilibrium constant of ammonia, and the calculation is shown in Formula 4:

[0060]

[0061] Finally, after completing data preprocessing and calculating physical derived variables, it is necessary to further extract local features from the data to identify short-term dynamic changes and local correlations between different water quality parameters. Since the original sensor data and physical derived variables usually exhibit high dimensionality, temporal characteristics, and multi-dimensional features, using a convolutional neural network to perform local pattern learning on them can effectively capture the water quality change trends on short time scales, extract key features, and perform dimensionality reduction, providing a more representative expression for subsequent feature fusion and prediction. After being processed by the convolutional neural network, the water quality data is converted into a feature vector, which contains multiple local patterns, and each channel represents the short-term dynamic changes of a water quality parameter. The feature vector will be further processed in the subsequent feature fusion stage to integrate the information of different water quality parameters, enhance the modeling ability of the water quality state, and improve the accuracy and stability of the prediction. The principle block diagram of multi-sensor data input and physical derived variable calculation is as Figure 2 shown.

[0062] (2) Multi-layer attention mechanism to achieve adaptive feature fusion

[0063] After completing feature extraction, it is necessary to perform adaptive feature fusion on the extracted features to solve the problem that existing methods lack an adaptive mechanism for features at each level, resulting in the inability to dynamically adjust the contribution degree of a specific feature, weakening some key features, and amplifying irrelevant or redundant features, thereby improving the prediction accuracy of the model. Therefore, a multi-layer attention mechanism is introduced to enable the model to dynamically allocate the importance of features at each level and ensure that key features are fully expressed during the fusion process. The adaptive feature fusion of the multi-layer attention mechanism includes three levels: sensor-level attention, time-level attention, and space-level attention. The attention mechanism at each level weights and adjusts the input features from different perspectives to ensure that key features are fully utilized and irrelevant or redundant information is suppressed, thereby improving the accuracy of water quality prediction.

[0064] ① Sensor-level attention

[0065] Under different environmental conditions, the data contribution degrees of each sensor may vary, and it is necessary to dynamically adjust the importance of features of different sensors to improve the adaptability and robustness of the model. Therefore, an adaptive attention network is introduced to adjust the weights of different sensor features by calculating the attention scores of different sensor features, enabling the model to reasonably allocate the information contributions of each sensor according to the actual situation. In the specific implementation process, first, the self-attention mechanism is used to calculate the correlation between the features of each sensor. Suppose the input data comes from N sensors, and the feature dimension of each sensor is d. After extracting local features through a convolutional neural network, a feature matrix F s ∈R N×d. Subsequently, attention scores are calculated through Query (Q), Key (K), and Value (V). Query, Key, and Value are obtained by linear transformation:

[0066]

[0067] Among them, and are learnable weight matrices, and Q s , K s , V s are the query, key, and value vectors in the sensor-level attention respectively. Then, the correlation between sensors, that is, the attention score, is calculated. The calculation formula is as follows:

[0068]

[0069] Among them, A s ∈R N×N is the normalized attention weight matrix, representing the mutual relationship between different sensors. The purpose of Softmax normalization is to ensure that the weights are distributed between 0 and 1 and sum to 1, enabling the attention mechanism to adaptively adjust the importance of each sensor. Finally, the calculated attention weights are used to perform weighted summation on the features of each sensor to obtain the enhanced sensor-level feature representation:

[0070] Z s =A s V s (7)

[0071] Among them, Z s ∈R N×d is the finally adaptively adjusted feature matrix, enabling the model to adaptively enhance or suppress the information of different sensors according to the changes in the data environment, thereby improving the robustness and adaptability of the model.

[0072] ② Temporal-level attention

[0073] In order to solve the fusion problem of short-term local features and long-term trend features, the time attention layer is introduced to calculate the importance of features at different time steps, so that the model can adaptively adjust the attention to different time steps according to the water quality conditions. The present invention uses Transformer as a time series modeling method to extract the features of the time dimension, and uses the self-attention mechanism of Transformer to calculate the correlation between different time steps. In the time-level attention, the time series features firstly input are passed through Transformer to calculate the dependency between each time step. The self-attention mechanism obtains the attention score by calculating the relationship between Query, Key and Value, thereby determining the importance of different time steps and forming the time step attention weight matrix A. T Then the time step features are weighted and summed according to the attention distribution to obtain the time level feature representation Z T =A T V T , where V T is the value vector in the time-level attention. Finally, the model is enabled to flexibly adjust the focus in different time periods, thereby improving the ability to perceive the dynamic changes in water quality.

[0074] ③Spatial level attention

[0075] Considering the complex spatial dependencies between sensor data at different locations, it is difficult to fully characterize the dynamic changes in water quality by relying solely on the information from a single sensor. In order to solve this problem, the present invention uses a graph convolutional network (GCN) to model the spatial relationship between sensors, calculate the correlation between different sensor locations, and weight spatial features to enhance the expressiveness of spatial features. In the specific implementation process, a sensor network graph is first constructed, the sensors are regarded as nodes in the graph, and the edge weight matrix W is constructed using information such as distance, flow velocity, and pollution diffusion model to represent the spatial correlation between sensors. The relationship between sensors is represented by the adjacency matrix A∈R N×N Then, the graph convolutional network is used to learn the spatial features and perform weighted calculations on the features of different sensor locations so that the sensor information related to the target location can get a greater weight. The basic calculation formula of the graph convolutional network is as follows:

[0076]

[0077] Among them, H (l) is the sensor feature representation of the lth layer, initially H (0) =X; Add self-connection to the adjacency matrix to ensure that each node retains its own information; yes The degree matrix of W (l) is the trainable weight matrix for this layer; σ(·) is the activation function. Finally, after the calculation of the multi-layer graph convolutional network, the model can learn the spatial-level feature Z sp , that is, the weighted relationship between sensors at different positions, enabling the model to more accurately capture the water quality change relationship between different positions, improving the expression ability of spatial features, strengthening the important information in space, and at the same time reducing the interference of irrelevant positions.

[0078] Finally, the construction process of the fused feature integrates the key feature information of each layer extracted by the sensor-level, time-level, and spatial-level attention mechanisms, and jointly models the features of each layer by means of splicing fusion. The obtained fused feature is represented as: X fusion = [Z s || Z T || Z sp , where [·||·||·] represents the splicing operation along the feature dimension. Through the fused feature representation, the model can accurately model and perceive the water quality change in multiple dimensions, improving the overall prediction performance and robustness. Then, the fused feature is input into the Transformer to further model the long-term dependence relationship and improve the prediction ability. The principle block diagram of the multi-layer attention mechanism for adaptive feature fusion is as shown in Figure 3 Figure.

[0079] 2. CNN and Transformer Combined Prediction Framework for Efficient Compression Based on Compressed Sensing

[0080] In order to comprehensively consider the interaction between water quality factors and fully explore the temporal dependence and key features of long time series data, the present invention uses CNN to extract local spatial features and combines the global temporal modeling ability of the Transformer, enabling it to simultaneously capture the spatial structure and temporal evolution law in water quality data and achieve efficient and accurate prediction of multi-dimensional water quality data. In addition, in order to further improve the computational efficiency and robustness of the model, the present invention innovatively proposes a compressed sensing pooling strategy, using the compressed sensing method to replace the pooling layer in the traditional CNN, performing sparse domain representation on the features extracted by the convolutional layer through sparse representation and measurement matrix, and adaptively sampling according to the sparse characteristics of the data to obtain key features under fewer data samplings, effectively reducing the transmission of redundant information, thereby improving the computational efficiency of the model, accelerating the training and inference processes, and achieving more efficient processing of high-dimensional time series data in the feature extraction and information compression processes. Compared with the traditional pooling operation, compressed sensing improves the robustness and adaptability of the model while reducing the computational amount through random measurement, sparse representation, and dynamic adaptation strategies.

[0081] First, the CNN is responsible for extracting local features from the input feature tensor that consists of the original multi-sensor data and physically derived variables. The CNN contains two layers of 1D-CNN. The output of each layer of convolution is passed through the ReLU activation function and the compressive sensing layer and then used as the input for the next layer of convolution. Through layer-by-layer abstraction, features at different levels are captured. Through the convolution operation, the relationship between local patterns and features can be captured simultaneously in the time and feature dimensions. Each layer of the ReLU activation function can increase the non-linear characteristics and avoid the vanishing gradient problem, thus accelerating the convergence speed during the training process. And the compressive sensing layer samples the feature map to capture more structural information in the feature map, realizes feature compression of the convolution output, retains key information and reduces the feature dimension, thereby accelerating subsequent calculations. Subsequently, the fully connected layer further integrates the data features of different sensors extracted, forming a fused feature representation. After the processing of the above layers, the CNN can effectively extract the local features of multi-sensor data and physically derived variables, providing a more compact representation and a more discriminative feature input for the subsequent model.

[0082] In the compressive sensing layer, the model realizes the pooling operation through the following hierarchical structure: ① Reshape the feature map output by the convolution layer to adjust its shape to meet the subsequent processing requirements; ② Use the fast Fourier transform (FFT) to perform a frequency domain transformation on the feature map to obtain a sparse complex representation, thereby enhancing the main feature components of the water quality data in the frequency domain while suppressing redundant information; ③ Use the Bernoulli measurement matrix to randomly sample the frequency domain features, and convert the input signal into a low-dimensional compressed signal through matrix multiplication. The binary encoding (0 or 1) of each sampled value determines whether the signal is collected at that point. The Bernoulli matrix of binary random variables can compress the signal in a random and effective way when sampling the signal, retains the main features of the signal, while greatly reducing the computational and storage complexity, improving the data processing efficiency, and finally generating a compressed complex signal; ④ To simplify subsequent processing, finally extract the real part of the compressed signal and reshape its shape to make it suitable for the subsequent deep learning layer structure. The whole process effectively reduces the data dimension, while retaining key information, enhancing the feature extraction ability for water quality time series data, and enabling the model to have better performance when processing high-dimensional data.

[0083] Secondly, the Transformer is responsible for capturing the long-distance dependencies and global temporal relationships between different sensors. Among them, the self-attention mechanism can model the long-range dependencies in the time series, enabling the model to focus on the information at any time step, thereby extracting the global time patterns and improving the ability to understand complex temporal features. At the same time, the position encoding structure endows the Transformer with the ability to process sequence data, making it independent of the fixed order of execution, so that parallel computing can be achieved, improving the computing efficiency and having good performance when processing long time series.

[0084] Finally, to further model the long-term dependencies and complex non-linear evolution patterns in the dissolved oxygen time series, the present invention inputs the fusion features obtained through the multi-layer attention mechanism into the Transformer structure to predict and model the dissolved oxygen, effectively improving the model's perception ability of the future dissolved oxygen trend changes. The overall structure of the proposed CS-CNN-Transformer model is as Figure 4 shown.

[0085] 3. Efficient, Transparent and Accurate Prediction Based on Explainable Artificial Intelligence

[0086] As Figure 5 shown, it is the difference between the black-box model and the explainable artificial intelligence model. The internal working principle of the black-box model is usually opaque. In model construction, the focus is usually on the input and output of the model, while the specific implementation details are often ignored. Although these models can achieve high-precision predictions, the lack of interpretability makes it complicated to understand the reasons for the high-precision prediction results.

[0087] This embodiment provides a method for enhancing the interpretability of the model based on the multi-layer attention mechanism, aiming to embed the multi-layer attention module in the model structure, and apply different response weights to the input features from three levels: sensor-level attention, time-level attention, and space-level attention. It can not only dynamically identify the importance distribution of different levels of input, but also output the specific weight map during the model training or prediction stage, realizing the visual analysis of "which inputs the model pays attention to and why it makes this prediction", thereby enhancing the interpretability and transparency of the model.

[0088] First, based on multi-sensor data and physically derived variables, the data to be predicted is input into the input layer of the convolutional neural network. Through a series of convolutional operations, pooling operations, and non-linear activation functions, preliminary feature representation encoding is performed on the original data. This not only compresses the dimension of the input data, reduces redundancy, but also extracts underlying features with semantic expression ability. The extracted features retain key structural information in the spatial dimension and at the same time reflect the response distribution of different variables in the channel dimension. Through this process, the model constructs a discriminative feature representation basis, providing the necessary input support for subsequent embedding of multi-layer attention mechanisms and performing feature weighting, thus effectively improving the learning efficiency and performance ability of the model.

[0089] Secondly, the core idea of the attention mechanism is to simulate the human focusing mechanism, enabling the model to apply different response intensities to different input features, thereby highlighting the information that is crucial for the current prediction task. In the present invention, in order to improve the model's ability to identify key features and enhance its discriminability and interpretability under high-dimensional and redundant inputs, a multi-layer attention module composed of sensor-level attention, time-level attention, and space-level attention is constructed and embedded in the model structure. The module realizes adaptive weighted processing of information in the feature dimension, time dimension, and space dimension by modeling and learning the importance of input features at different semantic levels. During the model training process, the network first performs forward propagation, calculates the predicted values of the input samples, and measures the difference between the predicted values and the true labels through a loss function (such as mean squared error, etc.). Subsequently, the error signal is gradually transmitted from the output layer to each layer of the network using backpropagation, and all parameters are updated simultaneously, including the network structure and parameters for generating weights in the multi-layer attention module. As the training iterates, the model gradually learns the importance of input features in completing a specific prediction task and dynamically adjusts the attention distribution accordingly, so that the response weights of key features continue to increase, while the weights of noise or irrelevant features tend to weaken. The whole process not only realizes the self-adaptability of feature weighting, but also prompts the model to focus on the factors that are most explanatory for the prediction task, thereby improving the prediction accuracy and generalization ability of the overall model. In the prediction stage, the multi-layer attention module can output multi-layer weight distribution results, which are used to assist in analyzing the decision basis and path of the model, and then construct an explicit feature response chain from input to output, realizing the transparency and traceability of the model decision-making process. Therefore, the multi-layer attention mechanism not only improves the prediction performance of the model in complex scenarios, but also effectively enhances its result interpretation ability.

[0090] Finally, after the model training is completed, the interpretability of the model decision-making process is enhanced by extracting the weight matrices generated by the multi-layer attention module. The outputs embedded in each layer of the attention mechanism module are called to obtain the attention distribution information of each layer, such as the attention heatmap or the weight tensor. Subsequently, these weight information are mapped back to the original positions of the input features to form a feature response visualization graph, intuitively showing the variable combinations, key time periods, and spatial positions focused by the model during the decision-making process. An explicit feature path chain from the input to the output can also be further constructed to reveal the inference mechanism and logical basis inside the model. The whole process not only makes the prediction results of the model traceable, but also enhances the user's trust in the transparency of the model.

[0091] In addition, based on the important variable screening and interaction analysis method of the SHAP algorithm of the present invention, the SHAP value is used to quantitatively measure the importance of each variable, variables with low contribution to the prediction result are screened out, and the interaction between variables is analyzed through the SHAP interaction value to reveal the hidden variable coupling relationship, making the model have higher interpretability and computational efficiency.

[0092] Based on the model obtained after the training of each module above, the SHAP value of each input variable is calculated using the SHAP algorithm to measure the contribution direction and contribution size of each variable to the prediction result, so as to provide a quantitative basis for variable screening. Subsequently, methods such as the SHAP Summary Plot are used to intuitively display the importance ranking of variables. In order to improve the computational efficiency and prediction accuracy of the model, variables with low contribution are removed through a contribution threshold (such as a subset of variables when the cumulative contribution reaches a certain value) to reduce the computational overhead. The contribution threshold is calculated based on the SHAP value and is used to measure the importance of variables to the prediction result. First, calculate the SHAP value of each input variable and sort them in descending order according to the importance of the variables (the average absolute value of the SHAP value); then, calculate the cumulative contribution of the variable importance, that is, start from the most important variable and accumulate its SHAP contribution in turn until the set contribution threshold is met. The specific calculation is shown in Formula 9; finally, according to the calculated cumulative contribution, variables with low contribution are removed, that is, variables outside the threshold range. The SHAP values of these variables are low and contribute little to the prediction ability of the model. Removing them can reduce the computational overhead, improve the running efficiency of the model, and at the same time reduce the model complexity and the risk of overfitting. In order to ensure that the selected variable subset can not only represent the important features of the data, but also improve the computational efficiency and prediction accuracy of the model, after the variable screening is completed, the model is retrained using the selected variable subset and compared with the original model for evaluation.

[0093]

[0094] Among them, |SHAP value i| represents the absolute value of the SHAP value of the i-th input variable, n represents the total number of input variables, and k represents the number of variables after screening.

[0095] After variable screening, the selected variable subset represents the core variables with a high contribution to the prediction result. However, the importance of a single variable cannot fully reveal the interaction relationship between variables. Variables in many complex systems are not independent but have non-linear coupling relationships. Therefore, to further optimize the model and ensure that the selected variable subset can reasonably explain the prediction result, SHAP interaction values are used to measure the synergy or coupling degree between variables. In the process of calculating SHAP interaction values, the key lies in measuring whether the interaction effect between two variables is significant and how they jointly act on the prediction result. First, different variable combinations are constructed, and all possible variable subsets S (the variable set S refers to all variable subsets that do not contain x i and x j ) are traversed, and the model prediction values for each subset under different variable combinations are calculated, that is, the model prediction outputs for the following four cases are calculated respectively:

[0096] f(S): The prediction result using only the variable subset S

[0097] f(S∪{i}): The prediction result after adding the variable x i to S

[0098] f(S∪{j}): The prediction result after adding the variable x j to S

[0099] f(S∪{i,j}): The prediction result after adding the variables x i and x j to S simultaneously. Then the obtained prediction values are used for subsequent calculation of the marginal interaction contribution between two variables.

[0100] Secondly, calculate the marginal interaction contribution of each pair of variables. The interaction contribution between variables refers to the difference between the individual and synergistic effects of the variables x i and x j , and analyze whether it has an additional impact on the prediction result. The calculation of the marginal interaction contribution is shown in Equation 10:

[0101] Δf = f(S∪{i,j}) - f(S∪(i)) - f(S∪(j)) + f(S) (10)

[0102] Since there may be multiple combinations of the variable subset S, all S need to be weighted and summed during the calculation to fairly measure all possible variable combinations. Therefore, to ensure the fairness of the calculation, the interaction contributions Δf of all variable subsets S are weighted and summed to obtain the final SHAP interaction value, and its mathematical model is shown in Equation 11:

[0103]

[0104] where F is the set of all features, and F\(i,j) represents the set of variables remaining after removing variables x i and x j from the full set of features F. S is the subset of all variables that do not contain x i and x j . The normalization coefficient ensures that the contributions of different variable combinations to the final interaction value are fair, which conforms to the "symmetry" principle of the SHAP theory, that is, the adjustment of the variable order does not affect the final result.

[0105] Finally, after calculating the SHAP contribution and the SHAP interaction value, the input variable selection of the model is optimized according to the importance of the variables (SHAP value) and the interaction relationship between the variables (SHAP interaction value). Through the following four cases, the processing methods of the subsequent model and how to adjust the input variables for training are explored to improve the calculation efficiency and prediction ability of the model.

[0106] (1) High SHAP contribution + strong SHAP interaction value

[0107] These variables not only have a high contribution to the prediction result (high SHAP value), but also have a significant interaction with other variables (high SHAP interaction value). This means that they can independently affect the model prediction result and further improve the prediction performance under the synergistic effect between variables. Therefore, they should be preferentially retained. At the same time, consider adding their interaction features (such as product term interaction feature extraction) when inputting into the model.

[0108] (2) High SHAP contribution + low SHAP interaction value

[0109] These variables have an important impact on the model prediction result (high SHAP value), but have a weak interaction with other variables (low SHAP interaction value), indicating that they can independently affect the model without relying on other variables. Only the variables need to be used as the model input, and no additional interaction features need to be constructed.

[0110] (3) Low SHAP contribution + strong SHAP interaction value

[0111] In this case, the contribution of these variables is low (low SHAP values), but they may have a significant impact on the prediction results under certain combinations (high SHAP interaction values). Therefore, it is decided whether to retain these variables based on the importance of the interaction objects. If a variable has a low contribution by itself but performs well when interacting with other variables with high contributions, it can be considered as part of the interaction features instead of being input separately.

[0112] (4) Low SHAP contribution + low SHAP interaction value

[0113] These variables not only have a low contribution to the prediction results (low SHAP values) but also have weak interaction with other variables (low SHAP interaction values), indicating that they are not substantially helpful for model training and may even introduce noise. Therefore, they should be removed to reduce the computational burden and lower the model complexity.

[0114] In summary, by dynamically identifying key influencing factors at different levels through a multi-layer attention mechanism, the focus of attention during the prediction process of the model is revealed, thereby achieving the transparency and traceability of the internal decision-making path of the model and enhancing the interpretability of the model decision-making process. In addition, the SHAP algorithm not only provides an interpretable variable screening method but also effectively reveals the interaction relationships between variables. By combining the analysis of SHAP contributions and SHAP interaction values to optimize the input variables, redundancy is reduced, the computational efficiency of the model is improved, and the interpretability and predictive ability of the model are enhanced. This method can ensure that the key variables of the model are fully utilized while reducing the computational cost, providing strong support for building an efficient and stable prediction model.

[0115] It should be understood that although the steps in the above flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise clearly stated in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the above flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential either, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0116] The above are only the preferred embodiments of the embodiments of the present application and are not used to limit the embodiments of the present application. For those skilled in the art, various changes and modifications can be made to the embodiments of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present application shall be included in the protection scope of the embodiments of the present application.

Claims

1. A multi-water quality factor prediction method based on feature compression and interpretable feature fusion, characterized in that: The steps include: Acquire multi-sensor data collected by several different types of sensors, and calculate physical derivative variables based on the multi-sensor data; The CS-CNN-Transformer prediction model is trained using multi-sensor data and physical derived variables as input variables, wherein the local features of the input variables are extracted through a CNN module, wherein the CNN module includes a 2-layer 1D-CNN, and the output of each convolution layer is used as the input of the next convolution layer after passing through a ReLU activation function and a compressed sensing layer; a multi-layer attention mechanism is introduced to perform adaptive feature fusion on the local features of the input to obtain fused features; the fused features are input into the Transformer module to output the predicted value; Determine the evaluation indicators and evaluate the CS-CNN-Transformer prediction model based on the evaluation indicators; Through the multi-layer attention mechanism, the key features that affect water quality changes are dynamically identified during the model training process; the SHAP algorithm is used to calculate the contribution of input variables and the interaction between input variables, and the input variables of the model are screened based on the contribution of input variables and the interaction between input variables.

2. The method for predicting multiple water quality factors based on feature compression and interpretable feature fusion according to claim 1 is characterized in that: The method also includes the following steps: preprocessing the multi-sensor data and the physically derived variables.

3. The method for predicting multiple water quality factors based on feature compression and interpretable feature fusion according to claim 1 is characterized in that: The physically derived variables include dissolved oxygen saturation, total dissolved solids concentration, and free ammonia concentration.

4. The method for predicting multiple water quality factors based on feature compression and interpretable feature fusion according to claim 1 is characterized in that: The compressed sensing layer performs pooling operations, including the following steps: After reshaping the feature map output by the convolutional layer, the frequency domain is converted using the fast Fourier transform to obtain the frequency domain features; The Bernoulli measurement matrix is ​​used to randomly sample the frequency domain features to generate a compressed signal; the real part of the compressed signal is extracted and reshaped to obtain the compressed sensing features.

5. The method for predicting multiple water quality factors based on feature compression and interpretable feature fusion according to claim 1 is characterized in that: The multi-layer attention mechanism is introduced to adaptively fuse the local features of the input to obtain fused features, which specifically includes the following steps: The input local features are calculated through the sensor-level attention mechanism, the time-level attention mechanism and the space-level attention mechanism to obtain the sensor-level features, the time-level features and the space-level features respectively; The sensor-level features, time features and space features are concatenated and fused to obtain fused features.

6. The method for predicting multiple water quality factors based on feature compression and interpretable feature fusion according to claim 1 is characterized in that: Calculate the contribution of input variables, the formula is as follows: Among them, |SHAP value i | represents the absolute value of the SHAP value of the i-th input variable, n represents the total number of input variables, and k represents the number of variables after screening.

7. The method for predicting multiple water quality factors based on feature compression and interpretable feature fusion according to claim 6 is characterized in that: The interaction between input variables is calculated as follows: Among them, φ i,j is the SHAP interaction value, F is the set of all features, and F\(i,j) means removing the variable x from the full set of features F. i and x j The remaining set of variables, S, is all variables that do not contain x i and x j f(S) represents the prediction result using only the variable subset S, and f(S∪{i}) represents adding variable x to S. i The prediction result, f(S∪{j}) means adding variable x on the basis of S j The prediction result, f(S∪{i,j}) means adding variable x to S at the same time i and x j prediction results.

Images