Water quality data online detection and analysis method for smart water affairs

Abstract

This invention relates to the field of data analysis, and more particularly to an online water quality data detection and analysis method for smart water management. The method includes: acquiring a water quality prediction model by collecting, preprocessing, and training water quality-related data; obtaining the optimal clustering result by performing cluster analysis and evaluation on historical water quality data; obtaining the true anomaly confidence score by mapping data vectors with deviations exceeding a threshold to the optimal clustering result and performing trend analysis; obtaining the true anomaly confidence index by comprehensively judging the true anomaly confidence score; and obtaining the water quality anomaly detection result by jointly judging the prediction deviation and the true anomaly confidence index. This addresses the problem that existing water quality monitoring methods based on LSTM prediction deviation thresholds cannot distinguish between true water quality anomalies and prediction errors caused by changes in the relationship between indicators.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, and in particular to an online detection and analysis method for water quality data in smart water management. Background Technology

[0002] In smart water management applications, online water quality monitoring equipment is typically deployed at water sources, water supply networks, or water treatment facilities to continuously collect data on multiple water quality indicators, such as chemical oxygen demand (COD), dissolved oxygen (DO), pH, turbidity, and ammonia nitrogen. The acquired multi-indicator time-series data is then used for online analysis of water quality status. Existing technologies utilize Long Short-Term Memory (LSTM) networks to model multi-indicator water quality data. This typically uses historical monitoring data for indicators such as dissolved oxygen, pH, turbidity, and ammonia nitrogen as input features, and COD monitoring data as the prediction target. A water quality prediction model is then trained. During actual operation, the currently collected multi-indicator water quality data is input into the prediction model to obtain the corresponding predicted COD value. This predicted COD value is then compared with the actual monitored value. When the deviation exceeds a preset range, the current water quality status is considered potentially abnormal, thus achieving online detection and analysis of water quality data.

[0003] However, since LSTM models are based on the correlations between various water quality indicators over historical periods, the relationships between these indicators can change due to factors such as water source switching, changes in pollutant input, seasonal fluctuations, and adjustments to water treatment processes during actual water operations. In such cases, even if no actual anomalies are observed in the water body, the model may not be sufficiently adapted to the new relationship patterns, leading to significant discrepancies between the predicted and actual chemical oxygen demand (COD) values. This can cause the system to misinterpret prediction deviations caused by changes in indicator relationships or model errors as actual water quality anomalies, resulting in false alarms and affecting the accuracy and reliability of online water quality monitoring results. Therefore, how to further distinguish whether prediction deviations originate from actual water quality anomalies or from changes in indicator relationships or model errors, based on online monitoring using water quality prediction models, has become a pressing technical problem that needs to be solved. Summary of the Invention

[0004] In view of this, the present invention aims to propose an online detection and analysis method for water quality data in smart water management, so as to solve the problem that existing water quality monitoring methods based on LSTM prediction deviation thresholds cannot distinguish between actual water quality anomalies and prediction errors caused by changes in the relationship between indicators.

[0005] To achieve the above objectives, the technical solution of the present invention is implemented as follows:

[0006] A method for online detection and analysis of water quality data for smart water management, the method comprising:

[0007] Step S1: Obtain a water quality prediction model by collecting, preprocessing, and training water quality-related data;

[0008] Step S2: Obtain the optimal clustering result by performing cluster analysis and evaluation on historical water quality data;

[0009] Step S3: Obtain the true anomaly confidence score by mapping data vectors with deviations exceeding the threshold to the optimal clustering results and performing trend analysis;

[0010] Step S4: Obtain the true anomaly confidence index by comprehensively judging the true anomaly confidence level;

[0011] Step S5: Obtain the water quality anomaly detection results by jointly judging the prediction bias and the true anomaly confidence index.

[0012] Furthermore, the process of acquiring a water quality prediction model through the collection, preprocessing, and model training of water quality-related data includes:

[0013] Online water quality monitoring equipment is deployed in water sources, water transmission networks, or water treatment facilities. A sampling cycle is set, and the water body is continuously collected according to the sampling cycle to obtain water quality-related data. The water quality-related data includes at least chemical oxygen demand monitoring data, dissolved oxygen monitoring data, pH value monitoring data, turbidity monitoring data, and ammonia nitrogen monitoring data.

[0014] The chemical oxygen demand (COD), dissolved oxygen (DO), pH, turbidity, and ammonia nitrogen (AM) monitoring data are processed by filling missing values, removing outliers, and aligning time series data. The processed COD, DO, pH, turbidity, and AM monitoring data are then standardized or normalized to obtain historical water quality data.

[0015] The dissolved oxygen monitoring data, pH monitoring data, turbidity monitoring data, and ammonia nitrogen monitoring data are used as input features, and the chemical oxygen demand monitoring data at the time points corresponding to the input features are used as prediction targets. The Long Short-Term Memory (LSTM) network is trained to obtain a water quality prediction model.

[0016] Furthermore, the step of obtaining the optimal clustering result by performing cluster analysis and evaluation on historical water quality data includes:

[0017] Candidate clustering result set was obtained by performing candidate cluster number analysis and clustering processing on historical water quality data;

[0018] The optimal clustering result is obtained by performing clustering evaluation on the candidate clustering result set.

[0019] Furthermore, the step of obtaining a candidate clustering result set by performing candidate cluster number analysis and clustering processing on historical water quality data includes:

[0020] For any target time point in the historical water quality data, the chemical oxygen demand monitoring data, dissolved oxygen monitoring data, pH value monitoring data, turbidity monitoring data and ammonia nitrogen monitoring data corresponding to the target time point are combined to obtain the historical water quality data vector corresponding to the target time point, and the set of all historical water quality data vectors corresponding to the target time points is used as the historical water quality data vector set for cluster analysis.

[0021] Obtain the number of water quality data index types corresponding to the historical water quality data vector set, and use the number of water quality data index types as the maximum candidate cluster number to construct a candidate cluster number set consisting of 1 to the maximum candidate cluster number; for any candidate cluster number in the candidate cluster number set, use the Gap Statistic algorithm to analyze the historical water quality data vector set to obtain the Gap value under the corresponding candidate cluster number; determine the candidate cluster number with the largest Gap value as the ideal cluster number;

[0022] A range of candidate cluster numbers is constructed with the ideal cluster number as the center; for any candidate cluster number in the range of candidate cluster numbers, the historical water quality data vector set is clustered using the K-means clustering algorithm to obtain the candidate clustering results under the corresponding candidate cluster number; the set of candidate clustering results corresponding to all candidate cluster numbers in the range of candidate cluster numbers is taken as the candidate clustering result set.

[0023] Furthermore, the step of obtaining the optimal clustering result by performing clustering evaluation processing on the candidate clustering result set includes:

[0024] For any candidate clustering result in the candidate clustering result set, obtain the number of candidate clusters, the total number of samples, and the number of samples in each cluster corresponding to the candidate clustering result; take the minimum number of samples in each cluster as the minimum number of cluster samples, and take the result of dividing the minimum number of cluster samples by the total number of samples as the minimum cluster proportion evaluation.

[0025] For any candidate clustering result in the candidate clustering result set, a fine cluster partitioning coefficient is set, and clusters whose sample size is greater than the product of the fine cluster partitioning coefficient and the total number of samples are determined as valid clusters; the mean number of cluster samples corresponding to all valid clusters is obtained; for any valid cluster, the absolute value of the difference between the number of cluster samples of the valid cluster and the mean number of cluster samples is used as the corresponding cluster size deviation assessment; the negative of the calculation result of the sum of the cluster size deviation assessments corresponding to all valid clusters is subjected to exponential mapping with the natural constant as the base, and the corresponding mapping result is used as the cluster size consistency assessment;

[0026] The result of adding the minimum cluster proportion assessment and the cluster size consistency assessment is used as the cluster size rationality factor corresponding to the candidate clustering result;

[0027] For any candidate clustering result in the candidate clustering result set, obtain the Gap value under the corresponding number of candidate clusters for the candidate clustering result; map the Gap value using a hyperbolic tangent function and add it to a constant 1, then divide the sum by a constant 2 to obtain the Gap value normalization evaluation; divide the cluster size rationality factor by a constant 2 to obtain the cluster size rationality enhancement evaluation; multiply the Gap value normalization evaluation and the cluster size rationality enhancement evaluation by their respective influence weights and then add them to obtain the clustering result evaluation score corresponding to the candidate clustering result;

[0028] The evaluation scores of all candidate clustering results in the candidate clustering result set are compared, and the candidate clustering result with the highest evaluation score is determined as the optimal clustering result.

[0029] Furthermore, the step of mapping data vectors with deviations exceeding a threshold to the optimal clustering result and performing trend analysis to obtain the true anomaly confidence score includes:

[0030] By performing clustering mapping on data vectors with deviations exceeding the threshold, the target's cluster set can be obtained;

[0031] By performing typical anomaly matching analysis on the target cluster data, the confidence level of typical real anomalies is obtained.

[0032] By performing trend continuity analysis on local residual sequence data, the confidence level of atypical true anomalies can be obtained.

[0033] Furthermore, the step of obtaining the target cluster set by performing clustering mapping processing on data vectors with deviations exceeding the threshold includes:

[0034] For any target time, acquire the chemical oxygen demand (COD), dissolved oxygen (DO), pH, turbidity, and ammonia nitrogen (AM) monitoring data corresponding to the target time, and combine these data to obtain a data vector corresponding to the target time. Input the data vector corresponding to the target time into a water quality prediction model to obtain predicted COD data corresponding to the target time.

[0035] The difference between the chemical oxygen demand (COD) monitoring data and the COD prediction data at the target time is taken as the prediction deviation at the target time. The absolute value of the prediction deviation is compared with a preset deviation threshold. When the absolute value of the prediction deviation is greater than the preset deviation threshold, the data vector at the target time is determined as the data vector with deviation exceeding the threshold.

[0036] For any data vector whose deviation exceeds the threshold, obtain the Euclidean distance between the data vector whose deviation exceeds the threshold and the corresponding cluster centers of each cluster in the optimal clustering result; determine the cluster corresponding to the cluster center with the smallest Euclidean distance as the target cluster set to which the data vector whose deviation exceeds the threshold belongs.

[0037] Furthermore, the step of obtaining typical true anomaly confidence scores by performing typical anomaly matching analysis on the target cluster data includes:

[0038] For any data vector whose deviation exceeds the threshold, obtain the number of cluster samples in the target cluster set corresponding to the data vector whose deviation exceeds the threshold, and obtain the total number of samples and the number of candidate clusters corresponding to the optimal clustering result; divide the total number of samples by the number of candidate clusters as the average number of cluster samples; use the difference between the number of cluster samples and the average number of cluster samples as the cluster size difference assessment.

[0039] For any data vector with deviation exceeding the threshold, obtain the number of cluster samples corresponding to all clusters in the optimal clustering result; take the maximum value among all cluster sample numbers as the maximum cluster sample number, and take the minimum value among all cluster sample numbers as the minimum cluster sample number; take the difference between the maximum and minimum cluster sample numbers as the cluster size normalization scale; add the cluster size difference assessment divided by the cluster size normalization scale calculation result to the constant 1, and multiply the sum by one-half to obtain the cluster size normalization assessment; take the difference between the constant 1 and the cluster size normalization assessment as the cluster size anomaly tendency assessment.

[0040] For any data vector whose deviation exceeds the threshold, obtain the Euclidean distance between the data vector whose deviation exceeds the threshold and the cluster center of the target cluster; perform an exponential mapping with the natural constant as the base on the negative of the Euclidean distance, and use the corresponding mapping result as the cluster center proximity assessment;

[0041] The result of adding the cluster size anomaly tendency assessment and the cluster center proximity assessment is used as the typical true anomaly confidence level corresponding to the data vector with deviation exceeding the threshold.

[0042] Furthermore, the step of obtaining the confidence level of atypical true anomalies by performing trend continuity analysis on the local residual sequence data includes:

[0043] For any data vector whose deviation exceeds the threshold, the prediction deviation corresponding to each time point within a local time range ending at the target time is obtained, and a local residual sequence is constructed; a linear fit is performed based on the local residual sequence to obtain the corresponding residual fitting line, and the slope of the residual fitting line is obtained.

[0044] For any residual data in the local residual sequence, the vertical distance from the corresponding point of the residual data to the residual fitting line is obtained, and the sum of the vertical distances from the corresponding points of all residual data in the local residual sequence to the residual fitting line is used as the trend deviation assessment; when the slope is not equal to 0, the trend direction function is set to 1; when the slope is equal to 0, the trend direction function is set to 0; the negative number of the trend deviation assessment is subjected to an exponential mapping with the natural constant as the base, and the mapping result is multiplied by the trend direction function to obtain the trend continuity assessment;

[0045] For the local residual sequence, the first-order autocorrelation coefficient between the residual data at adjacent time points is obtained; the first-order autocorrelation coefficient is added to constant 1 and then divided by constant 2 to obtain the autocorrelation normalization evaluation.

[0046] The result of adding the trend continuity assessment and the autocorrelation normalization assessment is used as the confidence level of the atypical true anomaly corresponding to the data vector with deviation exceeding the threshold.

[0047] Furthermore, the step of obtaining the true anomaly confidence index by comprehensively judging the true anomaly confidence level includes:

[0048] For any data vector whose deviation exceeds the threshold, obtain the typical true anomaly confidence level and the atypical true anomaly confidence level corresponding to the data vector whose deviation exceeds the threshold; divide the typical true anomaly confidence level by a constant 2 to obtain the typical true anomaly normalized assessment; divide the atypical true anomaly confidence level by a constant 2 to obtain the atypical true anomaly normalized assessment.

[0049] The normalized assessment of typical true anomalies and the normalized assessment of atypical true anomalies are compared, and the larger value between the normalized assessment of typical true anomalies and the normalized assessment of atypical true anomalies is determined as the true anomaly confidence index corresponding to the data vector whose deviation exceeds the threshold.

[0050] Compared with the prior art, the present invention has the following advantages:

[0051] The online water quality data detection and analysis method for smart water management described in this invention combines the time-series prediction results of multiple water quality indicators with historical state clustering structures. During real-time monitoring, it no longer relies solely on the magnitude of the deviation between predicted and measured values ​​of key indicators to make anomaly judgments. Instead, it further considers the distribution characteristics of the target data in the historical state space to determine whether the deviation possesses genuine anomaly attributes. In this way, even if the correlation between different water quality indicators changes and leads to increased model prediction errors in real-world scenarios such as water source fluctuations, water treatment process switching, or adjustments in operating conditions, the system can reduce the likelihood of misjudging such relationship drift as genuine water quality anomalies. This effectively reduces false alarms and improves the accuracy and stability of online water quality monitoring results.

[0052] Furthermore, this invention addresses novel anomalies not previously observed by introducing analysis of the continuous variation characteristics of local residuals. This enables the system to identify not only anomalies matching existing anomaly patterns but also atypical anomalies exhibiting continuous evolutionary trends. Therefore, in practical water supply safety monitoring and water environment management, this invention maintains the ability to rapidly identify known abnormal water quality states while enhancing the ability to perceive newly emerging anomalies. This improves the comprehensiveness, reliability, and engineering adaptability of anomaly identification, providing more reliable technical support for water source safety management, water treatment control, and anomaly early warning. Attached Figure Description

[0053] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0054] Figure 1 This is a flowchart of the online water quality data detection and analysis method for smart water management as described in an embodiment of the present invention. Detailed Implementation

[0055] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0056] See Figure 1 This is a flowchart of the online water quality data detection and analysis method for smart water management provided in Embodiment 1 of the present invention, as follows: Figure 1 As shown, online water quality data detection and analysis methods for smart water management may include:

[0057] Step S1: Obtain a water quality prediction model by collecting, preprocessing, and training water quality-related data.

[0058] First, online water quality monitoring equipment is deployed in water sources, water transmission networks, or water treatment facilities. A sampling cycle is set, and the water body is continuously collected according to the sampling cycle to obtain water quality-related data. The water quality-related data includes at least chemical oxygen demand monitoring data, dissolved oxygen monitoring data, pH value monitoring data, turbidity monitoring data, and ammonia nitrogen monitoring data.

[0059] The chemical oxygen demand (COD), dissolved oxygen (DO), pH, turbidity, and ammonia nitrogen (AM) monitoring data were processed by filling missing values, removing outliers, and aligning the data with the time series. The processed COD, DO, pH, turbidity, and AM monitoring data were then standardized or normalized to obtain historical water quality data.

[0060] The dissolved oxygen monitoring data, pH monitoring data, turbidity monitoring data, and ammonia nitrogen monitoring data are used as input features, and the chemical oxygen demand monitoring data at the time points corresponding to the input features are used as prediction targets. The Long Short-Term Memory (LSTM) network is trained to obtain a water quality prediction model.

[0061] It should be noted that, in this embodiment of the invention, the sampling period is set to once every 5 minutes. During the missing value imputation process, if the missing value is an isolated missing value or a short-term consecutive missing value, linear interpolation is used for imputation. If the consecutive missing value length exceeds 3 sampling periods, the average of forward imputation and backward imputation is used for compensation. During the outlier removal process, for each type of monitoring indicator, its mean and standard deviation are calculated. Data points deviating from the mean by more than 3 times the standard deviation are identified as outliers and removed. In the normalization process, the maximum-minimum value normalization strategy is preferably used.

[0062] This completes the process of acquiring a water quality prediction model through the collection, preprocessing, and training of water quality-related data.

[0063] Step S2: Obtain the optimal clustering result by performing cluster analysis and cluster evaluation on historical water quality data.

[0064] After completing the collection, preprocessing, and construction of water quality prediction models, simply relying on these models to predict chemical oxygen demand (COD) based on real-time input data is insufficient to further identify the sources of prediction deviations. In the actual operation of smart water systems, water quality is not static but exhibits various relatively stable yet distinct operating states due to factors such as water source switching, upstream water disturbances, changes in pollutant input, and adjustments to water treatment processes. For historical monitoring data, the multi-indicator combinations formed over different time periods correspond to different water quality state patterns. These patterns may include long-term stable routine operating states, short-term fluctuations, operating condition switching, and historically abnormal states. If these historical states cannot be structurally summarized, and all historical water quality data is treated as homogeneous data, it becomes difficult to determine whether subsequent data with prediction deviations exceeding thresholds matches a specific historical state or deviates from the historical state space to form a new deviation. In the online water quality monitoring scenario addressed by this invention, it is necessary to map the data vectors whose predicted deviations exceed the threshold to the historical state space to analyze their relationship with typical historical states. Therefore, it is essential to first construct a clustering result based on historical water quality data that characterizes the distribution features of historical water quality states, allowing discrete time-series monitoring data to be categorized into several representative state clusters within the clustering space. This approach serves two purposes: firstly, it fixes recurring state patterns from historical water quality operation into cluster structures, providing a state reference for subsequent online monitoring; secondly, it provides a spatial basis for identifying historically occurring abnormal states, elevating the judgment of deviation sources beyond simple numerical deviations to an analysis of "state attribution relationships."

[0065] However, when performing cluster analysis on historical water quality data, directly using the traditional K-means clustering method presents a key problem: the number of clusters needs to be pre-set, and this selection directly affects the representation of historical state patterns. If the number of clusters is too small, different water quality states that should be distinguished will be compressed into the same cluster, resulting in an overly coarse representation of historical states, making it difficult to identify subtle but meaningful state differences later. Conversely, if the number of clusters is too large, the originally continuously changing water quality states are easily over-segmented, forming numerous small clusters or even fragmented clusters with very few samples. While the clustering results may appear more refined in form, they lack stability in practical terms, making them unsuitable as a reference state space for subsequent online monitoring. Especially in smart water management scenarios, water quality changes are typically influenced by both hydrodynamic processes and treatment technologies, often exhibiting continuous evolution over a certain period. Therefore, the corresponding historical state clusters should generally have a relatively stable scale and structure in terms of sample distribution, rather than a large number of isolated fragmented clusters with no practical significance. In existing technologies, while the Gap Statistic algorithm can be used to evaluate the number of candidate clusters to compare the significant differences in cluster structure relative to random distribution under different cluster numbers, this method mainly reflects whether there is a statistically significant trend of separation in the overall cluster structure, and it is difficult to further reflect whether the clustering results are reasonable at the level of cluster size distribution. In other words, based solely on the Gap value, although it can initially identify that the data has good structural separation under a certain cluster number setting, it may still result in clustering results with extremely unbalanced cluster sizes. For example, a small number of anomalous samples may be over-segmented into multiple tiny clusters, or local continuous states may be unnecessarily broken down, which may cause the cluster space to lose its ability to truly represent historical typical states. If such clustering results are directly used as the basis for subsequent mapping, it will reduce the reliability of the analysis of data vector attribution for data exceeding the deviation threshold, thereby affecting the stability of the entire real anomaly identification process.

[0066] Therefore, in this invention, it is necessary to consider not only the saliency of the cluster structure under different numbers of candidate clusters during the historical water quality data clustering process, but also the rationality of the sample quantity distribution of each cluster. This ensures that the final clustering result can reflect the objective separation characteristics between historical water quality states while avoiding excessive splitting due to improper cluster number settings. Only by simultaneously considering structural separability and cluster size stability can the obtained clustering result truly serve as an effective structured representation of historical water quality states, providing a reliable foundation for subsequent clustering mapping of deviation exceeding the threshold, identification of typical anomalies, and determination of confidence in true anomalies.

[0067] In summary, this invention first performs candidate cluster number analysis and clustering on historical water quality data to obtain a candidate clustering result set. Specifically, for any target time point in the historical water quality data, the chemical oxygen demand (COD), dissolved oxygen (DO), pH, turbidity, and ammonia nitrogen (MN) monitoring data corresponding to the target time point are combined to obtain a historical water quality data vector corresponding to the target time point. The set of all historical water quality data vectors corresponding to all target time points is used as the historical water quality data vector set for cluster analysis. The number of water quality data index types corresponding to the historical water quality data vector set is obtained, and the number of water quality data index types is used as the maximum candidate cluster number. A candidate cluster number set consisting of 1 to the maximum candidate cluster number is constructed. Preferably, in this embodiment of the invention, the maximum candidate cluster number is set to 20. For any candidate cluster number in the candidate cluster number set, the Gap Statistic algorithm is used to analyze the historical water quality data vector set to obtain the Gap value under the corresponding candidate cluster number. The candidate cluster number with the largest Gap value is determined as the ideal cluster number. A range of candidate cluster numbers is constructed with the ideal cluster number as the center; for any candidate cluster number in the range of candidate cluster numbers, the historical water quality data vector set is clustered using the K-means clustering algorithm to obtain the candidate clustering results under the corresponding candidate cluster number; the set of candidate clustering results corresponding to all candidate cluster numbers in the range of candidate cluster numbers is taken as the candidate clustering result set.

[0068] After obtaining the candidate clustering result set, the clustering evaluation process is further performed on the candidate clustering result set to obtain the optimal clustering result. Specifically, for any candidate clustering result in the candidate clustering result set, the number of candidate clusters, the total number of samples, and the number of samples in each cluster are obtained. The minimum number of samples in each cluster is taken as the minimum cluster sample number, and the result of dividing the minimum cluster sample number by the total number of samples is used as the minimum cluster proportion evaluation. For any candidate clustering result in the candidate clustering result set, a fine cluster partitioning coefficient is set, and clusters whose sample number is greater than the product of the fine cluster partitioning coefficient and the total number of samples are determined as valid clusters. Preferably, in this embodiment of the invention, the fine cluster partitioning coefficient is set to 0.01, indicating that when the number of samples in a cluster does not exceed 1% of the total number of samples, the cluster is identified as a fine cluster. Obtain the average number of cluster samples corresponding to all valid clusters; for any valid cluster, use the absolute value of the difference between the number of cluster samples of that valid cluster and the average number of cluster samples as the corresponding cluster size deviation assessment; perform an exponential mapping with the natural constant as the base of the sum of the cluster size deviation assessments corresponding to all valid clusters, and use the resulting mapping result as the cluster size consistency assessment. Use the sum of the minimum cluster proportion assessment and the cluster size consistency assessment as the cluster size rationality factor corresponding to the candidate clustering result.

[0069] For any candidate clustering result in the candidate clustering result set, obtain the Gap value corresponding to the number of candidate clusters. Map the Gap value using a hyperbolic tangent function and add it to a constant 1. Divide the sum by a constant 2 to obtain the Gap value normalization evaluation. Divide the cluster size rationality factor by a constant 2 to obtain the cluster size rationality enhancement evaluation. Multiply the Gap value normalization evaluation and the cluster size rationality enhancement evaluation by their respective influence weights and add them to obtain the clustering result evaluation score corresponding to the candidate clustering result. In this embodiment of the invention, the influence weights of the two are set to 0.5. Compare the clustering result evaluation scores corresponding to all candidate clustering results in the candidate clustering result set, and determine the candidate clustering result with the highest clustering result evaluation score as the optimal clustering result.

[0070] In one implementation, it is assumed that the minimum number of cluster samples corresponding to the clustering result is The total number of samples corresponding to the clustering results is ;No. The number of samples in each cluster is The mean number of cluster samples corresponding to effective clusters is Then the formula for calculating the cluster size rationality factor is:

[0071]

[0072] in, Indicates the cluster size rationality factor; This represents the minimum number of cluster samples corresponding to the clustering result; This represents the total number of samples corresponding to the clustering results; Indicates the first The number of samples in each cluster class; Indicates the fine-grained dividing coefficient; This indicates the number of valid clusters.

[0073] Assumption The corresponding gap value is The impact weights of Gap value normalization assessment and cluster size rationality enhancement assessment are respectively... ,but The expression for calculating the evaluation score of the clustering results is as follows:

[0074]

[0075] in, express Evaluation score for clustering results; The weights representing the influence of Gap value normalization assessment and cluster size rationality enhancement assessment are set in the embodiments of this invention. ; express The corresponding gap value; express The value corresponds to the cluster size rationality factor.

[0076] It should be noted that the cluster size rationality factor is used to evaluate the stability of candidate clustering results in representing historical water quality states from the perspective of cluster sample quantity distribution. The first term in the formula... This value reflects the proportion of the smallest cluster in the current candidate clustering results. The larger the value, the less likely the smallest cluster is composed of a very small number of isolated samples, indicating that the current clustering result is less prone to fragmentation caused by excessive splitting. The second term in the formula first filters out small clusters with too few samples, then calculates the deviation of the number of samples in the remaining effective clusters from the average number of samples in the effective clusters, and applies an exponential decay mapping to this total deviation, thus reflecting the consistency of the main cluster size distribution except for small clusters. The closer the sample sizes of each effective cluster are, the smaller the summation term and the larger the result after exponential mapping, indicating that the current candidate clustering result is more balanced and stable in its division of the main historical water quality state. By jointly representing the proportion of the smallest cluster and the consistency of the main cluster size, the cluster size rationality factor can simultaneously take into account the ability to suppress extreme small clusters and the overall cluster structure stability, which is more in line with the actual characteristics of smart water management scenarios where historical water quality states usually form multiple relatively balanced state clusters in a continuous period and a gradual process.

[0077] The clustering result evaluation score is used to comprehensively rank the candidate clustering results under different numbers of candidate clusters. In this formula, The purpose is to first compress and map the Gap value using the hyperbolic tangent function, and then linearly shift it to the [0,1] interval, thereby reflecting the quality of the current candidate clustering result in terms of overall structural separation. The larger the Gap value, the more obvious the structural separation characteristics of the candidate clustering result are relative to the random distribution. On the other hand, The squared enhancement of the cluster size rationality factor is used to further amplify the advantage of candidate clustering results that also possess high cluster size stability in the final score. This avoids selecting candidate clustering results that, while exhibiting surface structural separability, suffer from significant fragmentation or cluster size imbalance simply based on a high Gap value. Through this method, the clustering result evaluation score can jointly evaluate candidate clustering results from two dimensions: structural separability and size rationality. Ultimately, the candidate clustering result with the highest score can both effectively represent the objective differences between historical water quality states and maintain the stability and engineering interpretability of cluster division. This provides a reliable historical state space basis for subsequent data vector mapping exceeding the deviation threshold and the identification of true anomalies.

[0078] It is important to further clarify that the two-level evaluation logic employed in this step is not simply adding an auxiliary indicator, but rather a targeted design addressing the characteristics of historical water quality states in smart water management scenarios, which involve multi-stage continuous evolution, gradual state transitions, and a typically low proportion of anomalous state samples. If the optimal cluster number is selected solely based on the Gap value, it is easy to prioritize candidate clustering results that have strong statistical separation effects but are unstable in sample distribution, leading to an unreliable basis for matching typical historical states during subsequent target data mapping. However, by introducing a cluster size rationality factor and using it in conjunction with the Gap value in the clustering result evaluation score, it is possible to ensure that the cluster structure has sufficient separation capability while constraining the cluster size distribution of candidate clustering results to prevent excessive dispersion. This ensures that the final optimal clustering result more accurately reflects the typical state distribution patterns during historical water quality operation. Thus, both the structural saliency of the historical water quality data clustering results and the stability of their cluster size have been evaluated, providing a reliable premise for subsequent typical anomaly matching analysis based on the optimal clustering results.

[0079] This completes the process of obtaining the optimal clustering result through cluster analysis and evaluation of historical water quality data.

[0080] Step S3 involves mapping data vectors with deviations exceeding a threshold to the optimal clustering results and performing trend analysis to obtain the true anomaly confidence level.

[0081] After completing the cluster analysis and evaluation of historical water quality data, the optimal clustering results that characterize the distribution features of historical water quality states have been obtained, thus providing a historical state space basis for subsequent state attribution analysis of data vectors with deviations exceeding the threshold. However, in the actual operation of smart water management, the water quality prediction model learns the correlation between multiple indicators over a historical period. When water source switching, changes in pollutant input, or adjustments to water treatment processes cause changes in the indicator relationships, even if no actual anomalies occur in the current water body, a significant deviation may appear between the predicted and actual chemical oxygen demand (COD) values. Therefore, simply relying on the prediction deviation exceeding the preset range is insufficient to directly determine whether the deviation originates from a real water quality anomaly; further analysis is necessary to determine whether the data corresponding to the deviation matches anomaly patterns in the historical state space.

[0082] Meanwhile, true water quality anomalies may manifest as either typical anomalies that have occurred historically or novel anomalies that have never appeared before. The former typically appears as small clusters with relatively concentrated state characteristics in historical clustering results, while the latter is often difficult to identify directly through clustering affiliation. However, it usually exhibits continuous evolutionary characteristics within a local timeframe, causing the residual sequence to show strong trend continuity and temporal correlation. Therefore, in this invention, it is necessary to further combine the affiliation of the target data in the optimal clustering results with the changing trend of the local residual sequence, based on the deviation exceeding the threshold, to conduct a joint analysis. This approach simultaneously addresses the identification of both typical and atypical historical anomalies, providing a foundation for subsequently obtaining the confidence level of true anomalies.

[0083] In summary, this invention first performs clustering mapping on data vectors with deviations exceeding a threshold to obtain the target's assigned cluster. Specifically, for any target time, it acquires the corresponding chemical oxygen demand (COD), dissolved oxygen (DO), pH, turbidity, and ammonia nitrogen (AM) monitoring data. These data are then combined to obtain the corresponding data vector for that time. This data vector is then input into a water quality prediction model to obtain predicted COD data for that time. The difference between the predicted COD data and the predicted COD data is used as the prediction deviation for that time. The absolute value of the prediction deviation is compared to a preset deviation threshold. When the absolute value of the prediction deviation exceeds the preset threshold, the data vector for that time is determined as the data vector with a deviation exceeding the threshold. In this embodiment, the deviation threshold is set as... For any data vector whose deviation exceeds a threshold, the Euclidean distance between the data vector exceeding the threshold and the corresponding cluster centers in the optimal clustering result is obtained. The cluster corresponding to the cluster center with the smallest Euclidean distance is determined as the target cluster set to which the data vector with the deviation exceeds the threshold belongs.

[0084] After obtaining the target cluster set corresponding to the data vector, the typical anomaly matching analysis is performed on the target cluster set data to obtain the typical true anomaly confidence score. Specifically, for any data vector with a deviation exceeding a threshold, the number of cluster samples in the target cluster set corresponding to the data vector with a deviation exceeding the threshold is obtained, and the total number of samples and the number of candidate clusters corresponding to the optimal clustering result are obtained. The result of dividing the total number of samples by the number of candidate clusters is taken as the average number of cluster samples. The difference between the number of cluster samples and the average number of cluster samples is taken as the cluster size difference assessment. For any data vector with a deviation exceeding the threshold, the number of cluster samples corresponding to all clusters in the optimal clustering result is obtained. The maximum value among all cluster sample numbers is taken as the maximum number of cluster samples, and the minimum value among all cluster sample numbers is taken as the minimum number of cluster samples. The difference between the maximum number of cluster samples and the minimum number of cluster samples is taken as the cluster size normalization scale. The result of dividing the cluster size difference assessment by the cluster size normalization scale is added to a constant 1, and the result is multiplied by one-half to obtain the cluster size normalization assessment. The difference between the constant 1 and the cluster size normalization assessment is taken as the cluster size anomaly tendency assessment. For any data vector whose deviation exceeds a threshold, the Euclidean distance between the data vector exceeding the threshold and the cluster center of the target cluster is obtained. The negative of the Euclidean distance is then subjected to an exponential mapping with the natural constant as the base, and the resulting mapping is used as the cluster center proximity assessment. The sum of the cluster size anomaly tendency assessment and the cluster center proximity assessment is used as the typical true anomaly confidence level corresponding to the data vector whose deviation exceeds the threshold.

[0085] After obtaining the typical true anomaly confidence scores corresponding to the data vectors, the atypical true anomaly confidence scores are obtained by performing trend continuity analysis on the local residual sequence data. Specifically, for the target time corresponding to any data vector with a deviation exceeding the threshold, the prediction deviations corresponding to each time point within a local time range ending at the target time are obtained, constructing a local residual sequence. In this embodiment, the local time range length is set to a 30-minute time window, corresponding to 6 sampling points. Linear fitting is performed based on the local residual sequence to obtain the corresponding residual fitting line, and the slope of the residual fitting line is obtained. For any residual data in the local residual sequence, the vertical distance from the corresponding point of the residual data to the residual fitting line is obtained, and the sum of the vertical distances from all the corresponding points of the residual data in the local residual sequence to the residual fitting line is used as the trend deviation evaluation. When the slope is not equal to 0, the trend direction function is set to 1; when the slope is equal to 0, the trend direction function is set to 0. The inverse of the trend deviation assessment is exponentially mapped to a base of the natural constant, and the mapping result is multiplied by the trend direction function to obtain the trend continuity assessment. For the local residual sequence, the first-order autocorrelation coefficient between the residual data at adjacent time points is obtained. The first-order autocorrelation coefficient is added to constant 1 and divided by constant 2 to obtain the autocorrelation normalized assessment. The result of adding the trend continuity assessment and the autocorrelation normalized assessment is used as the confidence level of the atypical true anomaly corresponding to the data vector with deviation exceeding the threshold.

[0086] In one implementation, assume the first Data vectors whose deviation exceeds the threshold at any given time. The number of cluster samples in the corresponding target cluster set is The total number of samples corresponding to the optimal clustering result is The number of candidate clusters is The maximum number of cluster samples corresponding to all clusters in the optimal clustering result is: The minimum number of cluster samples corresponding to all clusters in the optimal clustering result is ;No. Data vectors whose deviation exceeds the threshold at any given time. The Euclidean distance between the cluster center data vector and the cluster center data vector is The expression for calculating the typical true anomaly confidence level corresponding to a data vector with a deviation exceeding the threshold is:

[0087]

[0088] in, This represents the average number of cluster samples in each cluster of the optimal clustering result; This indicates the degree of deviation of the number of cluster samples belonging to the target cluster set relative to the average number of cluster samples. When a data vector with a deviation exceeding the threshold belongs to a small cluster with fewer samples, the deviation is relatively small or even negative, resulting in a larger cluster size anomaly tendency assessment after normalization. Conversely, when it belongs to a main cluster with a larger number of samples, it indicates that the data vector is more likely to correspond to a common, normal operating state in history, resulting in a lower cluster size anomaly tendency assessment. The proximity between the data vector whose deviation exceeds the threshold and the target cluster center is used to characterize the data vector's proximity to the cluster center. A smaller Euclidean distance indicates that the data vector better matches the historical state pattern of the cluster, and the greater the cluster center proximity assessment obtained after exponential mapping with the natural constant as the base. In this way, the typical true anomaly confidence score can jointly evaluate the data vector whose deviation exceeds the threshold from two aspects: whether the cluster it belongs to is a small cluster and whether the data vector is close to the cluster center. This allows us to characterize whether the current deviation is more likely to originate from a typical true anomaly state that has occurred in the past.

[0089] Furthermore, in one embodiment, it is assumed that for any data vector with a deviation exceeding a threshold... Corresponding target time To obtain the target time The prediction biases corresponding to each time point within a local time range at the end time are used to construct a local residual sequence. ,in Indicates the first The prediction bias corresponding to each time point This represents the length of the local time range; the residual fitting line is obtained after linear fitting of the local residual sequence. And the slope of the residual fitting line is denoted as The first-order autocorrelation coefficient corresponding to the local residual sequence is denoted as . , No. Prediction bias at time point To the residual and the straight line The vertical distance is denoted as The confidence level of the atypical true anomaly corresponding to the data vector with deviation exceeding the threshold. The calculation expression is:

[0090]

[0091] in:

[0092]

[0093] In the formula This represents the total deviation of each residual data point in the local residual sequence from the overall linear trend. The smaller this value, the closer the prediction deviation at each moment within the local time range is to the same trend evolution trajectory, the stronger the continuity of the prediction deviation in the time dimension, and the greater the trend continuity assessment obtained after exponential mapping. (Function) This is used to introduce trend direction constraints. When the slope of the residual fitting line is not 0, it indicates that the local residual sequence has a trend of continuous accumulation, continuous weakening, or continuous change within the target time range, and the function value is 1. When the slope of the residual fitting line is equal to 0, it indicates that the local residual sequence as a whole does not show a clear evolution direction, and the function value is 0, thus suppressing misjudgments caused solely by the exponential term when there is no obvious trend. In the formula... This is used to map the first-order autocorrelation coefficient from the [-1,1] interval to the [0,1] interval. A stronger correlation between residuals at adjacent time points indicates that the current prediction bias is more likely not an instantaneous random disturbance, but rather originates from a persistent and gradually propagating real anomaly. By combining trend continuity assessment with autocorrelation normalization assessment, the confidence level of atypical real anomalies can characterize novel anomalies that have never appeared in history but exhibit significant and continuous evolutionary characteristics over time.

[0094] It should also be noted that the confidence scores for typical true anomalies and atypical true anomalies correspond to two different anomaly identification mechanisms. The former is based on the historical state space and focuses on identifying anomalous state patterns that have "already occurred in history"; the latter is based on the evolutionary behavior of residual sequences within a local time range and focuses on identifying novel anomalous states that have "never occurred in history". Since real water quality anomalies in smart water management scenarios may manifest as both historically known anomalies and newly occurring anomalies, this invention constructs the above two types of confidence scores in this step so that the subsequent calculation of the true anomaly confidence index can simultaneously take into account the identification needs of typical and atypical anomalies, thereby improving the accuracy and completeness of the judgment of the source of prediction deviation.

[0095] This completes the process of mapping data vectors with deviations exceeding a threshold to the optimal clustering results and performing trend analysis to obtain the true anomaly confidence level.

[0096] Step S4: Obtain the true anomaly confidence index by comprehensively judging the true anomaly confidence level.

[0097] The above process is used to obtain data vectors with deviations exceeding the threshold, and to obtain the typical true anomaly confidence and atypical true anomaly confidence corresponding to the data vectors. The typical true anomaly confidence is used to characterize the similarity between the current data vector and the historical anomaly state pattern, while the atypical true anomaly confidence is used to characterize the continuous evolution characteristics of the residual sequence within a local time range.

[0098] Since real water quality anomalies may manifest as typical anomalies that have occurred in the past, or as new anomalies that have not appeared in historical data, the confidence index of the highest confidence level among the two types of confidence is selected as the real anomaly confidence index of the current target data. Based on this index, it is determined whether the prediction deviation caused by the current data corresponds to a real water quality anomaly.

[0099] Specifically, for any data vector whose deviation exceeds a threshold, the confidence scores for typical true anomalies and atypical true anomalies are obtained. The confidence score for typical true anomalies is divided by a constant 2 to obtain the normalized assessment of typical true anomalies. The confidence score for atypical true anomalies is divided by a constant 2 to obtain the normalized assessment of atypical true anomalies.

[0100] The normalized assessment of typical true anomalies and the normalized assessment of atypical true anomalies are compared, and the larger value between the normalized assessment of typical true anomalies and the normalized assessment of atypical true anomalies is determined as the true anomaly confidence index corresponding to the data vector whose deviation exceeds the threshold.

[0101] This completes the process of obtaining the true anomaly confidence index by comprehensively judging the confidence level of the true anomaly.

[0102] Step S5: Obtain the water quality anomaly detection results by jointly judging the prediction deviation and the true anomaly confidence index.

[0103] After obtaining the true anomaly confidence index, we can quantitatively characterize whether data vectors with deviations exceeding the threshold are more likely to originate from real water quality anomalies from two perspectives: historical typical anomaly matching and local residual evolution. However, in the smart water management online monitoring scenario, simply obtaining the true anomaly confidence index is not enough to directly form a final executable detection conclusion. This is because the system's online monitoring objective is not simply to output an intermediate evaluation value, but to provide a clear judgment of whether the water quality status is abnormal at each target time, in order to further support water source safety management, water treatment control, and anomaly early warning. Therefore, it is necessary to jointly determine the aforementioned prediction deviation and the true anomaly confidence index in this step, so that the aforementioned model prediction results, historical state matching results, and local residual evolution analysis results are ultimately unified and implemented into a directly outputtable water quality anomaly detection result.

[0104] Specifically, for any target time, the monitoring data of chemical oxygen demand (COD), dissolved oxygen (DO), pH, turbidity, and ammonia nitrogen (AM) at the target time are acquired. These data are then combined to obtain a data vector corresponding to the target time. This data vector is input into a water quality prediction model to obtain predicted COD data for that time. The difference between the predicted COD data and the predicted COD data at the target time is used as the prediction deviation. For any target time, the absolute value of the prediction deviation is compared to a preset deviation threshold. When the absolute value of the prediction deviation is greater than the preset deviation threshold, the data vector at the target time is determined as a data vector with a deviation exceeding the threshold, and the true anomaly confidence index corresponding to this data vector is further obtained. When the absolute value of the prediction deviation is less than or equal to the preset deviation threshold, the water quality state at the target time is determined to be non-abnormal. In this embodiment, the preset deviation threshold is set as follows: .

[0105] After obtaining the true anomaly confidence index corresponding to the data vector with deviation exceeding the threshold, the true anomaly confidence index is compared with a preset confidence threshold. When the true anomaly confidence index is greater than the preset confidence threshold, the water quality state at the target time is determined to be a true water quality anomaly, and the water quality anomaly detection result at the target time is output. When the true anomaly confidence index is less than or equal to the preset confidence threshold, the water quality state at the target time is determined to be a non-abnormal state. In this embodiment of the invention, the preset confidence threshold is set to 0.5.

[0106] It should be noted that the joint judgment method using prediction bias and true anomaly confidence index in this step is not a simple superposition of the aforementioned results, but rather a collaborative decision-making mechanism set up for two different judgment levels in the smart water management scenario: anomaly candidate detection and anomaly authenticity confirmation. Prediction bias primarily reflects whether there is a significant inconsistency between the actual monitored value and the model's predicted value at the current moment, while the true anomaly confidence index is used to further determine whether this inconsistency possesses true anomaly attributes. By combining the two, the false alarm problem caused by relying solely on prediction bias can be avoided, as well as the situation where relying solely on the true anomaly confidence index ignores the significance of the current bias, thus making the final output water quality anomaly detection results more consistent with actual engineering needs.

[0107] This completes the process of obtaining water quality anomaly detection results by jointly judging prediction bias and the confidence index of actual anomalies.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for online detection and analysis of water quality data for smart water management, characterized in that, The method includes: Step S1: Obtain a water quality prediction model by collecting, preprocessing, and training water quality-related data; Step S2: Obtain the optimal clustering result by performing cluster analysis and evaluation on historical water quality data; Step S3: Obtain the true anomaly confidence score by mapping data vectors with deviations exceeding the threshold to the optimal clustering results and performing trend analysis; Step S4: Obtain the true anomaly confidence index by comprehensively judging the true anomaly confidence level; Step S5: Obtain the water quality anomaly detection results by jointly judging the prediction bias and the confidence index of the actual anomaly; The method of obtaining the confidence level of true anomalies by mapping data vectors with deviations exceeding the threshold to the optimal clustering results and performing trend analysis includes: obtaining the target cluster by performing clustering mapping processing on data vectors with deviations exceeding the threshold; obtaining the typical true anomaly confidence level by performing typical anomaly matching analysis processing on the data of the target cluster; and obtaining the atypical true anomaly confidence level by performing trend continuity analysis processing on the local residual sequence data. The step of clustering and mapping data vectors with deviations exceeding a threshold to obtain the target cluster set includes: for any target time, acquiring chemical oxygen demand (COD), dissolved oxygen (DO), pH, turbidity, and ammonia nitrogen (AM) monitoring data corresponding to the target time; combining these data to obtain a data vector corresponding to the target time; inputting the data vector into a water quality prediction model to obtain COD prediction data corresponding to the target time; using the difference between the COD monitoring data and the COD prediction data as the prediction deviation for the target time; comparing the absolute value of the prediction deviation with a preset deviation threshold; when the absolute value of the prediction deviation is greater than the preset deviation threshold, determining the data vector corresponding to the target time as the data vector with deviations exceeding the threshold; for any data vector with deviations exceeding the threshold, acquiring the Euclidean distance between the data vector with deviations exceeding the threshold and the corresponding cluster centers in the optimal clustering result; and determining the cluster corresponding to the cluster center with the smallest Euclidean distance as the target cluster set corresponding to the data vector with deviations exceeding the threshold. The process of performing typical anomaly matching analysis on the target cluster data to obtain typical true anomaly confidence includes: for any data vector with deviation exceeding a threshold, obtaining the number of cluster samples in the target cluster corresponding to the data vector with deviation exceeding the threshold, and obtaining the total number of samples and candidate clusters corresponding to the optimal clustering result; dividing the total number of samples by the number of candidate clusters as the average number of cluster samples; using the difference between the number of cluster samples and the average number of cluster samples as the cluster size difference assessment; for any data vector with deviation exceeding a threshold, obtaining the number of cluster samples corresponding to all clusters in the optimal clustering result; using the maximum value among all cluster sample numbers as the maximum number of cluster samples, and the minimum value among all cluster sample numbers as the minimum number of cluster samples; and comparing the maximum number of cluster samples with the minimum number of cluster samples... The difference in the number of samples in small clusters is used as the cluster size normalization scale; the cluster size difference assessment divided by the cluster size normalization scale is added to a constant 1, and the sum is multiplied by one-half to obtain the cluster size normalization assessment; the difference between the constant 1 and the cluster size normalization assessment is used as the cluster size anomaly tendency assessment; for any data vector with deviation exceeding the threshold, the Euclidean distance between the data vector with deviation exceeding the threshold and the cluster center corresponding to the target cluster is obtained; the negative of the Euclidean distance is subjected to an exponential mapping with the natural constant as the base, and the corresponding mapping result is used as the cluster center proximity assessment; the calculation result of adding the cluster size anomaly tendency assessment and the cluster center proximity assessment is used as the typical true anomaly confidence level corresponding to the data vector with deviation exceeding the threshold; The process of obtaining atypical true anomaly confidence by performing trend continuity analysis on local residual sequence data includes: for any data vector whose deviation exceeds a threshold, obtaining the prediction deviation corresponding to each moment within a local time range ending at the target moment, and constructing a local residual sequence; performing linear fitting based on the local residual sequence to obtain the corresponding residual fitting line, and obtaining the slope of the residual fitting line; for any residual data in the local residual sequence, obtaining the vertical distance from the corresponding point of the residual data to the residual fitting line, and summing the vertical distances from the corresponding points of all residual data in the local residual sequence to the residual fitting line. The calculation result is used as a trend deviation assessment; when the slope is not equal to 0, the trend direction function is set to 1; when the slope is equal to 0, the trend direction function is set to 0; the negative number of the trend deviation assessment is subjected to an exponential mapping with the natural constant as the base, and the mapping result is multiplied by the trend direction function to obtain the trend continuity assessment; for the local residual sequence, the first-order autocorrelation coefficient between the residual data at adjacent time points is obtained; the first-order autocorrelation coefficient is added to constant 1 and divided by constant 2 to obtain the autocorrelation normalization assessment; the calculation result of adding the trend continuity assessment and the autocorrelation normalization assessment is used as the confidence level of the atypical true anomaly corresponding to the data vector with deviation exceeding the threshold; The step of obtaining the true anomaly confidence index by comprehensively judging the true anomaly confidence level includes: for any data vector with deviation exceeding a threshold, obtaining the typical true anomaly confidence level and the atypical true anomaly confidence level corresponding to the data vector with deviation exceeding the threshold; dividing the typical true anomaly confidence level by a constant 2 to obtain the typical true anomaly normalized assessment; dividing the atypical true anomaly confidence level by a constant 2 to obtain the atypical true anomaly normalized assessment; comparing the typical true anomaly normalized assessment and the atypical true anomaly normalized assessment, and determining the larger value between the typical true anomaly normalized assessment and the atypical true anomaly normalized assessment as the true anomaly confidence index corresponding to the data vector with deviation exceeding the threshold.

2. The water quality data online detection and analysis method for smart water affairs according to claim 1, characterized in that, The process of acquiring a water quality prediction model through the collection, preprocessing, and model training of water quality-related data includes: Online water quality monitoring equipment is deployed in water sources, water transmission networks, or water treatment facilities. A sampling cycle is set, and the water body is continuously collected according to the sampling cycle to obtain water quality-related data. The water quality-related data includes at least chemical oxygen demand monitoring data, dissolved oxygen monitoring data, pH value monitoring data, turbidity monitoring data, and ammonia nitrogen monitoring data. The chemical oxygen demand (COD), dissolved oxygen (DO), pH, turbidity, and ammonia nitrogen (AM) monitoring data are processed by filling missing values, removing outliers, and aligning time series data. The processed COD, DO, pH, turbidity, and AM monitoring data are then standardized or normalized to obtain historical water quality data. The dissolved oxygen monitoring data, pH monitoring data, turbidity monitoring data, and ammonia nitrogen monitoring data are used as input features, and the chemical oxygen demand monitoring data at the time points corresponding to the input features are used as prediction targets. The Long Short-Term Memory (LSTM) network is trained to obtain a water quality prediction model.

3. The water quality data online detection and analysis method for smart water affairs according to claim 1, characterized in that, The process of obtaining the optimal clustering result through cluster analysis and evaluation of historical water quality data includes: Candidate clustering result set was obtained by performing candidate cluster number analysis and clustering processing on historical water quality data; The optimal clustering result is obtained by performing clustering evaluation on the candidate clustering result set.

4. The water quality data online detection and analysis method for smart water affairs according to claim 3, characterized in that, The process involves analyzing and clustering historical water quality data to obtain a candidate clustering result set, including: For any target time point in the historical water quality data, the chemical oxygen demand monitoring data, dissolved oxygen monitoring data, pH value monitoring data, turbidity monitoring data and ammonia nitrogen monitoring data corresponding to the target time point are combined to obtain the historical water quality data vector corresponding to the target time point, and the set of all historical water quality data vectors corresponding to the target time points is used as the historical water quality data vector set for cluster analysis. Obtain the number of water quality data index types corresponding to the historical water quality data vector set, and use the number of water quality data index types as the maximum candidate cluster number to construct a candidate cluster number set consisting of 1 to the maximum candidate cluster number; for any candidate cluster number in the candidate cluster number set, use the Gap Statistic algorithm to analyze the historical water quality data vector set to obtain the Gap value under the corresponding candidate cluster number; determine the candidate cluster number with the largest Gap value as the ideal cluster number; A range of candidate cluster numbers is constructed with the ideal cluster number as the center; for any candidate cluster number in the range of candidate cluster numbers, the historical water quality data vector set is clustered using the K-means clustering algorithm to obtain the candidate clustering results under the corresponding candidate cluster number; the set of candidate clustering results corresponding to all candidate cluster numbers in the range of candidate cluster numbers is taken as the candidate clustering result set.

5. The water quality data online detection and analysis method for smart water affairs according to claim 3, characterized in that, The step of obtaining the optimal clustering result by performing clustering evaluation processing on the candidate clustering result set includes: For any candidate clustering result in the candidate clustering result set, obtain the number of candidate clusters, the total number of samples, and the number of samples in each cluster corresponding to the candidate clustering result; take the minimum number of samples in each cluster as the minimum number of cluster samples, and take the result of dividing the minimum number of cluster samples by the total number of samples as the minimum cluster proportion evaluation. For any candidate clustering result in the candidate clustering result set, a fine cluster partitioning coefficient is set, and clusters whose sample size is greater than the product of the fine cluster partitioning coefficient and the total number of samples are determined as valid clusters; the mean number of cluster samples corresponding to all valid clusters is obtained; for any valid cluster, the absolute value of the difference between the number of cluster samples of the valid cluster and the mean number of cluster samples is used as the corresponding cluster size deviation assessment; the negative of the calculation result of the sum of the cluster size deviation assessments corresponding to all valid clusters is subjected to exponential mapping with the natural constant as the base, and the corresponding mapping result is used as the cluster size consistency assessment; The result of adding the minimum cluster proportion assessment and the cluster size consistency assessment is used as the cluster size rationality factor corresponding to the candidate clustering result; For any candidate clustering result in the candidate clustering result set, obtain the Gap value under the corresponding number of candidate clusters for the candidate clustering result; map the Gap value using a hyperbolic tangent function and add it to a constant 1, then divide the sum by a constant 2 to obtain the Gap value normalization evaluation; divide the cluster size rationality factor by a constant 2 to obtain the cluster size rationality enhancement evaluation; multiply the Gap value normalization evaluation and the cluster size rationality enhancement evaluation by their respective influence weights and then add them to obtain the clustering result evaluation score corresponding to the candidate clustering result; The evaluation scores of all candidate clustering results in the candidate clustering result set are compared, and the candidate clustering result with the highest evaluation score is determined as the optimal clustering result.

Images