An online water quality monitoring anomaly typing correction and credible data generation method and system

Abstract

The application discloses an online water quality monitoring anomaly typing correction and credible data generation method and system, which comprises the following steps: generating a multi-source window sample based on original water quality monitoring data and auxiliary operation data; constructing a hierarchical multi-source consistency deep network, and training the network by using the multi-source window sample; determining the abnormal class label of the corresponding target segment to be processed according to the abnormal candidate score, the reconstruction result, the residual amount and the hierarchical classification probability of the multi-source window sample to be processed; generating a gating calibration decision for each target segment to be processed according to the abnormal class label, constructing a corresponding position-level correction mask matrix and a correction strength coefficient matrix; generating the correction result of the target segment to be processed, combining the classification probability and the residual amount, calculating the credibility score, and fusing the correction results of multiple overlapping segments into a global standardized credible sequence, and finally obtaining credible data. The application improves the availability, reliability and engineering application value of online monitoring data.

Description

Technical Field

[0001] This invention belongs to the field of online water environment monitoring and intelligent data processing technology, specifically relating to a method and system for online water quality monitoring anomaly classification correction and reliable data generation. Background Technology

[0002] With the development of online water quality monitoring technology, multi-parameter sensors can continuously acquire monitoring indicators such as dissolved oxygen, pH, turbidity, conductivity, temperature, ammonia nitrogen, chemical oxygen demand, total nitrogen, and total phosphorus, providing a continuous data foundation for water environment status identification, anomaly early warning, operational analysis, and control decisions. Compared with traditional manual sampling and testing, online monitoring has advantages such as high sampling frequency, strong timeliness, and continuous operation, and has become one of the important means of water environment monitoring.

[0003] However, in practical applications, the quality of online monitoring data is easily affected by various factors. On the one hand, sensors may experience problems such as contamination, aging and degradation, partial obstruction, calibration deviation, changes in cleaning cycles, and communication anomalies during long-term operation, leading to anomalies in monitoring data such as gradual drift, short-term contamination, missing measurements, constant output, and jamming. On the other hand, the monitored object itself may also experience real water quality anomalies due to pollution input, hydrodynamic changes, operating condition switching, or environmental disturbances. Because the above anomalies may have certain similarities in their time series manifestations, it is often difficult to directly distinguish between real water quality changes and equipment anomalies in actual data processing.

[0004] In existing technologies, one type of method mainly uses threshold discrimination, statistical rules, empirical filtering, or interpolation repair to process abnormal data. This type of method is relatively simple to implement, but it typically relies solely on the numerical change of a single indicator, making it difficult to fully utilize the coupling relationships between multiple monitoring indicators, and also difficult to effectively distinguish the source of anomalies by combining operating conditions, maintenance records, and missing measurement status. Another type of method introduces neural networks, time-series prediction models, or reconstruction models to detect anomalies or repair data. While this type of method improves processing capabilities to some extent, most solutions still handle anomaly detection, anomaly classification, and correction processes separately, lacking a unified technical chain from anomaly source identification and differentiated correction to reliable data output.

[0005] In addition, existing solutions have the following shortcomings in engineering applications: First, they often treat real water quality events and equipment anomalies as "outliers" for correction, which may easily weaken or eliminate the real change process. Second, they often use the same correction strategy to deal with anomalies from different sources, lacking differentiated processing mechanisms for different anomaly types such as gradual drift, short-term fouling, and hard failures. Third, after correction, they usually only output a single numerical result, lacking a quantitative description of the reliability of the result, which is not conducive to further utilization of data reliability by subsequent early warning, prediction, and control modules. Summary of the Invention

[0006] Purpose of the invention: The purpose of this invention is to provide a method and system for online water quality monitoring anomaly classification correction and reliable data generation, so as to improve the availability, reliability and engineering application value of online monitoring data.

[0007] Technical solution: The method described in this invention includes the following steps:

[0008] The raw water quality monitoring data and auxiliary operation data are mapped to the same time reference and windowed to form multi-source window samples that can be directly input into the subsequent deep network.

[0009] A hierarchical multi-source consistency deep network is constructed and trained using multi-source window samples. The hierarchical multi-source consistency deep network includes: an input embedding module, a multi-scale temporal coding module, a cross-index consistency graph coding module, an asymmetric context reconstruction module, and a hierarchical classification module. The input embedding module outputs an initial embedding feature tensor of the multi-source window samples. The multi-scale temporal coding module models the temporal evolution of each water quality monitoring index in the initial embedding feature tensor to obtain a temporal coding feature tensor. The cross-index consistency graph coding module obtains the cross-index consistency graph coding result based on the temporal coding feature tensor. The asymmetric context reconstruction module reconstructs the target segment using an asymmetric context reconstruction method to obtain the target segment reconstruction result. Based on the standardized context reconstruction residual, standardized cross-index consistency residual, and standardized state matching residual, anomaly candidate scores for the corresponding target segment are obtained. The hierarchical classification module outputs two levels of probabilities for the anomaly candidate segments.

[0010] For the anomaly candidate scores, reconstruction results, residuals, and two-level probabilities of the multi-source window samples to be processed, the anomaly category labels of the corresponding target segments to be processed are determined according to a preset priority order.

[0011] Based on the anomaly category label, a gating calibration decision is generated for each target segment to be processed. Based on the gating calibration decision, a position-level correction mask matrix and a correction intensity coefficient matrix corresponding to the target segment to be processed are constructed.

[0012] Based on anomaly category labels, gating calibration decisions, location-level correction mask matrices, and correction intensity coefficient matrices, the correction results of the target segments to be processed are first generated. Then, by combining two-level probabilities and residuals, a credibility score is calculated for each target segment to be processed. The correction results of multiple overlapping segments are then fused into a globally standardized credibility sequence, and finally, credibility data is obtained.

[0013] Furthermore, multi-source window sample formation methods include:

[0014] Collect raw water quality monitoring data output from each sensor in the online monitoring system, as well as auxiliary operational data related to the monitoring process;

[0015] The original water quality monitoring data and auxiliary operation data are mapped onto a unified time axis to obtain the water quality monitoring synchronization sequence and the auxiliary operation synchronization sequence.

[0016] Define a missing test mask matrix. When any indicator is missing or invalid at any time, the matrix element is defined as 1, otherwise it is 0.

[0017] For each water quality monitoring indicator, a reference statistic is constructed, including the reference median. and reference interquartile range ;

[0018] Robust standardization was performed on the water quality monitoring synchronization sequence and the auxiliary operation synchronization sequence to obtain standardized water quality monitoring sequence and standardized auxiliary operation data.

[0019] A sliding window approach was used to segment the standardized water quality monitoring sequence, standardized auxiliary operation data, and missing measurement mask matrix into windows, constructing a standardized water quality monitoring window, an auxiliary operation window, and a missing measurement mask window, thus obtaining a multi-source window sample set.

[0020] Furthermore, the input embedding module is used to input the first standardized water quality monitoring window. The first row sequence, auxiliary running window, and missing test mask window row sequence Input the embedding function to get the first The initial embedding features corresponding to each water quality monitoring indicator are then stacked according to the indicator dimension to obtain the first... The initial embedding feature tensor of a multi-source window sample;

[0021] The cross-indicator consistency graph encoding module, firstly, for the first... Each water quality monitoring indicator is in the window The temporal coding features in the data are pooled along the time dimension to obtain the node representation of the water quality monitoring index; then, a dynamic adjacency matrix is ​​constructed through attention similarity; further, graph aggregation is performed on the node representations to obtain the graph coding result; and all node graph representations are stacked to obtain the cross-index consistent graph coding result.

[0022] The asymmetric context reconstruction module uses the asymmetric context reconstruction function to reconstruct the target fragment, obtaining the reconstructed target fragment. The asymmetric context reconstruction function takes the left context, right context, cross-index consistency graph encoding result, and auxiliary running window as input, and outputs the target fragment reconstruction result.

[0023] The hierarchical classification module first constructs a window-level fused feature vector and then builds a first-level classification head to determine whether the abnormal candidate fragment is more likely to belong to the category of real water quality events or equipment / data anomalies. Its output is the first-level classification probability. The second-level classification head is then constructed to further subdivide only the equipment / data anomaly category, outputting the probabilities of progressive drift, short-term fouling, and hard failure / disconnection. Its output is the second-level classification probability.

[0024] Furthermore, the total network loss function for:

[0025] ;

[0026] in, , , , , These are, respectively, the primary classification loss, the secondary classification loss, the context reconstruction loss, the cross-metric consistency loss, and the state matching loss. , , , , These are the non-negative weight coefficients for the first-level classification loss, second-level classification loss, context reconstruction loss, cross-index consistency loss, and state matching loss, respectively.

[0027] The first-level classification loss is: ;in, Indicates the number of training multi-source window samples. Indicates the first The training multi-source window samples belong to the first classification. The true label of the class, Indicates the probability of prediction for the primary classification;

[0028] The second-order classification loss is: ;in, Indicates the first The training multi-source window samples belong to the [number]th [class] in the secondary classification. The true label of the class, Indicates the probability of secondary classification prediction;

[0029] The context reconstruction loss is: ,in, Represents the residual from the underlying context reconstruction;

[0030] The cross-metric consistency loss is: ,in, This represents the basic cross-indicator consistency residual;

[0031] The state matching loss is: ,in, Represents the basic state matching residual;

[0032] By minimizing the total loss function, the hierarchical multi-source consistency deep network can simultaneously learn anomaly candidate identification, anomaly classification, context reconstruction, cross-metric consistency maintenance, and state matching constraints during training.

[0033] Furthermore, the network is trained using multi-source window samples, including:

[0034] Define the network parameter set as follows This includes the parameter set of the input embedding module, the parameter set of the multi-scale temporal coding module, the parameter set of the cross-index consistency graph coding module, the parameter set of the asymmetric context reconstruction module, and the parameter set of the hierarchical classification module.

[0035] Using the total loss function To optimize the objective, gradient descent, stochastic gradient descent, or the Adam optimization algorithm are used to optimize the parameter set. Perform iterative updates until the total loss function converges or the preset number of training epochs is reached, resulting in the completed target parameter set. :

[0036] .

[0037] Furthermore, based on the anomaly candidate scores, reconstruction results, residuals, and two-level probabilities of the multi-source window samples to be processed, the anomaly category labels of the corresponding target segments to be processed are determined according to a preset priority order; including: Abnormal candidate segments are uniformly divided into four categories: real water quality events, gradual drift, short-term fouling, and hard failures / loss of contact. Construct segment-level auxiliary discrimination parameters, including: time-by-time reconstruction error in the target segment to be processed, proportion of abnormal persistence, difference between adjacent time moments of the target segment to be processed, proportion of consistency in the rising direction, proportion of consistency in the falling direction, proportion of consistency in the direction, intensity of local fluctuations, proportion of missing data in the target segment to be processed, and proportion of flatness in the target segment to be processed. Distinguishing between real water quality incidents and equipment / data anomalies: If the following conditions are met simultaneously: ; Then the target segment to be processed is determined to be a real water quality event, where, This indicates the probability that the target segment to be processed belongs to the category of real water quality events. This indicates the probability that the target fragment to be processed belongs to the device / data anomaly category. The standardized cross-index consistency residuals of the target segment to be processed. To match the standardized state residuals of the target segment to be processed, This represents the proportion of missing data in the target segment to be processed. The flatness ratio in the target segment to be processed. This is the probability threshold for the first-level classification. The threshold for cross-index consistency residuals. For state matching residual threshold, The threshold for the proportion of missing measurements. Indicates the flatness ratio threshold; Priority judgment for hard failures / loss of contact: If it is not judged as a real water quality event and meets any of the following conditions: ;or ;or and ; The target segment to be processed is then determined to be a hard fault / loss of connection type, where, Indicates the probability of asymptotic drift. This indicates the probability of short-term contamination. Indicates the probability of hard failure / loss of connection. The probability threshold for secondary classification of hard faults / loss of contact; Distinguishing between gradual drift and short-term fouling: If the target segment to be treated is not identified as a real water quality event, nor as a hard fault / loss, and simultaneously meets the following conditions: ; Then the target segment to be processed is determined to be of the progressive drift type, where, The threshold for the proportion of abnormal occurrences. This is the threshold for the directional consistency ratio. For local fluctuation intensity, The threshold is set as the local fluctuation intensity. If the target segment to be processed is neither identified as a real water quality event, nor as a hard fault / disconnection, nor as a gradual drift, then it is identified as a short-term fouling segment.

[0038] Furthermore, based on the anomaly category label, a gating calibration decision is generated for each target segment to be processed. Based on the gating calibration decision, a position-level correction mask matrix and a correction intensity coefficient matrix corresponding to the target segment to be processed are constructed; including:

[0039] Define the gating calibration decision corresponding to the target segment to be processed as follows: ,when When, it means to retain the original value and not perform automatic calibration; when When, it indicates that automatic calibration is being performed; when When, it indicates that a partial repair is being performed; when When this is the case, it indicates that automatic calibration is disabled, and missing reconstruction is only allowed for the failed location;

[0040] Based on the anomaly category label Establish the following gating mapping relationship:

[0041] ;

[0042] in, This indicates the anomaly category label of the target fragment to be processed. When, it indicates that the target segment to be processed belongs to the category of real water quality events; when When, it indicates that the target segment to be processed belongs to the progressive drift category; when When, it indicates that the target segment to be processed belongs to the short-term contamination category; when When this occurs, it indicates that the target segment to be processed belongs to the category of hard fault / loss of connection;

[0043] In gating calibration decision Based on this, a position-level correction mask matrix corresponding to the target segment to be processed is further constructed: when the matrix element is 1, it means that the position of the water quality monitoring index at the corresponding time of the target segment to be processed is allowed to be corrected in subsequent steps; when the matrix element is 0, it means that the original value is retained at the corresponding position.

[0044] when When, the correction mask for real water quality events is defined as 0; when When, automatic calibration is enabled for locations that deviate from the reconstructed baseline but are not missing measurements; when When, local repair is enabled at the location of the local anomaly; when At that time, only the failed location is open for missing reconstruction;

[0045] Construct the modified intensity coefficient matrix , Represents the modified intensity coefficient matrix The elements in The segmentation rules are as follows:

[0046] ;

[0047] in, Represents the position-level correction mask matrix The elements in Indicates the first One target fragment to be processed The Middle The water quality monitoring indicators are in the first Standardized observations at each time point express The reconstructed value at the corresponding position, This indicates that the automatic calibration deviates from the threshold. It is a constant.

[0048] Furthermore, the trusted data generation process is as follows:

[0049] Generate correction results for the target segment to be processed: When a certain position is not allowed to be corrected, the original value is directly retained; when a certain position is allowed to be corrected, and the target segment to be processed belongs to the progressive drift type or short-term contamination type, the original value and the reconstructed value are fused according to the correction intensity coefficient; when a certain position is allowed to be corrected, and the target segment to be processed belongs to the hard fault / disconnection type, the reconstructed value is directly used as the missing reconstruction result.

[0050] Calculate the category path confidence and credibility score;

[0051] The correction results of each target segment to be processed are merged into a globally standardized reliable sequence;

[0052] Perform denormalization on the globally standardized reliable sequence to obtain a globally reliable water quality monitoring sequence;

[0053] Finally, a reliable water quality dataset is output; the reliable water quality dataset includes: original water quality monitoring data, standardized corrected fragment results, globally standardized reliable sequences, globally reliable water quality monitoring sequences, anomaly category labels, gating calibration decisions, and reliability scores.

[0054] The system corresponding to the method described in this invention includes:

[0055] The multi-source window sample generation unit is used to map the raw water quality monitoring data and auxiliary operation data to the same time reference and perform window segmentation to form multi-source window samples that can be directly input into the subsequent deep network.

[0056] A network construction and training unit is used to construct a hierarchical multi-source consistency deep network and train the network using multi-source window samples. The hierarchical multi-source consistency deep network includes: an input embedding module, a multi-scale temporal coding module, a cross-index consistency graph coding module, an asymmetric context reconstruction module, and a hierarchical classification module. The input embedding module outputs an initial embedding feature tensor of the multi-source window samples based on the multi-source window samples. The multi-scale temporal coding module models the temporal evolution of each water quality monitoring index in the initial embedding feature tensor to obtain a temporal coding feature tensor. The cross-index consistency graph coding module obtains the cross-index consistency graph coding result based on the temporal coding feature tensor. The asymmetric context reconstruction module reconstructs the target segment using an asymmetric context reconstruction method to obtain the target segment reconstruction result. Based on the standardized context reconstruction residual, standardized cross-index consistency residual, and standardized state matching residual, the abnormal candidate score of the corresponding target segment is obtained. The hierarchical classification module outputs two levels of probability for the abnormal candidate segments.

[0057] The anomaly category label determination unit is used to determine the anomaly category label of the corresponding target segment according to a preset priority order based on the anomaly candidate score, reconstruction result, residual amount and two-level probability of the multi-source window sample to be processed.

[0058] The correction range determination unit is used to generate a gating calibration decision for each target segment to be processed based on the anomaly category label, and to construct the position-level correction mask matrix and correction intensity coefficient matrix corresponding to the target segment to be processed based on the gating calibration decision.

[0059] The trusted data generation unit is used to generate the correction results of the target segment to be processed based on the anomaly category label, gating calibration decision, position-level correction mask matrix and correction intensity coefficient matrix. Then, it calculates the trust score for each target segment to be processed by combining two-level probability and residual. The correction results of multiple overlapping segments are merged into a globally standardized trusted sequence to finally obtain trusted data.

[0060] The computer program product corresponding to the method of the present invention includes a computer program / instruction that implements the method when executed by a processor.

[0061] Beneficial effects: Compared with the prior art, the significant technical effects of the present invention are as follows: (1) The present invention constructs a complete processing flow from anomaly candidate identification, sequential anomaly classification, gating calibration decision to credible data generation, so that there is a clear connection between each link of online water quality monitoring data processing, which is conducive to improving the systematicness and implementation consistency of the overall technical solution; (2) The present invention uniformly classifies anomalies in online water quality monitoring into real water quality events, gradual drift, short-term pollution and hard fault / disconnection, and makes the anomaly category correspond to the subsequent processing action, so as to preserve the real water quality changes while (2) Targeted corrections are implemented for different types of equipment anomalies to reduce the possibility of misjudgment and miscorrection; (3) This invention integrates multi-parameter water quality monitoring data, auxiliary operation data and missing measurement information into a hierarchical multi-source consistency deep network for joint modeling, which can simultaneously utilize time evolution relationship, multi-index coupling relationship and state matching relationship to improve the accuracy and robustness of anomaly identification, anomaly classification and data correction process; (4) After obtaining the segment correction results, this invention further quantifies the credibility of the results and completes the fusion processing of overlapping windows based on credibility, finally forming a global credible water quality data sequence. This result can not only reflect the corrected monitoring values, but also reflect the reliability of different segment results, which is convenient for subsequent early warning, prediction and control modules to call. Attached Figure Description

[0062] Figure 1 This is a flowchart of the method of the present invention;

[0063] Figure 2 This is a schematic diagram comparing the original water quality monitoring data with the global reliable water quality monitoring sequence. (a) represents real water quality events, (b) represents gradual drift events, (c) represents short-term fouling events, and (d) represents hard failures / loss of connection events. Detailed Implementation

[0064] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0065] This invention provides a technical solution capable of effectively classifying anomalies from different sources in online water quality monitoring data, generating differentiated gating calibration strategies based on the classification results, and ultimately outputting reliable water quality data. This improves the usability, reliability, and engineering application value of online monitoring data. The method is applicable to anomaly identification, differentiated correction, and reliable data construction in online water quality monitoring scenarios involving rivers, lakes, reservoirs, water supply networks, drainage networks, sewage treatment systems, and other multi-parameter scenarios.

[0066] like Figure 1 As shown, the method of the present invention includes the following steps:

[0067] Step S1: Data Acquisition, Synchronization, Standardization, and Window Sample Construction. Step S1 aims to map the raw water quality monitoring data and auxiliary operation data to the same time reference, forming standardized window samples that can be directly input into the subsequent deep network.

[0068] S1-1: Data Acquisition. Acquire raw water quality monitoring data output from each sensor in the online monitoring system. and auxiliary operational data related to the monitoring process. The water quality monitoring indicators in the raw water quality monitoring data refer to indicators that directly characterize the water quality status, such as pH, dissolved oxygen, turbidity, conductivity, ammonia nitrogen, and total phosphorus; the auxiliary operation characteristics in the auxiliary operation data refer to characteristics used to characterize the monitoring process or on-site operation status, such as flow rate, water level, pump and valve status, cleaning records, calibration records, and communication status.

[0069] S1-2: Time synchronization. Since the sampling frequencies for different water quality monitoring indicators and auxiliary operational characteristics may vary, a uniform sampling step size is first set. The number of water quality monitoring indicators is The number of auxiliary operation features is The total number of sampling times after synchronization is and will and Mapped to a unified timeline The above provides a synchronized sequence for water quality monitoring. Synchronization sequence for auxiliary operation :

[0070] ;

[0071] ;

[0072] in, , They represent the time intervals. of Water quality monitoring indicators and One auxiliary operating feature, When a certain indicator has no new sampled value at a certain moment, it is not directly interpolated and repaired in this step. Instead, its missing value indicator is retained to avoid introducing artificial smoothing errors before anomaly classification. Accordingly, a missing value mask matrix is ​​defined. :

[0073] ;

[0074] in, Indicates the first The water quality monitoring indicators are in the first The missing or invalid state at the first synchronous sampling moment, when the first Each indicator at time If the test is missing or invalid, then... Otherwise, .

[0075] S1-3: Construction of Reference Statistics. To eliminate dimensional differences and suppress interference from extreme values, reference statistics are constructed for each water quality monitoring indicator. For the first... For each water quality monitoring indicator, calculate its reference median over the reference time period. and reference interquartile range The reference time period is preferably the stable operating period after the sensor is installed or after the most recent manual maintenance or calibration. The reference median and reference interquartile range are reference statistics used for subsequent standardization processing.

[0076] S1-4: Robust Normalization. Robust normalization is performed on the water quality monitoring synchronization sequence to obtain a standardized water quality monitoring sequence. :

[0077] ;

[0078] ;

[0079] ;

[0080] in, , To standardize water quality monitoring sequences at time The vector, [1, ], It is the transpose symbol. for The first in One element, , for No. Each water quality monitoring indicator at time The value, This is a constant, representing an extremely small positive number to prevent the denominator from being zero. It is used to assist in the synchronization sequence. Normalization is also performed to form a standardized auxiliary operation sequence. ,in, To standardize the auxiliary operation sequence at time The vector.

[0081] S1-5: Window Sample Construction. Let the left context length be... The target segment length is The right context length is The total window length is Let the sliding step size be... A sliding window approach was used to analyze the standardized water quality monitoring sequence. Standardized auxiliary operation sequence and missing test mask matrix Perform window segmentation. For the first... The window position and its window start time Recorded as:

[0082] ;

[0083] Then the first The standardized water quality monitoring windows are as follows:

[0084] ;

[0085] in, To standardize water quality monitoring sequences at time The vector, .Will The context is divided into left context, target segment, and right context according to time order.

[0086] ;

[0087] in, for The left context, for The target segment, for The right context. The same applies to the standardized auxiliary run sequence. and missing test mask matrix Perform synchronous segmentation to obtain auxiliary running windows. and missing test mask window .in, For the context of the left The auxiliary running window corresponding to the time, To match the target segment The auxiliary running window corresponding to the time, For the right context The auxiliary running window corresponding to the time; , and They are respectively with , and The missing test mask window corresponding to the time. This is used to construct the first... Multi-source window samples :

[0088] ;

[0089] Finally, a multi-source window sample set is constructed. :

[0090] ;

[0091] in, This represents the total number of multi-source window samples obtained by segmenting the standardized water quality monitoring sequence.

[0092] Step S2: Construct a hierarchical multi-source consistency deep network and extract anomaly candidate scores and anomaly classification representations. Step S2 aims to construct a hierarchical multi-source consistency deep network with deep learning as the core classifier for the multi-source window samples obtained in Step S1, outputting anomaly candidate scores, context reconstruction results, cross-index consistency residuals, state matching residuals, and hierarchical classification probabilities for use in Step S3. The output results of the above hierarchical multi-source consistency deep network are collectively referred to as anomaly classification representations. In this embodiment, the hierarchical multi-source consistency deep network includes: an input embedding module, a multi-scale temporal coding module, a cross-index consistency graph coding module, an asymmetric context reconstruction module, and a hierarchical classification module.

[0093] S2-1: Input embedded module. For the first... Multi-source window samples Standardize water quality monitoring window The row sequence Auxiliary running window and missing test mask window The row sequence Input the embedding function to get the first Initial embedding features corresponding to each water quality monitoring indicator :

[0094] ;

[0095] in, Indicates an embedded function. This represents the dimension of the latent features corresponding to each water quality monitoring indicator. The embedding function can be implemented using linear mapping, fully connected layers, one-dimensional convolutional layers, or a combination thereof. Furthermore, the initial embedded features corresponding to all water quality monitoring indicators are stacked according to the indicator dimension to obtain the [n]th [indicator / indicator]. The initial embedding feature tensor of multi-source window samples :

[0096] ;

[0097] in, This indicates an operation of stacking items according to water quality monitoring indicators.

[0098] S2-2: Multi-scale temporal coding module. This module converts the initial embedded feature tensor... The input multi-scale temporal coding module models the temporal evolution of each water quality monitoring indicator, resulting in a temporal coding feature tensor. :

[0099] ;

[0100] in, This represents a multi-scale temporal coding function. Here, The first dimension corresponds to the water quality monitoring index dimension, and the second dimension corresponds to the latent feature dimension. The third dimension corresponds to the time dimension. .remember express The The water quality monitoring indicators are in the first Temporal coding features within a window. In a preferred embodiment, the multi-scale temporal coding module includes a multi-path dilated convolution unit and a cross-temporal attention unit. The multi-path dilated convolution unit is used to extract local abrupt change patterns, slow change patterns, and multi-scale trend patterns; the cross-temporal attention unit is used to capture contextual dependencies over a longer time range. For ease of description, the above multi-path dilated convolution and cross-temporal attention processing are uniformly represented as a multi-scale temporal coding function. .

[0101] S2-3: Cross-Indicator Consistency Map Encoding Module. To characterize the coupling relationships between different water quality monitoring indicators within the same window, a time-series encoded feature tensor is used. Construct a cross-indicator consistency graph encoding result. First, for the first... The water quality monitoring indicators are in the first Temporal coding features in each window Pooling along the time dimension yields the nodal characterization of this water quality monitoring index:

[0102] ;

[0103] in, Indicates the first The water quality monitoring indicators are in the first Node representation in a window This represents the pooling operation along the time dimension. Then, a dynamic adjacency matrix is ​​constructed using attention similarity. :

[0104] ;

[0105] ;

[0106] in, Indicates the first The water quality monitoring indicators are in the first Node representation in a window In the process of normalization summation, the first... The water quality monitoring indicators are in the first Node representation in a window Indicates the index for normalized summation. Indicates the first The first window The water quality monitoring indicators and the first The consistency correlation strength among individual water quality monitoring indicators and For a trainable parameter matrix, It is an exponential function. Furthermore, graph aggregation is performed on the node representations to obtain the graph encoding results:

[0107] ;

[0108] in, Indicates the first The first window The graph-coded node representation of each water quality monitoring indicator obtained by graph aggregation Graph encoding parameter matrix, The activation function is non-linear. Stacking all node graph representations yields the cross-index consistency graph encoding result. :

[0109] ;

[0110] in, This indicates a stacking operation.

[0111] S2-4: Asymmetric Context Reconstruction Module. To characterize the degree of deviation of the target fragment relative to the context, this invention does not employ the symmetric reconstruction method of target fragment input-target fragment reconstruction, but rather an asymmetric context reconstruction method. Furthermore, it utilizes an asymmetric context reconstruction function... The target fragment is reconstructed to obtain the target fragment reconstruction result. :

[0112] Specifically, firstly regarding the first The left context of the window Right context Auxiliary running window corresponding to the target segment Pooling is performed separately to obtain the left context summary vector. Right context summary vector and auxiliary runtime summary vector :

[0113] ;

[0114] ;

[0115] ;

[0116] in, The pooling operation performed along the time dimension can be either average pooling or attention pooling. Then, the cross-metric consistency graph encoding result is pooled to obtain the cross-metric consistency summary vector. :

[0117] ;

[0118] in, This represents the pooling operation performed along the dimensions of water quality monitoring indicators. Further, all the above summary vectors are concatenated, and then the concatenation result is used to generate the target fragment reconstruction result through a fully connected mapping. :

[0119] ;

[0120] in, This represents a vector concatenation operation. and These represent the trainable weight matrix and bias vector of the reconstructed mapping, respectively; This represents the dimension rearrangement operation. For ease of description, the above pooling, concatenation, and fully connected mapping processes can be uniformly represented as an asymmetric context reconstruction function:

[0121] ;

[0122] in, This represents an asymmetric context reconstruction function consisting of pooling, vector concatenation, fully connected mapping, and dimension rearrangement. Since the input of this function does not include the target fragment... This inherently avoids the network directly copying the original value of the target segment, allowing the residual from subsequent context reconstruction to be used to measure the deviation of the target segment from its preceding and following contexts, cross-metric consistency, and auxiliary operating state.

[0123] S2-5: Residual Calculation. Calculate three types of basic residuals: context reconstruction residuals, cross-index consistency residuals, and state matching residuals. Define the... Context reconstruction residuals for each window for:

[0124] ;

[0125] The larger the residual of this context reconstruction, the more difficult it is to interpret the target segment from context and operating condition information, and the more likely it is to be an anomaly. Define the first... Cross-indicator consistency residuals for each window for:

[0126] ;

[0127] The larger the consistency residual across indicators, the weaker the coupling relationship between the various water quality monitoring indicators within the window, and the more obvious the decoupling of a single water quality monitoring indicator.

[0128] Consistent summary vector across indicators Perform a linear mapping to obtain the predicted state summary vector. :

[0129] ;

[0130] in, and These are trainable parameters. Define the state matching residual. for:

[0131] ;

[0132] The larger the state matching residual, the more inconsistent the water quality changes within the window are with the current operating conditions, maintenance status, or link status.

[0133] S2-6: Residual Standardization and Anomaly Candidate Scoring. The three types of residuals—context reconstruction residuals, cross-index consistency residuals, and state matching residuals—are standardized. A residual statistical reference sample set is selected from the multi-source window sample set constructed in step S1 to calculate the mean and standard deviation of the three basic residuals. This residual statistical reference sample set can consist of historically stable operating window samples, labeled training window samples, or a combination thereof. This residual statistical reference sample set is used to calculate the distribution parameters of the three basic residuals and does not directly participate in subsequent hierarchical classification label learning. Let the mean and standard deviation of the context reconstruction residuals, cross-index consistency residuals, and state matching residuals calculated on the residual statistical reference sample set be respectively... , , ,in, , These represent the mean and standard deviation of the context reconstruction residuals, respectively. , These represent the mean and standard deviation of the consistency residuals across indicators, respectively. , Let these represent the mean and standard deviation of the state-matching residuals, respectively. Define the standardized context reconstruction residuals. Standardized cross-indicator consistency residuals Matching residuals with standardized states They are respectively:

[0134] ;

[0135] ;

[0136] ;

[0137] Then define the first Each window corresponds to anomaly candidate scores for the target segment. ,in, , and These are the weight coefficients of the standardized context reconstruction residual, the standardized cross-index consistency residual, and the standardized state matching residual, respectively, and they satisfy the following conditions: .when At that time, the first The target fragment corresponding to each window is identified as an abnormal candidate fragment and sent to the hierarchical classification module and step S3. This represents the threshold for abnormal candidate scores.

[0138] S2-7: Hierarchical Classification Module. To ensure that deep learning is the core classifier rather than an auxiliary module in this invention, a hierarchical classification module is further set up in step S2 to output probabilities at two levels for abnormal candidate segments. First, a window-level fused feature vector is constructed. :

[0139] ;

[0140] in, This represents a vector concatenation operation. This represents a pooling operation performed along the dimensions of water quality monitoring indicators. A primary classification head is constructed to determine whether the anomaly candidate fragment is more likely to belong to the category of real water quality events or equipment / data anomalies; its output is the primary classification probability. :

[0141] ;

[0142] in, This represents the normalized exponential function, used to convert a linear output into a probability distribution. This represents the trainable weight matrix of the first-level classifier head. This represents the bias vector of the first-level classification head.

[0143] The secondary classification head is constructed to further subdivide only the device / data anomaly category, outputting the probabilities of progressive drift, short-term contamination, and hard failure / loss of connection categories. The output is the secondary classification probability. :

[0144] ;

[0145] in, This represents the trainable weight matrix of the secondary classification head. This represents the bias vector of the secondary classification head.

[0146] S2-8: Network Training Objectives. To enable the hierarchical multi-source consistency deep network to possess hierarchical classification capabilities, context reconstruction capabilities, cross-metric consistency modeling capabilities, and state matching capabilities, the multi-source window sample set output from step S1 is used. Select anomaly candidate samples with known anomaly category labels to construct a training sample set. :

[0147] ;

[0148] in, Indicates the first A training multi-source window of samples, Indicates the training multi-source window sample sequence number. This indicates the corresponding primary category label. This indicates the corresponding secondary category label. This represents the number of training samples from the multi-source window. The primary classification label is... Used to characterize the Each training multi-source window sample corresponds to a target segment belonging to either a real water quality event or an equipment / data anomaly; secondary classification label. Used to characterize the Each training multi-source window sample corresponds to a target segment belonging to the progressive drift, short-term contamination, or hard fault / disconnection category. The labels can be obtained from historical maintenance records, manual annotation results, expert rule judgment results, or a combination thereof.

[0149] The first Training multi-source window samples Inputting a multi-source consistency deep network yields the first-level classification prediction probability. Secondary classification prediction probability Basic context reconstruction residual Basic cross-indicator consistency residuals Matching residuals with the base state .

[0150] Define the total loss function for:

[0151] ;

[0152] in, , , , , These are, respectively, the primary classification loss, the secondary classification loss, the context reconstruction loss, the cross-metric consistency loss, and the state matching loss. , , , , These are the non-negative weight coefficients for the primary classification loss, secondary classification loss, context reconstruction loss, cross-metric consistency loss, and state matching loss, respectively. The primary classification loss is:

[0153] ;

[0154] in, Indicates the first The training multi-source window samples belong to the first classification. The true label of the class, This indicates the category number in the primary classification.

[0155] The second-order classification loss is: ;in, Indicates the first The training multi-source window samples belong to the [number]th [class] in the secondary classification. The actual label of the class.

[0156] The context reconstruction loss is: ;

[0157] The cross-metric consistency loss is: ;

[0158] The state matching loss is: .

[0159] By minimizing the total loss function mentioned above, the hierarchical multi-source consistency deep network can simultaneously learn anomaly classification, context reconstruction, cross-metric consistency maintenance, and state matching constraints during training, and provide a basic residual for anomaly candidate scores.

[0160] S2-9: Network Training and Parameter Update. Define the parameter set of a hierarchical multi-source consistency deep network as follows: This includes the parameter sets for the input embedding module, the multi-scale temporal coding module, the cross-index consistency graph coding module, the asymmetric context reconstruction module, and the hierarchical classification module. The total loss function is used... To optimize the objective, gradient descent, stochastic gradient descent, or the Adam optimization algorithm are used to optimize the parameter set. Perform iterative updates until the total loss function converges or the preset number of training epochs is reached, resulting in the completed target parameter set. :

[0161] .

[0162] In a preferred embodiment, validation samples can be further divided from the training sample set to monitor the loss convergence process and determine the time to stop training; however, the present invention is not limited to a specific method of dividing the training set, validation set, or test set. For the first... One unprocessed multi-source window sample ,in, For the first One standardized water quality monitoring window awaiting processing. For the first A pending auxiliary running window, For the first One unprocessed missing test mask window, This indicates the sample number to be processed, used to distinguish it from the sample number used during the training phase. The first... The standardized water quality monitoring window to be processed is further represented as follows:

[0163] ;

[0164] in, for The left context, for The target segment, for The right context. The "first" mentioned in subsequent steps. "One target segment to be processed" or simply "target segment to be processed" refers to... Similarly, , .in, To and The auxiliary running window corresponding to the time, To and The auxiliary running window corresponding to the time, To and The auxiliary running window corresponding to the time; , and They are respectively with , and The missing test mask window in terms of time.

[0165] Will The set of input target parameters is The hierarchical multi-source consistency deep network is obtained to obtain the first Reconstruction results of the target fragments to be processed , No. Standardized context reconstruction residuals of each target segment to be processed , No. Standardized cross-index consistency residuals of each target segment to be processed , No. Standardized state matching residuals of each target segment to be processed , No. Anomaly candidate scores for each target segment to be processed , No. The first-level classification probability of each target fragment to be processed and the Secondary classification probability of each target fragment to be processed Will satisfy The target segment to be processed is identified as an abnormal candidate segment and input into step S3; for those that do not meet the requirements... The target fragment to be processed is not subjected to the anomaly classification in step S3, but is directly regarded as a non-anomaly candidate fragment, and its original value is retained in subsequent processing. The multi-source window sample to be processed... The samples can be window samples that did not participate in the current round of training from the multi-source window sample set output in step S1, or window samples that are added during subsequent online monitoring and processed by step S1.

[0166] Step S3: Anomaly Classification of Candidate Segments. Step S3 aims to determine the anomaly category label of the target segment to be processed, based on the reconstruction results, residuals, and hierarchical classification probabilities output in Step S2, according to a preset priority order, for the target segment identified as anomaly candidate segments. The anomaly category label of the target fragment to be processed is used as the input for step S4. Let the first-level classification probability of the target fragment output in step S2 be:

[0167] ;

[0168] in, This indicates the probability that the target segment to be processed belongs to the category of real water quality events. This represents the probability that the target segment to be processed belongs to the device / data anomaly category. Let the secondary classification probability of the target segment to be processed output in step S2 be:

[0169] ;

[0170] in, Indicates the probability of asymptotic drift. This indicates the probability of short-term contamination. This represents the probability of a hard fault / loss of connection. In this invention, the anomaly category label of the target segment to be processed... The system employs a sequential discrimination method, classifying data according to the following priority order: real water quality events, hard faults / loss of connection, gradual drift, and short-term fouling. Once the criteria for a previous category are met, the criteria for a subsequent category are not processed. This ensures that the same target segment ultimately corresponds to only one anomaly category label.

[0171] S3-1: Establishment of a Four-Category Anomaly Classification System. To ensure that each anomaly category corresponds one-to-one with subsequent processing actions, this invention uniformly classifies candidate anomaly fragments into four categories: real water quality events, gradual drift, short-term fouling, and hard faults / loss of connection. When, it indicates that the target segment to be processed belongs to the category of real water quality events; when When, it indicates that the target segment to be processed belongs to the progressive drift category; when When, it indicates that the target segment to be processed belongs to the short-term contamination category; when When the time is specified, it indicates that the target segment to be processed belongs to the hard fault / disconnection category. Among them, the real water quality event category is used to represent real abnormal changes that occur in the monitored object itself; the gradual drift category is used to represent abnormalities in which the sensor response gradually deviates from the baseline over time; the short-term fouling category is used to represent fragmented abnormalities caused by local adhesion, local obstruction, local pollution interference, or short-term noise; and the hard fault / disconnection category is used to represent abnormalities such as missing measurement, communication interruption, constant value output, jamming, or other measurement chain interruption.

[0172] S3-2: Construction of Fragment-Level Auxiliary Discriminant. To enhance the interpretability and stability of the hierarchical classification results, the following auxiliary discriminant is further constructed based on the output of step S2. First, define the segment-level auxiliary discriminant in the target fragment to be processed. Time-by-time reconstruction error at each moment for:

[0173] ;

[0174] in, Indicates the target fragment to be processed. The Column vector, Indicates the target fragment to be reconstructed. The Column vector. Further, define the anomaly duration ratio. for:

[0175] ;

[0176] in, To reconstruct the error threshold time-by-time, Let be the indicator function. The anomaly persistence ratio is used to measure the persistence of an anomaly in the target segment to be processed. Let be... Indicates the first One target fragment to be processed The Middle The water quality monitoring indicators are in the first Standardized observations at each time point express The reconstructed value at the corresponding position. Define the time difference between adjacent moments of the target segment to be processed. for:

[0177] ;

[0178] in, Indicates the first One target fragment to be processed The Middle The water quality monitoring indicators are in the first Standardized observations at each time point;

[0179] Based on the difference between adjacent time points, the following is defined:

[0180] ;

[0181] ;

[0182] ;

[0183] in, For direction discrimination threshold, The proportion of consistency in the upward direction. For the consistency ratio of the descent direction, The directional consistency ratio is used to measure whether the target segment to be processed exhibits a continuous unidirectional shift as a whole, thus characterizing the gradual drift feature. Local fluctuation intensity is defined. for:

[0184] ;

[0185] The local fluctuation intensity is used to measure the degree of local instability within the target segment to be processed, in order to distinguish between slow migrations and short-term disturbances. The proportion of missing data in the target segment to be processed is defined. for:

[0186] ;

[0187] in, Indicates the missing test mask for the target segment to be processed. The element at the corresponding position in the middle. Defines the flatness ratio in the target segment to be processed. for:

[0188] ;

[0189] in, The flatness threshold is used to determine flatness. The flatness ratio is used to identify stuck, constant value output, or other hard fault phenomena.

[0190] S3-3: Distinguishing between real water quality events and equipment / data anomalies. In this step, primary classification is performed by combining primary classification probabilities and rule constraints. If the following conditions are met simultaneously:

[0191] ;

[0192] Then the target segment to be processed is determined to be a real water quality event, that is... .in, This is the probability threshold for the first-level classification. The threshold for cross-index consistency residuals. For state matching residual threshold, The threshold for the proportion of missing measurements. This represents the flatness ratio threshold. The above judgment logic indicates that if the target segment to be processed is more inclined to the category of real water quality events in the primary classification, and its cross-indicator consistency and state matching relationship are good, and there is no significant missing measurement or significant flatness phenomenon, then it can be determined that the anomaly of the target segment to be processed originates from the real changes of the monitoring object itself, rather than equipment or data anomalies.

[0193] S3-4: Priority Judgment of Hard Faults / Loss of Contact. If step S3-3 does not determine it to be a real water quality event, then further priority is given to determining whether the target segment to be processed belongs to the hard fault / loss of contact category. If any of the following conditions are met:

[0194] ;

[0195] or

[0196] ;

[0197] or

[0198] and ;

[0199] The target segment to be processed is then determined to be a hard fault / loss of connection type, i.e. .in, The threshold for the proportion of missing measurements. The flatness ratio threshold, This is the probability threshold for the secondary classification of hard faults / loss of connection. The above discrimination logic means that if the target segment to be processed has significant missing data, significant flatness, or is identified as a hard fault / loss of connection with the highest probability in the secondary classification, it should be preferentially classified into the hard fault / loss of connection category to avoid subsequent erroneous automatic calibration.

[0200] S3-5: Differentiation between gradual drift and short-term fouling. If the target segment to be processed does not meet the discrimination criteria for the real water quality event category in step S3-3, nor the discrimination criteria for the hard fault / disconnection category in step S3-4, then a secondary subdivision is performed between the gradual drift and short-term fouling categories. If the following conditions are met simultaneously:

[0201] ;

[0202] Then the target segment to be processed is determined to be of the progressive drift type, that is... .in, The threshold for the proportion of abnormal occurrences. This is the threshold for the directional consistency ratio. This is the threshold for local fluctuation intensity. The above judgment logic indicates that if the target segment to be processed continuously deviates from the context over a relatively long period of time, and the overall direction of change has strong consistency, while the local fluctuations are not large, then it is more consistent with the characteristics of gradual drift accumulating gradually. If the target segment to be processed is neither judged as a real water quality event nor as a hard fault / loss, and does not meet the judgment criteria for gradual drift, then it is judged as a short-term fouling event. .

[0203] Step S4: Perform gated calibration decision based on anomaly typing results. Step S4 aims to determine the anomaly category label output from Step S3. To generate gating calibration decisions for the target fragment to be processed after the anomaly classification is completed in step S3. Furthermore, the scope of subsequent corrections will be determined to ensure that real water quality events are not miscorrected, correctable drift is automatically calibrated, short-term fouling is locally repaired, and only missing reconstruction is performed for hard faults / loss of connection.

[0204] S4-1: Definition of Gating Calibration Decision. The gating calibration decision corresponding to the target segment to be processed is defined as follows: .when When, it means to retain the original value and not perform automatic calibration; when When, it indicates that automatic calibration is being performed; when When, it indicates that a partial repair is being performed; when When this is the case, it indicates that automatic calibration is disabled, and missing reconstruction is only allowed for the failed locations.

[0205] S4-2: Mapping of anomaly categories to gating decisions. Based on the anomaly category labels output in step S3. Establish the following gating mapping relationship:

[0206] ;

[0207] The engineering implications of this mapping relationship are as follows. When At that time, the target segment to be processed belongs to the category of real water quality events, and the changes in the segment reflect the actual evolution of the monitored object itself. Therefore, step S4 outputs... Automatic calibration is not allowed thereafter to avoid incorrectly correcting real events to a normal state; when At this time, the target segment to be processed belongs to the progressive drift category. The segment as a whole exhibits continuous shift with a consistent direction, making it suitable for automatic calibration of the original observations through reconstruction results. Therefore, step S4 outputs... ;when When the target segment to be processed is classified as short-term contamination, the segment typically exhibits anomalies only in localized locations or time periods. Therefore, automatic calibration of the entire segment is not recommended; instead, localized repair should be performed only on the anomalous locations. Thus, step S4 outputs... ;when At this time, the target segment to be processed belongs to the hard fault / disconnection category. The segment has measurement chain interruption phenomena such as missing measurement, jamming, constant value output or communication interruption. Automatic calibration of failed observations is not allowed. Only missing reconstruction of the failure location is allowed. Therefore, step S4 outputs .

[0208] S4-3: Construction of the position-level correction mask. To ensure that subsequent step S5 can accurately determine the correction range, the gating calibration decision... Based on this, a position-level correction mask matrix corresponding to the target fragment to be processed is further constructed. :

[0209] ;

[0210] in, Represents the position-level correction mask matrix The elements in, when When, it indicates the first The water quality monitoring indicators are in the target segment to be treated. The position at each moment allows for correction in subsequent steps; when When this occurs, it indicates that the original value is retained at the corresponding position. This is based on the gating calibration decision obtained in step S4-2. Generate according to the following segmentation rules :

[0211] ;

[0212] in, To automatically calibrate deviations from the threshold, To repair local deviations from the threshold, Represents the characteristic function, For the missing test mask window corresponding to the target segment to be processed Middle position Element.

[0213] The meaning of the above segmentation rule is: when When, corresponding to real water quality events, all locations do not participate in the correction; when When, corresponding to the progressive drift type, automatic calibration is only enabled for positions that deviate from the reconstructed baseline and are not missing measurements; when For short-term contamination, local repair is only available for locations with significant local deviations, large hourly reconstruction errors, and no missing measurements; when In the case of hard faults / disconnection, missing reconstruction is only enabled for locations with missing measurements or invalid locations.

[0214] S4-4: Construction of the modified strength coefficients. To ensure that subsequent modifications are not limited to discrete operations of "complete modification" or "no modification," a modified strength coefficient matrix is ​​further constructed. :

[0215] ;

[0216] in, Represents the modified intensity coefficient matrix The elements in, when When, it indicates that no correction will be performed at that position; when When, it indicates that the reconstructed value is used completely at that position; when When, it indicates that the original standardized observation value and the reconstructed value are fused at that location according to the corresponding ratio. The segmentation rules are as follows:

[0217] ;

[0218] The above segmentation rules mean that: for real water quality events, no correction is performed; for gradual drift events, the soft calibration intensity is determined based on the deviation between the original standardized observation value and the reconstructed value; for short-term fouling events and hard faults / disconnection events, for all locations where the location-level correction mask allows correction, the reconstructed value in the subsequent steps is used at full intensity for local repair or missing reconstruction.

[0219] Step S5: Generate corrected results, confidence scores, and confidence data based on the gating calibration decision. Step S5 aims to build upon the anomaly category labels output in Step S3. The gating calibration decision output in step S4 Position-level correction mask matrix and the modified intensity coefficient matrix First, generate the corrected result of the target segment to be processed. Then, combining the classification probability and residual output from step S2, a credibility score is calculated for each target segment to be processed. The correction results of multiple overlapping segments are then fused into a globally standardized credibility sequence, ultimately yielding credible data. Indicates the first The final corrected result of the target fragment to be processed in the normalized space, its i.e. The water quality monitoring indicators are in the first The correction value at each local time is denoted as For target segments not identified as anomalous candidate segments, the anomalous classification in step S3 and the gating calibration decision in step S4 are not performed; instead, the original target segment is directly retained, i.e., let And make it adjust the ratio. In a preferred embodiment, the credibility score of the target fragment to be processed can be set to a preset default value of 1.

[0220] S5-1: Generation of the correction result for the target segment to be processed. For any position in the target segment to be processed... Its correction value Defined as:

[0221] ;

[0222] in, For the target fragment to be processed Middle position elements, The reconstruction result of the target fragment to be processed Middle position elements, For position-level correction masks, To correct the intensity coefficient. The meaning of the above unified expression is: when a certain position is not allowed to be corrected, that is... If the original value is retained, then the original value is retained; if a certain position allows for correction, and the target segment to be processed belongs to the progressive drift type or the short-term contamination type, then... and Then, the original value and the reconstructed value are fused according to the correction intensity coefficient; when a certain position is allowed to be corrected, and the target segment to be processed belongs to the hard fault / disconnection category, i.e. and If the value is missing, the reconstructed value is directly used as the missing reconstruction result. Furthermore, the first... Correction ratio of each target segment to be processed for:

[0223] ;

[0224] The correction ratio This indicates the proportion of the target segment to be corrected, and will be used in subsequent credibility scoring.

[0225] S5-2: Calculation of Category Path Confidence and Credibility Score. Because this invention employs a hierarchical classification structure, the final category confidence of a target fragment depends not only on the probability of a single-level classification but also on its joint probability along the hierarchical classification path. Define the... The class path confidence of each target fragment to be processed is The calculation method is as follows:

[0226] ;

[0227] To avoid ambiguity in the credibility score when the standardized residuals are negative, the non-negative residual burden is defined as follows:

[0228] ;

[0229] ;

[0230] ;

[0231] in, , , These represent the residual burden of context reconstruction, the residual burden of cross-metric consistency, and the residual burden of state matching, respectively. , , They represent the first The standardized context reconstruction residual, standardized cross-index consistency residual, and standardized state matching residual of each target fragment to be processed are further defined. The credibility score of each target fragment to be processed is: The calculation formula is as follows:

[0232] ;

[0233] in, For the Sigmoid function, , , , , They are respectively , , , , The non-negative weighting coefficients. The meaning of the above credibility scores is as follows: the higher the confidence of the category path, the higher the credibility score of the corresponding target segment to be processed; the greater the residual burden of context reconstruction, the residual burden of cross-index consistency, the residual burden of state matching, and the correction ratio, the lower the credibility score of the corresponding target segment to be processed.

[0234] S5-3: Global Fusion of Overlapping Window Correction Results. Since different target segments may overlap on the time axis when constructing them using a sliding window approach, it is necessary to fuse the correction results of each target segment into a globally standardized reliable sequence. For target segments identified as real water quality events, and those not identified as anomalous candidate segments, their corresponding positions are preferentially preserved as original standardized observations during global fusion; reliability-weighted fusion is only performed on segments requiring automatic calibration, local repair, or missing segment reconstruction.

[0235] Define a set , indicating all coverage of the first Each water quality monitoring indicator at global time The set of fragment locations. The elements in are , indicating the first The first of the target segments to be processed Local time position and global time correspond.

[0236] For the Water quality monitoring indicators and global time Define the standardized credibility value after fusion. : When the global time When the corresponding location belongs to a real water quality event segment or a non-abnormal candidate segment, ;otherwise, ;

[0237] in, Indicates the first The first of the target segments to be processed The water quality monitoring indicators are in the first The correction value at a local moment. It is a constant, representing a very small positive number to prevent the denominator from being zero;

[0238] This yields a globally standardized reliable sequence. :

[0239] ;

[0240] in, Indicates the global synchronous sampling time. Standardized and reliable water quality monitoring vectors ; , ;

[0241] This fusion method uses a credibility score for the target fragment to be processed. The weights are assigned to segments with higher credibility, thus giving them a greater weight in the global fusion process.

[0242] S5-4: Denormalization of the globally normalized trusted sequence and output of trusted data. To facilitate the subsequent system's direct use of data in physical dimensions, the globally normalized trusted sequence... Performing destandardization yields a globally reliable water quality monitoring sequence. For the first Each water quality monitoring indicator at time credible water quality monitoring values Defined as:

[0243] ;

[0244] Therefore, there is

[0245] ;

[0246] in, Indicates the global synchronization sampling time credible water quality monitoring vectors .

[0247] Finally, reliable data is output. This reliable data includes the following: raw water quality monitoring data. Standardized corrected fragment results Globally standardized trusted sequences Globally reliable water quality monitoring sequence Exception category tags Gating calibration decision and credibility score .

[0248] In one specific embodiment, such as Figure 2 As shown, for real water quality events, the globally reliable water quality monitoring sequence preserves the original real changes; for gradual drift events, the globally reliable water quality monitoring sequence calibrates the drifted part; for short-term fouling events, the globally reliable water quality monitoring sequence repairs local abnormal locations; and for hard faults / disconnection events, the globally reliable water quality monitoring sequence reconstructs the failed segments, thus obtaining more reliable water quality monitoring results.

[0249] The present invention also provides a system corresponding to the method, comprising:

[0250] The multi-source window sample generation unit is used to map the raw water quality monitoring data and auxiliary operation data to the same time reference and perform window segmentation to form multi-source window samples that can be directly input into the subsequent deep network.

[0251] A network construction and training unit is used to construct a hierarchical multi-source consistency deep network and train the network using multi-source window samples. The hierarchical multi-source consistency deep network includes: an input embedding module, a multi-scale temporal coding module, a cross-index consistency graph coding module, an asymmetric context reconstruction module, and a hierarchical classification module. The input embedding module outputs an initial embedding feature tensor of the multi-source window samples based on the multi-source window samples. The multi-scale temporal coding module models the temporal evolution of each water quality monitoring index in the initial embedding feature tensor to obtain a temporal coding feature tensor. The cross-index consistency graph coding module obtains the cross-index consistency graph coding result based on the temporal coding feature tensor. The asymmetric context reconstruction module reconstructs the target segment using an asymmetric context reconstruction method to obtain the target segment reconstruction result. Based on the standardized context reconstruction residual, standardized cross-index consistency residual, and standardized state matching residual, the abnormal candidate score of the corresponding target segment is obtained. The hierarchical classification module outputs two levels of probability for the abnormal candidate segments.

[0252] The anomaly category label determination unit is used to determine the anomaly category label of the corresponding target segment according to a preset priority order based on the anomaly candidate score, reconstruction result, residual amount and two-level probability of the multi-source window sample to be processed.

[0253] The correction range determination unit is used to generate a gating calibration decision for each target segment to be processed based on the anomaly category label, and to construct the position-level correction mask matrix and correction intensity coefficient matrix corresponding to the target segment to be processed based on the gating calibration decision.

[0254] The trusted data generation unit is used to generate the correction results of the target segment to be processed based on the anomaly category label, gating calibration decision, position-level correction mask matrix and correction intensity coefficient matrix. Then, it calculates the trust score for each target segment to be processed by combining two-level probability and residual. The correction results of multiple overlapping segments are merged into a globally standardized trusted sequence to finally obtain trusted data.

[0255] The present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the method described.

Claims

1. A method for online water quality monitoring anomaly classification correction and reliable data generation, characterized in that, Includes the following steps: The raw water quality monitoring data and auxiliary operation data are mapped to the same time reference and windowed to form multi-source window samples that can be directly input into the subsequent deep network. Construct a hierarchical multi-source consistency deep network and train the network using multi-source window samples; The hierarchical multi-source consistency deep network includes: an input embedding module, a multi-scale temporal coding module, a cross-index consistency graph coding module, an asymmetric context reconstruction module, and a hierarchical classification module. The input embedding module outputs an initial embedding feature tensor for the multi-source window samples. The multi-scale temporal coding module models the temporal evolution of each water quality monitoring index in the initial embedding feature tensor to obtain a temporal coding feature tensor. The cross-index consistency graph coding module obtains the cross-index consistency graph coding result based on the temporal coding feature tensor. The asymmetric context reconstruction module reconstructs the target segment using an asymmetric context reconstruction method to obtain the target segment reconstruction result. Based on the standardized context reconstruction residual, standardized cross-index consistency residual, and standardized state matching residual, anomaly candidate scores for the corresponding target segment are obtained. The hierarchical classification module outputs two levels of probabilities for the anomaly candidate segments. For the anomaly candidate scores, reconstruction results, residuals, and two-level probabilities of the multi-source window samples to be processed, the anomaly category labels of the corresponding target segments to be processed are determined according to a preset priority order. Based on the anomaly category label, a gating calibration decision is generated for each target segment to be processed. Based on the gating calibration decision, a position-level correction mask matrix and a correction intensity coefficient matrix corresponding to the target segment to be processed are constructed. Based on anomaly category labels, gating calibration decisions, location-level correction mask matrices, and correction intensity coefficient matrices, the correction results of the target segments to be processed are first generated. Then, by combining two-level probabilities and residuals, a credibility score is calculated for each target segment to be processed. The correction results of multiple overlapping segments are then fused into a globally standardized credibility sequence, and finally, credibility data is obtained.

2. The method according to claim 1, characterized in that, Multi-source window sample formation methods include: Collect raw water quality monitoring data output from each sensor in the online monitoring system, as well as auxiliary operational data related to the monitoring process; The original water quality monitoring data and auxiliary operation data are mapped onto a unified time axis to obtain the water quality monitoring synchronization sequence and the auxiliary operation synchronization sequence. Define a missing test mask matrix. When any indicator is missing or invalid at any time, the matrix element is defined as 1, otherwise it is 0. For each water quality monitoring indicator, a reference statistic is constructed, including the reference median. and reference interquartile range ; Robust standardization was performed on the water quality monitoring synchronization sequence and the auxiliary operation synchronization sequence to obtain standardized water quality monitoring sequence and standardized auxiliary operation data. A sliding window approach was used to segment the standardized water quality monitoring sequence, standardized auxiliary operation data, and missing measurement mask matrix into windows, constructing a standardized water quality monitoring window, an auxiliary operation window, and a missing measurement mask window, thus obtaining a multi-source window sample set.

3. The method according to claim 1, characterized in that, The input embedding module is used to input the first standardized water quality monitoring window. The first row sequence, auxiliary running window, and missing test mask window row sequence Input the embedding function to get the first The initial embedding features corresponding to each water quality monitoring indicator are then stacked according to the indicator dimension to obtain the first... The initial embedding feature tensor of a multi-source window sample; The cross-indicator consistency graph encoding module, firstly, for the first... Each water quality monitoring indicator is in the window The temporal coding features in the data are pooled along the time dimension to obtain the node representation of the water quality monitoring index; then, a dynamic adjacency matrix is ​​constructed through attention similarity; further, graph aggregation is performed on the node representations to obtain the graph coding result; and all node graph representations are stacked to obtain the cross-index consistent graph coding result. The asymmetric context reconstruction module uses asymmetric context reconstruction functions to reconstruct the target fragment, thus obtaining the reconstructed target fragment. The asymmetric context reconstruction function takes the left context, right context, cross-index consistency graph encoding results, and auxiliary running window as input, and outputs the target fragment reconstruction result; The hierarchical classification module first constructs a window-level fused feature vector and then builds a first-level classification head to determine whether the abnormal candidate fragment is more likely to belong to the category of real water quality events or equipment / data anomalies. Its output is the first-level classification probability. The second-level classification head is then constructed to further subdivide only the equipment / data anomaly category, outputting the probabilities of progressive drift, short-term fouling, and hard failure / disconnection. Its output is the second-level classification probability.

4. The method according to claim 1, characterized in that, Network total loss function for: ； in, , , , , These are, respectively, the primary classification loss, the secondary classification loss, the context reconstruction loss, the cross-metric consistency loss, and the state matching loss. , , , , These are the non-negative weight coefficients for the first-level classification loss, second-level classification loss, context reconstruction loss, cross-index consistency loss, and state matching loss, respectively. The first-level classification loss is: ;in, Indicates the number of training multi-source window samples. Indicates the first The training multi-source window samples belong to the first classification. The true label of the class, Indicates the probability of prediction for the primary classification; The second-order classification loss is: ;in, Indicates the first The training multi-source window samples belong to the [number]th [class] in the secondary classification. The true label of the class, Indicates the probability of secondary classification prediction; The context reconstruction loss is: ,in, Represents the residual from the underlying context reconstruction; The cross-metric consistency loss is: ,in, This represents the basic cross-indicator consistency residual; The state matching loss is: ,in, Represents the basic state matching residual; By minimizing the total loss function, the hierarchical multi-source consistency deep network can simultaneously learn anomaly candidate identification, anomaly classification, context reconstruction, cross-metric consistency maintenance, and state matching constraints during training.

5. The method according to claim 1, characterized in that, Training the network using multi-source window samples includes: Define the network parameter set as follows This includes the parameter set of the input embedding module, the parameter set of the multi-scale temporal coding module, the parameter set of the cross-index consistency graph coding module, the parameter set of the asymmetric context reconstruction module, and the parameter set of the hierarchical classification module. Using the total loss function To optimize the objective, gradient descent, stochastic gradient descent, or the Adam optimization algorithm are used to optimize the parameter set. Perform iterative updates until the total loss function converges or the preset number of training epochs is reached, resulting in the completed target parameter set. : 。 6. The method according to claim 1, characterized in that, Based on the anomaly candidate scores, reconstruction results, residuals, and two-level probabilities of the multi-source window samples to be processed, the anomaly category labels of the corresponding target segments to be processed are determined according to a preset priority order; including: Abnormal candidate segments are uniformly divided into four categories: real water quality events, gradual drift, short-term fouling, and hard failures / loss of contact. Construct segment-level auxiliary discrimination parameters, including: time-by-time reconstruction error in the target segment to be processed, proportion of abnormal persistence, difference between adjacent time moments of the target segment to be processed, proportion of consistency in the rising direction, proportion of consistency in the falling direction, proportion of consistency in the direction, intensity of local fluctuations, proportion of missing data in the target segment to be processed, and proportion of flatness in the target segment to be processed. Distinguishing between real water quality incidents and equipment / data anomalies: If the following conditions are met simultaneously: ； Then the target segment to be processed is determined to be a real water quality event, where, This indicates the probability that the target segment to be processed belongs to the category of real water quality events. This indicates the probability that the target fragment to be processed belongs to the device / data anomaly category. The standardized cross-index consistency residuals of the target segment to be processed. To match the standardized state residuals of the target segment to be processed, This represents the proportion of missing data in the target segment to be processed. The flatness ratio in the target segment to be processed. This is the probability threshold for the first-level classification. The threshold for cross-index consistency residuals. For state matching residual threshold, The threshold for the proportion of missing measurements. Indicates the flatness ratio threshold; Priority judgment for hard failures / loss of contact: If it is not judged as a real water quality event and meets any of the following conditions: ;or ;or and ; The target segment to be processed is then determined to be a hard fault / loss of connection type, where, Indicates the probability of asymptotic drift. This indicates the probability of short-term contamination. Indicates the probability of hard failure / loss of connection. The probability threshold for secondary classification of hard faults / loss of contact; Distinguishing between gradual drift and short-term fouling: If the target segment to be treated is not identified as a real water quality event, nor as a hard fault / loss, and simultaneously meets the following conditions: ； Then the target segment to be processed is determined to be of the progressive drift type, where, The percentage of abnormal persistence. For the proportion of directional consistency, The threshold for the proportion of abnormal occurrences. This is the threshold for the directional consistency ratio. For local fluctuation intensity, The threshold is set as the local fluctuation intensity. If the target segment to be processed is neither identified as a real water quality event, nor as a hard fault / disconnection, nor as a gradual drift, then it is identified as a short-term fouling segment.

7. The method according to claim 1, characterized in that, Based on the anomaly category label, a gating calibration decision is generated for each target segment to be processed. Based on the gating calibration decision, a position-level correction mask matrix and a correction intensity coefficient matrix corresponding to the target segment to be processed are constructed; including: Define the gating calibration decision corresponding to the target segment to be processed as follows: ,when When, it means to retain the original value and not perform automatic calibration; when When, it indicates that automatic calibration is being performed; when When, it indicates that a partial repair is being performed; when When this is the case, it indicates that automatic calibration is disabled, and missing reconstruction is only allowed for the failed location; Based on the anomaly category label Establish the following gating mapping relationship: ； in, This indicates the anomaly category label of the target fragment to be processed. When, it indicates that the target segment to be processed belongs to the category of real water quality events; when When, it indicates that the target segment to be processed belongs to the progressive drift category; when When, it indicates that the target segment to be processed belongs to the short-term contamination category; when When this occurs, it indicates that the target segment to be processed belongs to the category of hard fault / loss of connection; In gating calibration decision Based on this, a position-level correction mask matrix corresponding to the target segment to be processed is further constructed: when the matrix element is 1, it means that the position of the water quality monitoring index at the corresponding time of the target segment to be processed is allowed to be corrected in subsequent steps; when the matrix element is 0, it means that the original value is retained at the corresponding position. when When, the correction mask for real water quality events is defined as 0; when When, automatic calibration is enabled for locations that deviate from the reconstructed baseline but are not missing measurements; when When, local repair is enabled at the location of the local anomaly; when At that time, only the failed location is open for missing reconstruction; Construct the modified intensity coefficient matrix , Represents the modified intensity coefficient matrix The elements in The segmentation rules are as follows: ； in, Represents the position-level correction mask matrix The elements in Indicates the first One target fragment to be processed The Middle The water quality monitoring indicators are in the first Standardized observations at each time point express The reconstructed value at the corresponding position, This indicates that the automatic calibration deviates from the threshold. It is a constant.

8. The method according to claim 1, characterized in that it is reliable. The data generation process is as follows: Generate the correction result of the target segment to be processed: when a certain position is not allowed to be corrected, the original value is directly retained; when a certain position is allowed to be corrected, and the target segment to be processed belongs to the progressive drift type or short-term contamination type, the original value and the reconstructed value are fused according to the correction intensity coefficient. When a certain position allows for correction, and the target segment to be processed belongs to the hard fault / loss category, the reconstructed value is directly used as the missing reconstruction result; Calculate the category path confidence and credibility score; The correction results of each target segment to be processed are merged into a globally standardized reliable sequence; Perform denormalization on the globally standardized reliable sequence to obtain a globally reliable water quality monitoring sequence; Finally, a reliable water quality dataset is output; the reliable water quality dataset includes: original water quality monitoring data, standardized corrected fragment results, globally standardized reliable sequences, globally reliable water quality monitoring sequences, anomaly category labels, gating calibration decisions, and reliability scores.

9. An online water quality monitoring anomaly classification correction and reliable data generation system, characterized in that, include: The multi-source window sample generation unit is used to map the raw water quality monitoring data and auxiliary operation data to the same time reference and perform window segmentation to form multi-source window samples that can be directly input into the subsequent deep network. The network construction and training unit is used to construct a hierarchical multi-source consistent deep network and train the network using multi-source window samples. The hierarchical multi-source consistency deep network includes: an input embedding module, a multi-scale temporal coding module, a cross-index consistency graph coding module, an asymmetric context reconstruction module, and a hierarchical classification module. The input embedding module outputs an initial embedding feature tensor for the multi-source window samples. The multi-scale temporal coding module models the temporal evolution of each water quality monitoring index in the initial embedding feature tensor to obtain a temporal coding feature tensor. The cross-index consistency graph coding module obtains the cross-index consistency graph coding result based on the temporal coding feature tensor. The asymmetric context reconstruction module reconstructs the target segment using an asymmetric context reconstruction method to obtain the target segment reconstruction result. Based on the standardized context reconstruction residual, standardized cross-index consistency residual, and standardized state matching residual, anomaly candidate scores for the corresponding target segment are obtained. The hierarchical classification module outputs two levels of probabilities for the anomaly candidate segments. The anomaly category label determination unit is used to determine the anomaly category label of the corresponding target segment according to a preset priority order based on the anomaly candidate score, reconstruction result, residual amount and two-level probability of the multi-source window sample to be processed. The correction range determination unit is used to generate a gating calibration decision for each target segment to be processed based on the anomaly category label, and to construct the position-level correction mask matrix and correction intensity coefficient matrix corresponding to the target segment to be processed based on the gating calibration decision. The trusted data generation unit is used to generate the correction results of the target segment to be processed based on the anomaly category label, gating calibration decision, position-level correction mask matrix and correction intensity coefficient matrix. Then, it calculates the trust score for each target segment to be processed by combining two-level probability and residual. The correction results of multiple overlapping segments are merged into a globally standardized trusted sequence to finally obtain trusted data.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1-8.

Images