Water leakage anomaly identification method for water supply network based on NMF dimension reduction

Abstract

The application provides a water supply network leakage anomaly identification method based on NMF dimension reduction, and relates to the technical field of water supply network anomaly identification, and comprises the following steps: embedding water listening sample data containing expert annotation information into a sample weight matrix as a row vector to construct a sample dimension loss term; mapping water listening sample data containing industry feature priori to a feature basis matrix as a row to construct a feature dimension loss term; combining the two loss terms to construct a total loss function and solving the total loss function to obtain an optimal sample weight matrix; performing clustering analysis based on the optimal sample weight matrix to obtain clustering attribution results of the water listening sample data, so as to determine whether a water supply network to be identified has a leakage anomaly. The application constructs a double-constraint NMF framework that fuses expert knowledge of a sample dimension and industry priori of a feature dimension, takes into account the accuracy and interpretability of water listening feature dimension reduction, and solves the problems of poor interpretability and less use of priori of existing methods.

Description

Technical Field

[0001] This invention relates to the field of water supply network anomaly identification technology, specifically a water supply network leakage anomaly identification method based on NMF dimensionality reduction. Background Technology

[0002] Leaks in urban water supply networks can easily lead to water waste and economic losses. Hydrophone testing, as a non-invasive method for analyzing network leaks and anomalies, determines leaks by collecting multi-source non-negative features such as time-domain energy, frequency-domain power, and hydraulic parameters. Due to its ease of operation and lack of network damage, it has become the mainstream technology for leak detection. However, existing feature reduction and anomaly analysis technologies have many core pain points in engineering applications and are difficult to adapt to actual detection needs.

[0003] Leakage samples in pipeline networks are naturally scarce. Currently, only a small number of expert-annotated leakage / normal samples can be obtained through annotation based on expert experience. Traditional unsupervised dimensionality reduction methods cannot utilize the value of expert annotations. Deep learning models require a large amount of labeled data for training, and their generalization ability is weak in scenarios with few samples, making leakage identification prone to missed or false positives. The water industry has formed a mature leakage analysis feature classification system, which has clarified the core feature categories and physical relationships. However, existing dimensionality reduction algorithms have not incorporated such industry priors into the model design. Core leakage discrimination features are easily interfered with by environmental noise, resulting in a high false positive rate in high-noise scenarios.

[0004] In summary, existing NMF-based hydrophone feature dimensionality reduction methods have shortcomings in interpretability, prior knowledge utilization, discriminative power, and computational efficiency. They struggle to balance accuracy and interpretability in dimensionality reduction, failing to adequately meet the practical engineering needs of leak detection in water supply networks. Therefore, a dual-constraint NMF dimensionality reduction method that integrates expert knowledge in the sample dimension and industry prior knowledge in the feature dimension is urgently needed to improve the performance and interpretability of leak detection. Summary of the Invention

[0005] The purpose of this invention is to provide a water supply network leakage anomaly identification method based on NMF dimensionality reduction, which addresses the problems of poor interpretability of dimensionality reduction results, high algorithm complexity, and limited utilization of prior knowledge in existing water listening leakage anomaly identification methods.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following solution:

[0007] A method for identifying leaking anomalies in water supply networks based on NMF dimensionality reduction includes: a nonnegative matrix factorization (NMF) model composed of a sample weight matrix and a feature basis matrix; hydrophone sample data containing partial expert annotation information; and hydrophone sample data containing a subset of core features with prior industry characteristics. The method comprises the following steps:

[0008] S1: Embed the hydrophone sample data containing expert annotation information into the sample weight matrix as a row vector to construct the sample dimension loss term;

[0009] S2: Embed the hydrophone sample data containing the core feature subset with industry feature priors into the mapping feature basis matrix as rows to construct the feature dimension loss term;

[0010] S3: Combine the sample dimension loss term and the feature dimension loss term to construct the total loss function, solve for it until convergence, and obtain the optimal sample weight matrix and the optimal feature basis matrix.

[0011] S4: Perform cluster analysis based on the optimal sample weight matrix to obtain the clustering results of the hydrophone sample data;

[0012] S5: Determine whether there is any leakage abnormality in the water supply network to be identified based on the clustering results.

[0013] Existing technologies can only obtain a small number of expert-annotated leakage / normal samples by labeling based on expert experience. However, traditional unsupervised dimensionality reduction methods cannot utilize expert annotation information. The water industry has formed a mature feature classification system for leakage analysis, but existing dimensionality reduction algorithms do not incorporate such industry priors into the model design. This invention integrates expert annotation information in the sample dimension and industry feature priors in the feature dimension into the two matrices of NMF, forming a dual-matrix collaborative constraint optimization mechanism. This fully utilizes expert experience and industry priors, and transforms them into quantifiable loss terms through mathematical modeling. This makes the algorithm more robust and adaptable when dealing with actual pipeline network data with few samples, high noise, and feature coupling.

[0014] Furthermore, the hydrophone sample data containing expert annotation information includes two categories: leaky hydrophone sample data and normal hydrophone sample data.

[0015] Furthermore, the sample dimension loss term in S1 is constructed in the following way: based on the leaky hydrophone sample data and the normal hydrophone sample data, a compaction constraint and a separation constraint are applied to the sample weight matrix to minimize the distance between leaky hydrophone sample data and between normal hydrophone sample data in the sample dimension, and to maximize the distance between leaky hydrophone sample data and normal hydrophone sample data in the sample dimension.

[0016] Furthermore, the feature dimension loss term in S2 is constructed in the following way: based on the core feature subset of industry feature priors, sparse loss and within-group variance loss constraints are applied to the feature basis matrix, so that each row of the feature basis matrix establishes a correspondence with a core feature subset.

[0017] Furthermore, in S3, an alternating optimization strategy is used to solve the total loss function. This alternating optimization strategy includes:

[0018] With the feature basis matrix fixed, update the sample weight matrix;

[0019] With the sample weight matrix fixed, update the feature basis matrix;

[0020] Alternate iterations until convergence or the preset number of iterations is reached to obtain the optimal sample weight matrix and the optimal feature basis matrix.

[0021] Furthermore, the core feature subset of the hydrophone sample data includes one or more of the following: the acoustic core feature subset, the hydraulic feature subset, and the statistical feature subset.

[0022] Furthermore, in S4, the number of clusters is set to two categories: normal and leaky. The mean vector of the water listening sample data containing expert annotation information is used as the initial center for clustering to obtain two cluster centers: leaky and normal.

[0023] Furthermore, S5 determines whether there is any leakage anomaly in the water supply network to be identified based on the clustering results, specifically including:

[0024] Extract the low-dimensional feature vector of the sample to be identified in the optimal sample weight matrix;

[0025] Calculate the distance from the low-dimensional feature vector to the leakage cluster center and the distance to the normal cluster center;

[0026] The presence of water leakage anomalies in the sample is determined based on the distance comparison results.

[0027] Furthermore, determining whether the sample has any leakage anomalies based on the distance comparison results specifically includes:

[0028] If the distance from the low-dimensional feature vector to the center of the leakage cluster is less than the distance to the center of the normal cluster, then the sample is determined to be a leakage anomaly.

[0029] Otherwise, the sample is considered normal.

[0030] A computer storage medium storing a computer program, which, when executed by at least one processor, implements the steps of the water supply network leakage anomaly identification method based on NMF dimensionality reduction as described above.

[0031] The beneficial effects of this invention are as follows: By employing low computational complexity NMF and fully integrating two types of core prior knowledge, based on the nonnegativity of NMF, this invention transforms industry prior feature classification and weighting rules into algorithmic constraints, giving the dimensionality reduction results an interpretable physical meaning. On the other hand, it introduces expert annotation information, integrating hydrophone sample data containing expert annotation information into the sample weight matrix to apply constraints, achieving a balance between algorithmic complexity and recognition performance. Furthermore, it utilizes a small amount of expert-annotated leak-type hydrophone sample data and normal-type hydrophone sample data to optimize the model, thereby improving the accuracy of leak anomaly identification. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the overall process of a water supply network leakage anomaly identification method based on NMF dimensionality reduction according to the present invention.

[0033] Figure 2 This is a schematic diagram illustrating the workflow of a water supply network leakage anomaly identification method based on NMF dimensionality reduction according to the present invention.

[0034] Figure 3 The accompanying drawings are for reference only. Figure 1 A diagram illustrating the content of the input data. Detailed Implementation

[0035] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the present invention or its application or use. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0036] Unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps described in these embodiments do not limit the scope of the invention.

[0037] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0038] Furthermore, for clarity and brevity, descriptions of well-known structures, functions, and configurations may have been omitted. Those skilled in the art will recognize that various changes and modifications can be made to the examples described herein without departing from the spirit and scope of this disclosure.

[0039] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0040] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0041] The present invention will now be described in detail with reference to the accompanying drawings and embodiments:

[0042] like Figures 1-3 As shown, a method for identifying leaking anomalies in water supply networks based on NMF dimensionality reduction includes: a nonnegative matrix factorization (NMF) model composed of a sample weight matrix and a feature basis matrix; hydrophone sample data containing partial expert annotation information; and hydrophone sample data containing a subset of core features with prior industry characteristics. The method includes the following steps:

[0043] S1: Embed the hydrophone sample data containing expert annotation information into the sample weight matrix as a row vector to construct the sample dimension loss term;

[0044] S2: Embed the hydrophone sample data containing the core feature subset with industry feature priors into the mapping feature basis matrix as rows to construct the feature dimension loss term;

[0045] S3: Combine the sample dimension loss term and the feature dimension loss term to construct the total loss function, solve for it until convergence, and obtain the optimal sample weight matrix and the optimal feature basis matrix.

[0046] S4: Perform cluster analysis based on the optimal sample weight matrix to obtain the clustering results of the hydrophone sample data;

[0047] S5: Determine whether there is any leakage abnormality in the water supply network to be identified based on the clustering results.

[0048] Specifically, Non-negative Matrix Factorization (NFF) is a dimensionality reduction and feature extraction method that decomposes a high-dimensional original data matrix into the product of two low-dimensional non-negative matrices under non-negativity constraints. Let the original hydrophone sample data matrix be X, which can be decomposed as: X ≈ WH; where: W is the sample weight matrix, reflecting the distribution of samples on latent features; and H is the feature basis matrix, reflecting the correlation between the original features and the latent features.

[0049] The acquisition of hydrophone sample data includes:

[0050] Based on the hydrophone database of Water Supply Plant E in Sichuan Province, combined with the hydrophone leakage detection feature classification standards in the industry (such as those of the AWWA), k core feature subsets and m features (both non - negative physical quantities, and k << m) are constructed. For example:

[0051] Acoustic core feature subset: corresponding to the leakage acoustic features recommended by AWWA (root - mean - square energy in time domain, peak factor, sample entropy; main frequency in frequency domain, spectral energy, spectral kurtosis, etc.);

[0052] Hydraulic feature subset: corresponding to the leakage hydraulic - related features recommended by AWWA (pipe network pressure fluctuation value, flow deviation, pressure peak difference);

[0053] Environmental auxiliary feature subset: corresponding to the interference features recommended by AWWA (ambient temperature, ground humidity, background noise intensity);

[0054] Construct the original hydrophone feature matrix , where n represents the total number of hydrophone samples collected.

[0055] Obtaining expert - labeled information:

[0056] Based on the expert experience of Water Supply Plant E in Sichuan Province, a small number of samples in are manually labeled, denoted as the expert - labeled sample matrix and the labeled - tag matrix , is a subset of , label represents the number of samples labeled by experts; the expert labels in this invention only provide whether the sample is a leaking sample. Therefore, is a label row 1 - column vector, indicates that the num - th labeled sample is a leaking sample, indicates that the num - th labeled sample is a normal sample.

[0057] Further divide the labeled - sample subset: the leaking - labeled sample matrix , leak is the number of leaking - labeled samples, the normal - labeled sample matrix , normal is the number of normal - labeled samples, satisfying:

[0058] ;

[0059] Data pre - processing:

[0060] To make the NMF reconstruction loss be equally considered, and the feature values still have physical meanings after standardization, that is, the values are positive, this invention performs min - max standardization on all features. The formula is:

[0061] ;

[0062] in:

[0063] Let be the value of the i-th sample and the j-th feature before standardization; , These are the minimum and maximum values ​​of the j-th feature across all samples, respectively. The standardized eigenvalues ​​take values ​​in the interval [0,1], satisfying the nonnegativity requirement of NMF.

[0064] For the original hydrophone sample matrix The above standardization process yields the standardized matrix used as the model input: , and its subsets: , , .

[0065] S1 specifically involves introducing compaction and separation constraints into the weight matrix for hydrophone sample data containing expert annotations, and constructing a loss term for the sample dimension. The formula applies compaction and separation constraints to the sample weight matrix, minimizing the distance between leaky hydrophone samples of the same class and between normal hydrophone samples in the sample dimension, while maximizing the distance between leaky hydrophone samples of different classes and between normal hydrophone samples in the sample dimension. The specific formula is as follows:

[0066] ;

[0067] in, α For the tight constraint weight hyperparameter, β To separate the constraint weight hyperparameters; For the i-th leaking labeled sample in the sample weight matrix row vectors in For the i-th normally labeled sample in Row vectors in;

[0068] for The mean vector, for The mean vector.

[0069] Specifically, in S2, based on AWWA industry priors, the feature basis matrix is... Each row is associated with a core feature class (e.g., acoustic core features / hydraulic features / environmental auxiliary features), constructing a composite feature dimension loss term that includes sparsity loss and within-group variance loss. ,make The formula is as follows: It has clear engineering and physical significance, and the correlation between key feature dimensions conforms to industry priors.

[0070] ;

[0071] Where: γ is the sparse constraint weight hyperparameter, and δ is the within-group variance constraint weight hyperparameter;

[0072] The index of the AWWA feature subset corresponding to the feature basis of row t (e.g., when t=1). (where j represents a subset of the acoustic core features, and j can represent sub-features such as "root mean square energy in the time domain" within the subset of acoustic core features).

[0073] The first term of the polynomial is a sparse loss constraint: minimizing the feature subset corresponding to the feature basis in the t-th row. The response values ​​of all external features force The second term represents the feature basis of row t in the corresponding feature subset. The variance within the feature base ensures the differentiated response of the feature base to different sub-features within the corresponding industry feature subset, thus avoiding feature base degradation.

[0074] The total loss function of S3 also includes the basic reconstruction loss, Decomposed into sample weight matrix With characteristic basis matrix :

[0075] ;

[0076] The basic loss function of NMF is constructed using Euclidean distance to measure the reconstruction error between the original hydrophone data sample matrix and the decomposition matrix. The formula is as follows:

[0077] ;

[0078] in: is the Frobenius norm of the original hydrophone data matrix.

[0079] Total loss function It can be represented as:

[0080] ;

[0081] Verification and solution of the total loss function using convex optimization:

[0082] The total loss function of NMF constructed in this invention Regarding variables and Since joint nonconvexity makes it impossible to directly obtain the global optimum, this invention employs an alternating optimization strategy: fixing one variable while performing convex optimization on the other, iterating repeatedly until convergence. The theoretical basis for this strategy is as follows:

[0083] fixed hour, It is about The quadratic function has a positive semi-definite Hessian matrix and is a convex function; Both the sparsity constraint and the within-group variance constraint are linear terms or sums of squares, and both are convex functions; and It is irrelevant and can be considered a constant. The sum of convex functions is still a convex function, therefore: It is about convex functions;

[0084] fixed hour, It is about The quadratic function has a positive semi-definite Hessian matrix and is a convex function; It consists of sample compaction constraint terms and separation constraint terms, both of which are in L2 norm squared form and are convex functions;

[0085] and Since it is irrelevant and can be considered a constant, the sum of convex functions remains a convex function. Therefore: It is about convex functions;

[0086] In summary, although the total loss function is related to... and When the two variables are both non-convex, but one variable is fixed, the other variable satisfies the convex optimization condition. Therefore, iterative convergence to a stable solution can be guaranteed by alternating convex optimization.

[0087] The steps to solve the total loss function include:

[0088] Initialization parameters:

[0089] ;

[0090] ;

[0091] ;

[0092] ;

[0093] ;

[0094] in, Indicates the number of iterations. This represents a non-negative initialization sample weight matrix. This represents the non-negative initialization of the eigenbase matrix. The learning rate represents the hyperparameter of the sample weight matrix. The learning rate represents the hyperparameter of the feature basis matrix. Indicates the convergence threshold. This indicates the maximum number of iterations.

[0095] fixed ,renew ,calculate The gradient is calculated, and non-negativity constraint updates are performed:

[0096] ;

[0097] ;

[0098] fixed ,renew ,calculate The gradient is calculated, and non-negativity constraint updates are performed:

[0099] ;

[0100] ;

[0101] Convergence determination and output results: In addition to monitoring the total loss, the update magnitude of the sample weight matrix and feature basis matrix can also be monitored. The iteration stops when both are less than a certain threshold.

[0102] Convergence criterion: ;

[0103] Indicates the first The total loss after rounds of iteration.

[0104] Output: Optimal sample weight matrix Optimal eigenbase matrix .in: , .

[0105] In S4, this embodiment uses k-means modeling, based on the optimal sample weight matrix obtained in S3. The cluster number is set to cluster=2, corresponding to the normal class and the leaking class. The mean vector of the water listening sample data containing expert annotation information is used. Using the initial centers, k-means clustering is performed, ultimately resulting in two types of cluster centers: ,

[0106] in, Indicates the cluster center of leakage type, This represents the cluster center of the normal class.

[0107] S5 specifically includes:

[0108] Distance calculation: Low-dimensional feature vector of any sample dimension to be identified. Calculate the Euclidean distance from it to the two cluster centers:

[0109] ;

[0110] ;

[0111] in The distance from the sample to be identified to the center of the leakage category. This represents the distance from the sample to the center of the normal class.

[0112] Anomaly detection: If < If the sample is closer to the cluster center of the leakage category, it is considered a leakage sample; otherwise, it is considered normal.

[0113] The method of this invention has been applied to the detection of abnormal water leakage in the pipeline network of E Water Plant in Sichuan Province. Based on the actual operation data of E Water Plant, and taking the operation data of the pure K-Means method from July 2024 to April 2025 as the benchmark, the operation effect of the method of this invention from May 2025 to the present is compared.

[0114] Evaluation indicator definition:

[0115] Leakage Precision: The percentage of samples that are actually leaking among those identified as leaking abnormalities, reflecting the false detection rate;

[0116] Leakage recall rate: The percentage of real leak samples that are correctly identified as leaking abnormalities, reflecting the false negative rate;

[0117] F1 score: The harmonic mean of precision and recall, which comprehensively measures classification accuracy;

[0118] Sum of Squares within a Cluster (SSE): The sum of the squared distances from all samples to the corresponding cluster center. The smaller the value, the more compact the samples within the cluster and the better the stability.

[0119] Comparison of the performance of the model using the method of this invention and the pure K-Means method:

[0120] Table 1. Comparison of Model Performance

[0121]

[0122] As can be seen from Table 1, the method of the present invention has a significant improvement in the accuracy of leakage anomaly identification and clustering stability compared with the pure K-Means method originally used in Water Plant E.

[0123] Furthermore, potential applications of this invention include:

[0124] Rapid screening of routine leakage in urban water supply networks: Suitable for water companies to conduct periodic leakage checks on the entire network of the main urban area and new urban area. It is adapted to the characteristics of wide network coverage, non-negative hydrological characteristics, scarce leakage labeling samples, and lack of algorithm expertise among operation and maintenance personnel.

[0125] Accurate identification of leakage in high-noise pipe networks in old urban areas: Applicable to scenarios with high environmental noise, aging pipe networks and materials, and extremely scarce leakage labeling data, such as old urban areas and core business districts.

[0126] Assistance for locating leaks in complex pipeline networks in industrial parks: Applicable to pipeline network scenarios with complex hydraulic conditions and special vibration interference, such as industrial parks. It is necessary to identify leakage anomalies and provide engineering guidance for locating leak points.

[0127] Portable hydrophone testing for water supply networks in villages and remote areas: Suitable for remote areas such as villages and towns with limited computing resources, insufficient technical maintenance personnel, and a lack of leak labeling samples. It mainly uses portable devices such as handheld hydrophones for testing.

[0128] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Based on the technical essence of the present invention, any simple modifications, equivalent substitutions, and improvements made to the above embodiments within the spirit and principles of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for identifying water supply network leakage anomalies based on NMF dimensionality reduction, comprising: A nonnegative matrix factorization (NMF) model composed of a sample weight matrix and a feature basis matrix, using hydrophone sample data containing partial expert annotation information and hydrophone sample data containing a subset of core features with prior industry characteristics, is characterized by the following steps: S1: Embed the hydrophone sample data containing expert annotation information into the sample weight matrix as row vectors. By applying compaction constraints and separation constraints to the sample weight matrix, minimize the distance between leaky hydrophone sample data and between normal hydrophone sample data in the sample dimension, maximize the distance between leaky hydrophone sample data and between normal hydrophone sample data in the sample dimension, and construct the sample dimension loss term. S2: Embed the hydrophone sample data containing the core feature subset with industry feature priors into the mapping feature basis matrix as rows. By applying sparse loss and within-group variance loss constraints to the feature basis matrix, establish a correspondence between each row of the feature basis matrix and a core feature subset, and construct the feature dimension loss term. S3: Combine the sample dimension loss term and the feature dimension loss term to construct the total loss function, solve for it until convergence, and obtain the optimal sample weight matrix and the optimal feature basis matrix. S4: Perform cluster analysis based on the optimal sample weight matrix to obtain the clustering results of the hydrophone sample data; S5: Determine whether there is any leakage abnormality in the water supply network to be identified based on the clustering results.

2. The method for identifying water supply network leakage anomalies based on NMF dimensionality reduction according to claim 1, characterized in that, The hydrophone sample data containing expert annotation information includes two categories: leak-prone hydrophone sample data and normal hydrophone sample data.

3. The method for identifying water supply network leakage anomalies based on NMF dimensionality reduction according to claim 1, characterized in that, In S3, an alternating optimization strategy is used to solve the total loss function. The alternating optimization strategy includes: With the feature basis matrix fixed, update the sample weight matrix; With the sample weight matrix fixed, update the feature basis matrix; Alternate iterations until convergence or the preset number of iterations is reached to obtain the optimal sample weight matrix and the optimal feature basis matrix.

4. The method for identifying water supply network leakage anomalies based on NMF dimensionality reduction according to claim 1, characterized in that, The core feature subset of hydrophone sample data includes one or more of the following: acoustic core feature subset, hydraulic feature subset, and statistical feature subset.

5. The method for identifying water supply network leakage anomalies based on NMF dimensionality reduction according to claim 1, characterized in that, In S4, the number of clusters is set to two categories: normal and leaking. The mean vector of the water listening sample data containing expert annotation information is used as the initial center for clustering to obtain two cluster centers: leaking and normal.

6. The method for identifying water supply network leakage anomalies based on NMF dimensionality reduction according to claim 1, characterized in that, S5 determines whether there is any leakage anomaly in the water supply network to be identified based on the clustering results, specifically including: Extract the low-dimensional feature vector of the sample to be identified in the optimal sample weight matrix; Calculate the distance from the low-dimensional feature vector to the leakage cluster center and the distance to the normal cluster center; The presence of water leakage anomalies in the sample is determined based on the distance comparison results.

7. The method for identifying water supply network leakage anomalies based on NMF dimensionality reduction according to claim 6, characterized in that, The determination of whether the sample has any leakage anomalies based on the distance comparison results specifically includes: If the distance from the low-dimensional feature vector to the center of the leakage cluster is less than the distance to the center of the normal cluster, then the sample is determined to be a leakage anomaly. Otherwise, the sample is considered normal.

Images