Water conservancy project leakage abnormity automatic identification method and system based on unsupervised learning

Through the unsupervised learning method, improved clustering and outlier detection algorithms are used to automatically identify leakage anomalies in water conservancy projects, solving the problems of artificial experience dependence and deep neural network complexity, and achieving efficient and accurate leakage anomalies recognition.

CN119066587APending Publication Date: 2024-12-03CHINA WATER RESOURCES BEIFANG INVESTIGATION DESIGN & RES CO LTD

Patent Information

Application Number
CN202411231377.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-04
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing leak abnormality recognition methods rely on manual experience, resulting in inconsistent standards, low efficiency, inconsistent results, and deep neural network methods have data dependence, high computing resources, and complex parameter adjustment, which affects the stability of actual application.

Method used

Using an unsupervised learning-based method, through improved K-means clustering algorithm, local outlier factor detection method and DBSCAN algorithm, automatically extract and analyte detection data, identify leakage abnormal areas, reduce manual intervention, improve efficiency and accuracy sex.

Benefits of technology

It realizes automatic identification of leakage abnormalities, reduces manual errors, improves identification efficiency and accuracy, reduces computing resource requirements, and is suitable for different water conservancy engineering applications.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066587A_ABST
    Figure CN119066587A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of engineering geophysical exploration leakage detection, and discloses a water conservancy project leakage abnormity automatic identification method and system based on unsupervised learning, and the method comprises the steps: obtaining detection data, carrying out the preprocessing of the detection data, carrying out the fitting of the detection data through an improved K-means clustering algorithm, obtaining the feature clustering point data of an abnormal region, and carrying out the recognition of the leakage abnormity of a water conservancy project. The method comprises the following steps of: acquiring different leakage abnormal clustering point clusters, correcting the leakage abnormal clustering point clusters, classifying by using an improved DBSCAN algorithm to obtain different leakage abnormal clustering point clusters, communicating the clustering point clusters belonging to the same classification together to form an abnormal clustering point region, aggregating clustering points in the abnormal clustering point region into continuous abnormal region surfaces by using a two-dimensional convex hull algorithm, and then taking each abnormal region surface as a clustering point, according to the method, a three-dimensional abnormal area is formed through classification by using an improved DBSCAN algorithm, and then abnormal area surfaces are aggregated into a leakage channel in a three-dimensional space by using a three-dimensional convex hull algorithm, so that the efficiency can be improved and the deviation caused by artificial experience can be reduced under the condition of ensuring the precision.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of engineering geophysical exploration leakage detection, and in particular to a method and system for automatically identifying leakage anomalies in water conservancy projects based on unsupervised learning. Background Art

[0002] Leakage detection is a key link in safety detection during the construction and operation and maintenance of water conservancy projects. Geophysical methods are the most commonly used and effective non-destructive detection methods. Geophysical methods provide non-invasive and quantitative information to help judge and delineate leakage anomalies by measuring and analyzing the difference characteristics of underground dielectric electrical parameters, elastic parameters, dielectric constants, etc. The delineation of potential leakage disease areas or leakage risk areas in the results of these geophysical methods based on the differences in physical parameters is called leakage anomaly identification.

[0003] The current leakage anomaly identification mainly relies on the experience and judgment of the interpreters. The leakage anomaly is identified by analyzing the data produced by different geophysical exploration methods, obtaining the leakage anomaly of a single one-dimensional curve or a single two-dimensional data profile, and then relying on human judgment to associate the spatial positions of multiple leakage anomalies, and then draw the leakage anomaly channel. Although this method is widely used in practice, it has the following problems and challenges: High reliance on experience: The work experience of the interpreter is crucial to judging and identifying leakage anomalies, which may lead to inconsistent standards and lack of consistency in results.

[0004] Time-consuming and energy-consuming: Each leakage anomaly identification process requires a lot of time and energy, especially when dealing with complex geological conditions.

[0005] Low processing efficiency by novices: Inexperienced operators may cause subjective errors and operational mistakes, thus affecting the accuracy of recognition results.

[0006] Software compatibility and collaboration difficulties: Data interpretation from different geophysical exploration methods usually requires the use of different software tools, and collaboration and data integration across devices may present certain technical difficulties.

[0007] Currently, interpretation methods based on deep neural networks are constantly developing, but they rely on a large amount of pre-training data and long-term network optimization, as well as the requirement for high-configuration computer resources. In addition, the black-box nature of deep neural networks and the challenges of parameter adjustment make them unstable and complex in practical applications.

[0008] Therefore, there is an urgent need for an automatic identification method and system for leakage anomalies in water conservancy projects based on unsupervised learning, which can reduce subjective errors and operational errors that may be caused by insufficient human experience, and innovatively apply it to leakage detection in water conservancy projects, providing new ideas and methods for explaining and interpreting leakage detection results. Summary of the invention

[0009] In order to solve the above technical problems, the present invention provides a method and system for automatically identifying leakage anomalies in water conservancy projects based on unsupervised learning, which can improve efficiency and reduce deviations caused by differences in manual experience while ensuring accuracy.

[0010] The present invention provides a method and system for automatically identifying leakage anomalies in water conservancy projects based on unsupervised learning, comprising the following steps: S1. Obtaining test data; S2, preprocessing the detection data to form a detection data set; S3, fitting the detection data set through the improved K-means clustering algorithm to obtain the characteristic clustering point data of the abnormal area; S4, correcting the abnormal area feature cluster point data obtained in step S3 by using a local outlier factor detection method; S5. Using the improved DBSCAN algorithm, the corrected abnormal area characteristic cluster point data is classified to obtain different leakage abnormal cluster point clusters; S6, connecting the different leakage abnormal clusters classified in step S5 and the clusters of the same classification together to form an abnormal cluster area; using a two-dimensional convex hull algorithm, the clusters in the abnormal cluster area are aggregated into a continuous abnormal area surface; S7, taking each abnormal region surface as a cluster point, and using the improved DBSCAN algorithm in step S5 to classify according to the spatial characteristics of the abnormal region surface to form a three-dimensional abnormal region, and then using a three-dimensional convex hull algorithm to aggregate the abnormal region surface into a leakage channel in the three-dimensional space; S8. The abnormal area surface and the leakage channel in the three-dimensional space are used as the leakage identification area.

[0011] Furthermore, the preprocessing of the detection data specifically includes: S21, performing outlier elimination and denoising processing on the detection data; S22, normalizing the detection data after outlier removal and denoising; S23, performing low-value filtering on the normalized detection data.

[0012] Furthermore, the step S3 specifically includes: S31, randomly set R data points as cluster centers; S32, based on the detection data, determining the weight of each data point in the detection data set; S33, respectively calculate the Euclidean distance between each data point in the detection data set and each cluster center, and assign each data point to the cluster center with the closest Euclidean distance. The calculation formula is as follows: ; in, x is a data point, c i It is i Cluster centers, d is the total number of dimensions of the data, x j and c ij They are x and c i In the j The value of the dimension; Indicates the distance between two points; S34. For each cluster, calculate the weighted mean of all data points within it and use the weighted mean as the new cluster center. c i , the calculation formula is as follows: ; in, S i It is i A set of data points that are clustered, m yes S i The number of data points in It is i In the cluster n The weight of the data point; x in It is i The first cluster center n data points; S35, repeat step S33 and step S34 until the objective function is less than a preset threshold. SSE The calculation formula is as follows: .

[0013] Furthermore, the S32 specifically includes: Based on the detection data, determining an abnormal energy value of each data point in the detection data set; According to the abnormal energy value of each data point in the detection data set, the weight of each data point in the detection data set is calculated. The calculation formula is as follows: ; in, It is i Clustern The abnormal energy value of the data point.

[0014] Furthermore, the improved DBSCAN algorithm in S5 specifically includes: S51, setting initial parameters, specifically including: : Neighborhood radius, used to determine the neighborhood radius of a point; MinPts: minimum number of neighborhood points, used to define the minimum threshold of core points; in, ; In the formula, is the preset value, indicating the default neighborhood radius; is the weighted energy value of the characteristic clustering point data in the abnormal area; Wherein, based on the abnormal region characteristic cluster point data, the weighted energy value of the abnormal region characteristic cluster point data is determined; S52, selecting an unvisited point p from all data points and marking it as visited; S53, calculate point p The number of points in the neighborhood; S54. If the point p If the number of points in the neighborhood is greater than or equal to MinPts, point p is marked as a core point, and a new cluster is created with point p as the cluster center, and point p is added to the cluster; S55, for point p All points within the neighborhood do the following: If the point has not been visited, it is marked as visited, and the process returns to step S53 and is repeated; If the point is a core point, All points in the neighborhood are added to the current cluster; If the point is not assigned to any cluster, it is assigned to the current cluster; S56. Repeat S52 to S54 until all data points are marked as visited.

[0015] Furthermore, in S4, the local outlier factor detection method specifically includes: S41, determine the number of initial gathering points K ,definition k The reachable distance is ; ; in, The first k distance, is the distance between cluster point A and cluster point B; S42, calculate the first kThe local reachable density is calculated as follows: ; in, is the first k Locally accessible density, For the gathering point A k The set of nearest neighbors, is the reachable distance from cluster point A to cluster point B; S43, calculate each cluster point k The local outlier factor is calculated as follows: ; in The gathering point A is k Local outlier factor; if , then point A is an outlier; if , then point A is not an outlier.

[0016] The present invention also provides a water conservancy project leakage anomaly automatic identification system based on unsupervised learning, which specifically includes: A data acquisition module, used for acquiring detection data; A data preprocessing module, connected to the data acquisition module, for preprocessing the detection data to form a detection data set; A K-means algorithm module, connected to the data preprocessing module, is used to fit the detection data set through an improved K-means clustering algorithm to obtain abnormal area feature clustering point data; A local outlier factor detection module, connected to the K-means algorithm module, is used to correct the abnormal area feature cluster point data obtained in step S3 by using a local outlier factor detection method; A DBSCAN algorithm module, connected to the local outlier factor detection module, is used to classify the corrected abnormal area feature clustering point data through an improved DBSCAN algorithm to obtain different leakage abnormality clustering point clusters; The abnormal area surface generation module is connected to the DBSCAN algorithm module and is used to connect the different leakage abnormal clusters after classification and the clusters belonging to the same classification to form an abnormal cluster area; the abnormal cluster area is aggregating the clusters in the abnormal cluster area into a continuous abnormal area surface using a two-dimensional convex hull algorithm; A leakage channel generation module is connected to the abnormal region surface generation module, and is used to take each abnormal region surface as a cluster point, and classify it according to the spatial characteristics of the abnormal region surface by using the improved DBSCAN algorithm in step S5 to form a three-dimensional abnormal region, and then use the three-dimensional convex hull algorithm to aggregate the abnormal region surface into a leakage channel in the three-dimensional space; The leakage identification area output module is connected to the abnormal area surface generation module and the leakage channel generation module, and is used to use the abnormal area surface and the leakage channel in the three-dimensional space as the leakage identification area.

[0017] The embodiments of the present invention have the following technical effects: 1. Automatically extract and analyze geophysical detection data through improved unsupervised learning algorithms to reduce errors caused by inconsistent manual standards.

[0018] 2. Automatically identify and fit abnormal data through the improved kmeans algorithm, and generate abnormal area cluster data representing different leakage types through the improved DBSCAN algorithm to realize the classification of different types of leakage. Based on the two-dimensional convex hull algorithm, the abnormal area clusters are aggregated into continuous abnormal area surfaces. Based on the three-dimensional convex hull algorithm, the abnormal area surfaces are associated to form leakage channels in three-dimensional space; avoiding the subjectivity and errors of manual judgment in traditional methods.

[0019] 3. Compared with the deep neural network method, the present invention does not require a large amount of label data and high-performance computing resources, nor does it require a long training process and complex parameter adjustment. This makes it easy to operate, fast in calculation speed, high in efficiency, and can maintain good recognition accuracy and stability.

[0020] 4. The present invention is easy to operate: the whole process adopts unsupervised learning, which reduces manual intervention and complex parameter settings; high efficiency: compared with traditional methods, the efficiency of leakage anomaly interpretation is greatly improved.

[0021] 5. Accuracy and stability: Data is processed through machine learning algorithms, which effectively avoids errors caused by human judgment and reduces the deviation of interpretation results; Application expansion capability: The software processing flow adopts modular design, has good optimization space and adaptability, and is suitable for the application needs of different water conservancy projects. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 It is a flow chart of a method for automatically identifying leakage anomalies in water conservancy projects based on unsupervised learning provided by an embodiment of the present invention; Figure 2 The detection data to be used for leakage anomaly identification is obtained through geophysical exploration technology provided by an embodiment of the present invention; Figure 3 It is a flow field method data diagram for leakage anomaly identification provided by an embodiment of the present invention; Figure 4 is a data distribution diagram of abnormal region feature clustering points obtained by an improved k-means algorithm provided by an embodiment of the present invention; Figure 5 It is an abnormal area surface formed by different leakage abnormality clusters obtained by aggregating the two-dimensional convex hull algorithm and improving the DBSCAN algorithm classification provided by the embodiment of the present invention; Figure 6 is a three-dimensional schematic diagram of a three-dimensional space leakage channel provided by an embodiment of the present invention; Figure 7 This is a result diagram of leakage anomaly identification provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0025] Embodiment 1 Figure 1 Flow chart of a method for automatically identifying leakage anomalies in water conservancy projects based on unsupervised learning provided by an embodiment of the present invention. Figure 1 , specifically including: S1. Obtaining test data; The detection data is obtained by geophysical exploration technology. The water conservancy engineering building used in this embodiment is a reservoir, and the geophysical exploration detection methods used are pseudo-random flow field method and transient electromagnetic method. The pseudo-random flow field method survey line SK0 is arranged in the reservoir water area, and the data is a three-dimensional surface map with elevation fluctuations. Multiple survey lines DFSK1-DFSK5 of the transient electromagnetic method are arranged on the dam surface of the reservoir dam, and the data is a two-dimensional profile map of vertical detection.

[0026] Figure 2 It is the detection data for leakage anomaly identification obtained by geophysical exploration technology provided by the embodiment of the present invention, and its position in the reservoir water area can be seen.

[0027] S2, preprocessing the detection data to form a detection data set; Furthermore, the preprocessing of the detection data specifically includes: S21, performing outlier elimination and denoising processing on the detection data; S22, normalizing the detection data after outlier removal and denoising; S23, performing low-value filtering on the normalized detection data.

[0028] Specifically, the detection data is imported into a special data processing software, and preprocessing parameters are set. The settings of preprocessing parameters usually include filter type, cutoff frequency, outlier threshold, etc., and data preprocessing is performed, including outlier removal, denoising and other operations. The detection data after outlier removal and denoising is normalized and preprocessed as a whole, and all data are mapped to [0-1] using the maximum and minimum values ​​of the data to improve the subsequent calculation rate. Low-value filtering is performed on the normalized data, which is helpful for the later calculation of the data. Usually, the 60% percentile is taken, and the 40% strongest area is retained as the data anomaly body.

[0029] S3, fitting the detection data set through the improved K-means clustering algorithm to obtain the characteristic clustering point data of the abnormal area; Furthermore, the step S3 specifically includes: S31, randomly set R data points as cluster centers; S32, based on the detection data, determining the weight of each data point in the detection data set, specifically including: Based on the detection data, the abnormal energy value of each data point in the detection data set is determined; abnormal data has a larger energy value, thereby obtaining more clustering points, that is, during the algorithm iteration process, the leakage abnormal area has a higher energy weight value, and a stronger fitting effect is obtained during the iteration process.

[0030] According to the abnormal energy value of each data point in the detection data set, the weight of each data point in the detection data set is calculated. The calculation formula is as follows: ; in, is the abnormal energy value of the nth data point in the ith cluster.

[0031] S33, respectively calculate the Euclidean distance between each data point in the detection data set and each cluster center, and assign each data point to the cluster center with the closest Euclidean distance. The calculation formula of the Euclidean distance is as follows: ; in, x is a data point,c i It is i Cluster centers, d is the total number of dimensions of the data, x j and c ij They are x and c i In the j The value of the dimension; Indicates the distance between two points; S34. For each cluster, calculate the weighted mean of all data points within it and use the weighted mean as the new cluster center. c i , the calculation formula is as follows: ; in, S i It is i A set of data points that are clustered, m yes S i The number of data points in It is i In the cluster n The weight of the data point; x in It is i The first cluster center n data points; S35, repeat step S33 and step S34 until the objective function is less than a preset threshold. SSE The calculation formula is as follows: .

[0032] It is worth further explaining that the objective function is less than the preset threshold, that is, the distance between the new cluster center and the old cluster center is less than the preset threshold, and the cluster points converge to the characteristic parts of the leakage anomaly, that is, the cluster points can characterize the distribution of the fitted leakage anomaly.

[0033] The K-means clustering algorithm usually realizes classification, but cannot achieve the fitting effect of abnormal data. This embodiment improves the K-means clustering algorithm by weighting data, so that the clustering algorithm is improved into a fitting algorithm, and the abnormal area in the fitting data sample is realized, so that the distribution characteristics of the abnormal area in the processing data can be effectively extracted, even if the fitting points can effectively cover the abnormal area of ​​the data profile, and the leakage abnormal area is represented by a number of abnormal area feature cluster points.

[0034] S4, correcting the abnormal area feature cluster point data obtained in step S3 by using a local outlier factor detection method, specifically including: S41, determine the number of initial gathering points K ,definition k The reachable distance is ; ; in, The first k distance, is the distance between cluster point A and cluster point B; S42, calculate the first k The local reachability density is calculated as follows: ; in, is the first k Locally accessible density, For the gathering point A k The set of nearest neighbors, is the reachable distance from cluster point A to cluster point B; S43, calculate each cluster point k The local outlier factor is calculated as follows: ; in The gathering point A is k Local outlier factor; if , then point A is an outlier; if , then point A is not an outlier.

[0035] It is worth noting that when the local outlier factor is greater than 1, it means that the density of point A is low, the overall distribution deviates from the leakage anomaly area, and it belongs to the background energy noise cluster point. When its ratio is 1, the density between cluster point A and neighbor B is similar and has the same characteristics; when the ratio is less than 1, it means that the density of cluster point A is higher and it is more likely to be located at the leakage anomaly position; this method can be used to correct the characteristic cluster point data of the abnormal area and remove the data points that deviate from the leakage anomaly area.

[0036] S5. Using the improved DBSCAN algorithm, the corrected abnormal area characteristic cluster point data is classified to obtain different leakage abnormal cluster point clusters; Furthermore, the improved DBSCAN algorithm in S5 specifically includes: S51, setting initial parameters, specifically including: : Neighborhood radius, used to determine the neighborhood radius of a point; MinPts: minimum number of neighborhood points, used to define the minimum threshold of core points; the specific value can be set according to expert experience in a specific embodiment.

[0037] in, ; In the formula, H is the preset value, indicating the default neighborhood radius; It is the weighted energy value of the characteristic clustering point data in the abnormal area.

[0038] Furthermore, based on the characteristic clustering point data of the abnormal area, the weighted energy value of the characteristic clustering point data of the abnormal area is determined, indicating the radius of the clustering point neighborhood division Affected by the leakage anomaly, the greater the anomaly intensity at the gathering point, the larger the coverage radius.

[0039] S52, selecting an unvisited point p from all data points and marking it as visited; S53, calculate point p The number of points in the neighborhood; S54. If the point p If the number of points in the neighborhood is greater than or equal to MinPts, point p is marked as a core point, and a new cluster is created with point p as the cluster center, and point p is added to the cluster; S55, for point p All points within the neighborhood do the following: If the point has not been visited, it is marked as visited, and the process returns to step S53 and is repeated; If the point is a core point, All points in the neighborhood are added to the current cluster; If the point is not assigned to any cluster, it is assigned to the current cluster; S56. Repeat S52 to S54 until all data points are marked as visited.

[0040] The improved DBSCAN algorithm is used to divide the two-dimensional leakage anomaly area. In the traditional DBSCAN algorithm, the neighborhood radius is fixed and is not suitable for data with different density distributions. The improved DBSCAN algorithm considers dynamically adjusting the neighborhood radius according to the weighted energy value of the cluster point data, so that clusters and outliers can be more accurately divided in areas with different density changes to better reflect the relative density or similarity between data points. This improved distance calculation method can be selected according to specific application scenarios and data characteristics, so that the calculation results of the division of abnormal areas are clearer.

[0041] S6. Connect the different leakage anomaly clusters classified in step S5 and the clusters of the same classification together to form an abnormal cluster area; use a two-dimensional convex hull algorithm to aggregate the clusters in the abnormal cluster area into a continuous abnormal area surface.

[0042] S7, taking each abnormal region surface as a cluster point, and using the improved DBSCAN algorithm in step S5 to classify according to the spatial characteristics of the abnormal region surface to form a three-dimensional abnormal region, and then using a three-dimensional convex hull algorithm to aggregate the abnormal region surface into a leakage channel in the three-dimensional space; Furthermore, the spatial characteristics of the abnormal region surface include type, location and other abnormal characteristics; this step aims to group the abnormal region surfaces into different classification clusters.

[0043] S8, taking the abnormal area surface and the leakage channel in the three-dimensional space as the leakage identification area; The role of the convex hull algorithm here is to connect multiple two-dimensional abnormal area surfaces of different categories to form a continuous three-dimensional structure, so as to better understand and analyze the spatial distribution and shape of the leakage channel.

[0044] Specifically, taking an actual work area as an example, the method provided by the present invention is used to perform leakage anomaly detection.

[0045] Figure 3 It is a flow field method data diagram for leakage anomaly identification provided by an embodiment of the present invention; Figure 4 It is a data distribution diagram of characteristic clustering points of abnormal regions obtained by an improved k-means algorithm provided by an embodiment of the present invention, that is, a fitted leakage abnormal region; Figure 5 It is an abnormal area surface formed by different leakage abnormality clusters obtained by the two-dimensional convex hull algorithm aggregation and improved DBSCAN algorithm classification provided by the embodiment of the present invention. There are three leakage abnormal areas, which are numbered 1, 2, and 3 respectively. Figure 2~Figure 4 The figure shows the two-dimensional data leakage anomaly identification process, and the statistical table of the identified leakage anomaly area data is shown in Table 1. The statistical table includes the location, area, and anomaly range of the leakage area.

[0046] Table 1 is a statistical table of abnormal area data leakage in two dimensions

[0047] Leakage detection is performed on multiple two-dimensional leakage detection data to be identified in three-dimensional space to generate leakage anomaly area surfaces. For the leakage anomaly area surfaces of multiple two-dimensional sections in three-dimensional space, the improved DBSCAN algorithm classification and three-dimensional convex hull algorithm are used again to connect adjacent and connected leakage anomaly areas in space and spread them into three-dimensional leakage channels. In this example, there are three leakage channels in the reservoir, which are numbered 1, 2, and 3 respectively and correspond to the three leakage anomaly areas identified by the flow field method. The statistical data table of leakage anomaly areas of detection data is shown in Table 2 (taking channel 1 as an example), and the statistical data table of leakage channels is shown in Table 3 (taking channel 1 as an example). The three-dimensional space schematic result of leakage channels after associating discrete leakage anomaly area surfaces in three-dimensional space is shown in Figure 2. Figure 6 shown. Figure 7 The result diagram of leakage anomaly identification provided in the embodiment of the present invention, i.e., the comprehensive interpretation result diagram of leakage interpretation, displays the reservoir model, survey line layout, detection data, leakage anomaly area surface and leakage channel, so as to facilitate the establishment of spatial connection between reservoir, data and leakage.

[0048] Table 2 Statistical table of abnormal area data leakage in three-dimensional data

[0049] Table 3 Data statistics of leakage channels

[0050] It can be seen from the examples that the automatic leakage anomaly identification method proposed in the present invention can effectively fit the abnormal boundary, identify the leakage area, associate the leakage channel, reflect the leakage characteristics, and count the leakage related information, so as to facilitate the subsequent implementation of targeted leakage plugging treatment measures.

[0051] Embodiment 2 The present invention also provides a water conservancy project leakage anomaly automatic identification system based on unsupervised learning, which specifically includes: A data acquisition module, used for acquiring detection data; A data preprocessing module, connected to the data acquisition module, for preprocessing the detection data to form a detection data set; A K-means algorithm module, connected to the data preprocessing module, is used to fit the detection data set through an improved K-means clustering algorithm to obtain abnormal area feature clustering point data; A local outlier factor detection module, connected to the K-means algorithm module, is used to correct the abnormal area feature cluster point data obtained in step S3 by using a local outlier factor detection method; A DBSCAN algorithm module, connected to the local outlier factor detection module, is used to classify the corrected abnormal area feature clustering point data through an improved DBSCAN algorithm to obtain different leakage abnormality clustering point clusters; The abnormal area surface generation module is connected to the DBSCAN algorithm module and is used to connect the different leakage abnormal clusters after classification and the clusters belonging to the same classification to form an abnormal cluster area; the abnormal cluster area is aggregating the clusters in the abnormal cluster area into a continuous abnormal area surface using a two-dimensional convex hull algorithm; A leakage channel generation module is connected to the abnormal region surface generation module, and is used to take each abnormal region surface as a cluster point, and classify it according to the spatial characteristics of the abnormal region surface by using the improved DBSCAN algorithm in step S5 to form a three-dimensional abnormal region, and then use the three-dimensional convex hull algorithm to aggregate the abnormal region surface into a leakage channel in the three-dimensional space; The leakage identification area output module is connected to the abnormal area surface generation module and the leakage channel generation module, and is used to use the abnormal area surface and the leakage channel in the three-dimensional space as the leakage identification area.

[0052] It should be noted that the terms used in the present invention are only for describing specific embodiments, rather than limiting the scope of the present application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of more restrictions, the elements defined by the sentence "include one..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0053] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. The method for automatically identifying leakage anomalies in water conservancy projects based on unsupervised learning is characterized by: The steps include: S1. Obtaining test data; S2, preprocessing the detection data to form a detection data set; S3, fitting the detection data set through the improved K-means clustering algorithm to obtain the characteristic clustering point data of the abnormal area; S4, correcting the abnormal area feature cluster point data obtained in step S3 by using a local outlier factor detection method; S5. Using the improved DBSCAN algorithm, the corrected abnormal area characteristic cluster point data is classified to obtain different leakage abnormal cluster point clusters; S6, connecting the different leakage abnormal clusters classified in step S5 and the clusters of the same classification together to form an abnormal cluster area; for the abnormal cluster area, using a two-dimensional convex hull algorithm, aggregating the clusters in the abnormal cluster area into a continuous abnormal area surface; S7, taking each abnormal region surface as a cluster point, and using the improved DBSCAN algorithm in step S5 to classify according to the spatial characteristics of the abnormal region surface to form a three-dimensional abnormal region, and then using a three-dimensional convex hull algorithm to aggregate the abnormal region surface into a leakage channel in the three-dimensional space; S8. The abnormal area surface and the leakage channel in the three-dimensional space are used as the leakage identification area.

2. The method for automatically identifying leakage anomalies in water conservancy projects based on unsupervised learning according to claim 1 is characterized in that: The preprocessing of the detection data specifically includes: S21, performing outlier elimination and denoising processing on the detection data; S22, normalizing the detection data after outlier removal and denoising; S23, performing low-value filtering on the normalized detection data.

3. The method for automatically identifying leakage anomalies in water conservancy projects based on unsupervised learning according to claim 1 is characterized in that: The S3 specifically includes: S31, randomly set R data points as cluster centers; S32, based on the detection data, determining the weight of each data point in the detection data set; S33, respectively calculate the Euclidean distance between each data point in the detection data set and each cluster center, and assign each data point to the cluster center with the closest Euclidean distance. The calculation formula is as follows: ; in, x is a data point, c i It is i Cluster centers, d is the total number of dimensions of the data, x j and c ij They are x and c i In the j The value of the dimension; Indicates the distance between two points; S34. For each cluster, calculate the weighted mean of all data points within it and use the weighted mean as the new cluster center. c i , the calculation formula is as follows: ; in, S i It is i A set of data points that are clustered, m yes S i The number of data points in It is i In the cluster n The weight of the data point; x in It is i The first cluster center n data points; S35, repeat step S33 and step S34 until the objective function is less than a preset threshold. SSE The calculation formula is as follows: 。 4. The method for automatically identifying leakage anomalies in water conservancy projects based on unsupervised learning according to claim 3 is characterized in that: The S32 specifically includes: Based on the detection data, determining an abnormal energy value of each data point in the detection data set; According to the abnormal energy value of each data point in the detection data set, the weight of each data point in the detection data set is calculated. The calculation formula is as follows: ; in, It is i Cluster n The abnormal energy value of the data point.

5. The method for automatically identifying leakage anomalies in water conservancy projects based on unsupervised learning according to claim 1 is characterized in that: The improved DBSCAN algorithm in S5 specifically includes: S51, setting initial parameters, specifically including: : Neighborhood radius, used to determine the neighborhood radius of a point; MinPts: minimum number of neighborhood points, used to define the minimum threshold of core points; in, ; In the formula, is the preset value, indicating the default neighborhood radius; is the weighted energy value of the characteristic clustering point data in the abnormal area; Wherein, based on the abnormal region characteristic cluster point data, the weighted energy value of the abnormal region characteristic cluster point data is determined; S52, selecting an unvisited point p from all data points and marking it as visited; S53, calculate point p The number of points in the neighborhood; S54. If the point p If the number of points in the neighborhood is greater than or equal to MinPts, point p is marked as a core point, and a new cluster is created with point p as the cluster center, and point p is added to the cluster; S55, for point p All points within the neighborhood do the following: If the point has not been visited, it is marked as visited, and the process returns to step S53 and is repeated; If the point is a core point, All points in the neighborhood are added to the current cluster; If the point is not assigned to any cluster, it is assigned to the current cluster; S56. Repeat S52 to S54 until all data points are marked as visited.

6. The method for automatically identifying leakage anomalies in water conservancy projects based on unsupervised learning according to claim 5 is characterized in that: In S4, the local outlier factor detection method specifically includes: S41, determine the number of initial gathering points K ,definition k The reachable distance is ; ; in, The first k distance, is the distance between cluster point A and cluster point B; S42, calculate the first k The local reachability density is calculated as follows: ; in, is the first k Locally accessible density, For the gathering point A k The set of nearest neighbors, is the reachable distance from cluster point A to cluster point B; S43, calculate each cluster point k The local outlier factor is calculated as follows: ; in The gathering point A is k Local outlier factor; if , then point A is an outlier; if , then point A is not an outlier.

7. The automatic recognition system of leakage anomalies in water conservancy projects based on unsupervised learning is characterized by: Specifically include: A data acquisition module, used for acquiring detection data; A data preprocessing module, connected to the data acquisition module, for preprocessing the detection data to form a detection data set; A K-means algorithm module, connected to the data preprocessing module, is used to fit the detection data set through an improved K-means clustering algorithm to obtain abnormal area feature clustering point data; A local outlier factor detection module, connected to the K-means algorithm module, is used to correct the abnormal area feature cluster point data obtained in step S3 by using a local outlier factor detection method; A DBSCAN algorithm module, connected to the local outlier factor detection module, is used to classify the corrected abnormal area feature clustering point data through an improved DBSCAN algorithm to obtain different leakage abnormality clustering point clusters; The abnormal area surface generation module is connected to the DBSCAN algorithm module and is used to connect the different leakage abnormal clusters after classification and the clusters belonging to the same classification to form an abnormal cluster area; the abnormal cluster area is aggregating the clusters in the abnormal cluster area into a continuous abnormal area surface using a two-dimensional convex hull algorithm; A leakage channel generation module is connected to the abnormal region surface generation module, and is used to take each abnormal region surface as a cluster point, and classify it according to the spatial characteristics of the abnormal region surface by using the improved DBSCAN algorithm in step S5 to form a three-dimensional abnormal region, and then use the three-dimensional convex hull algorithm to aggregate the abnormal region surface into a leakage channel in the three-dimensional space; The leakage identification area output module is connected to the abnormal area surface generation module and the leakage channel generation module, and is used to use the abnormal area surface and the leakage channel in the three-dimensional space as the leakage identification area.

Citation Information

Patent Citations

  • Dam safety monitoring data anomaly detection method based on LV-DBSCAN

    CN116910672A

  • Dam safety monitoring data abnormal value identification method

    CN117609907A

Cited By

  • Road abnormal event detection and early warning method and system based on unsupervised lifelong learning

    CN119889015A

  • Method and System for Detecting and Warning Road Abnormal Events Based on Unsupervised Lifelong Learning

    CN119889015B

  • Automatic refrigerator transportation identification system

    CN120430707A