Intelligent monitoring method for water supply network leakage based on drone vision and voiceprint analysis

Abstract

The present invention discloses an intelligent monitoring method for water supply network leakage based on drone vision and voiceprint analysis, which belongs to the field of intelligent monitoring technology, including obtaining characteristic data and attribute data of a water supply pipeline to be inspected, and connecting the characteristic data and attribute data to a geographic information system integrated with a water supply pipeline map; performing optimal path planning according to the characteristic data and attribute data; enabling the drone to inspect according to the optimal path, so that the drone collects pipeline laying pavement video data and audio signals of the water supply pipeline to be inspected during the inspection process; processing and analyzing the pipeline laying pavement video data and audio signals based on a multi-scale hybrid attention network and a pipeline acoustic detection network, and obtaining the location information of the water supply network leakage. The present invention uses drone vision recognition technology and voiceprint analysis technology to solve the problem of low efficiency of pipeline network leakage monitoring faced by urban water supply systems.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent monitoring, and specifically relates to an intelligent monitoring method for water supply network leakage based on drone vision and voiceprint analysis. Background Art

[0002] At present, with the rapid development of artificial intelligence technology, the intelligent construction of urban water supply systems has become one of the hot research directions of scholars at home and abroad, and solving the problem of water supply pipeline leakage is an important support for the intelligent construction of urban water supply systems.

[0003] In view of the current situation of urban water supply pipe networks, relevant experts have proposed corresponding response strategies and solutions. At present, the commonly used leakage detection methods based on instruments and equipment include flow method, pressure method, listening method, correlation analysis method, tracer gas method, detection radar method, etc. These methods are easy to operate and carry, but they require high professionalism and are not suitable for large-scale detection. In addition, most researchers have established pipe network leakage prediction and positioning models based on algorithms or programming software, but the modeling has high requirements for computer language and the process is relatively complicated. At the same time, due to the lack of sufficient pipe network data and various uncertain factors, the model accuracy is low and the technology is immature in practical applications.

[0004] For water supply network leakage, there are three main forms of network leakage: seepage, open leakage and hidden leakage. Seepage refers to the leakage of a single leak point less than 400~500 L / h, which often occurs in pipe joints, pipe fittings with poor sealing, and pipes with corrosion and perforation. The leakage can be reduced by replacing pipe fittings, but this method has a large amount of engineering, high labor and equipment costs, and low feasibility. At present, water supply companies mainly reduce the leakage by reducing the water pressure of the pipe network. Open leakage refers to the underground water supply pipeline leakage point that can be directly determined visually. The occurrence of open leakage is due to the bursting of the pipe network and the leakage of valves. The main reasons for the bursting are external force damage, insufficient corrosion resistance of the pipe, and aging of the pipe over time. The amount of open leakage is large. Hidden leakage refers to the water supply pipeline leakage point buried underground, which requires certain means and methods to be determined. The occurrence of hidden leakage is related to factors such as the pressure, operation, soil conditions and pipeline conditions of the pipe network system. The characteristics of hidden leakage are that the amount of water leaked is small and it is not easy to be exposed to the surface. It requires careful inspection by leak detection workers to be discovered, and it usually does not have a significant impact on water pressure. Therefore, it may take a long time from the occurrence of the leakage to the final discovery. During this period, water resources are seriously wasted. The amount of water lost through hidden leakage accounts for about 55% of the total leakage.

[0005] For both visible and hidden leaks in pipe networks, the traditional method of detecting visible leaks is to install hardware facilities such as water flow meters or water pressure gauges at both ends of the water supply pipes to measure the water flow information flowing in and out of the pipes, and to detect whether the pipe has obvious leaks by monitoring the difference in the inflow and outflow information. In order to improve the detection accuracy of leaks, the traditional detection method needs to increase the installation density of water flow meters or water pressure gauges, which is costly and has a limited lifespan of hardware equipment, and there is also the risk of polluting water sources. There may not be signs on the ground when hidden leaks occur. The traditional method of detecting hidden leaks is for water industry experts to knock on the pipes and use a listening stick to listen to the sound waves transmitted back from the pipes, and estimate whether the pipes are leaking and the general location of the leaks based on subjective experience. However, this detection method relies too much on expert experience and knowledge, and the inspection accuracy is unstable and inefficient. Summary of the invention

[0006] The present invention provides an intelligent monitoring method for water supply network leakage based on drone vision and voiceprint analysis to solve the problem of low efficiency of network leakage monitoring faced by urban water supply systems.

[0007] The technical solution adopted by the present invention is:

[0008] On the one hand, the present invention provides a method for intelligently monitoring water supply network leakage based on vision and voiceprint analysis, comprising:

[0009] Acquire characteristic data and attribute data of the water supply pipeline to be inspected, and connect the characteristic data and attribute data to a geographic information system integrated with a water supply pipeline map;

[0010] Performing optimal path planning according to the feature data and attribute data;

[0011] The drone is made to perform inspection according to the optimal path, so that the drone collects pipeline laying road surface video data and audio signals of the water supply pipeline to be inspected during the inspection process;

[0012] Based on a multi-scale hybrid attention network and a pipeline acoustic detection network, the pipeline laying pavement video data and audio signals are processed and analyzed to obtain the location information of water supply network leakage.

[0013] On the other hand, the present invention provides an intelligent monitoring device for water supply network leakage based on visual and voiceprint analysis, comprising:

[0014] An acquisition module, used to acquire characteristic data and attribute data of the water supply pipeline to be inspected, and connect the characteristic data and attribute data to a geographic information system integrated with a water supply pipeline map;

[0015] A planning module, used for performing optimal path planning according to the feature data and attribute data;

[0016] A collection module is used to enable the drone to perform inspections according to the optimal path, and collect pipeline laying road surface video data and audio signals of the water supply pipeline to be inspected during the inspection process;

[0017] The analysis module is used to process and analyze the pipeline laying pavement video data and audio signals based on a multi-scale hybrid attention network and a pipeline acoustic detection network to obtain the location information of the water supply network leakage.

[0018] On the other hand, the present invention provides an electronic device, comprising: a shell, a processor, a memory, a circuit board and a power supply circuit, wherein the circuit board is placed inside the space enclosed by the shell, and the processor and the memory are arranged on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the above-mentioned method.

[0019] In yet another aspect, the present invention provides a computer-readable storage medium storing one or more programs, wherein the one or more programs can be executed by one or more processors to implement the above method.

[0020] Due to the adoption of the above technical solution, the beneficial effects achieved by the present invention are as follows:

[0021] Through drone visual recognition technology and voiceprint analysis technology, the present invention provides an efficient and accurate intelligent monitoring solution for pipe network leakage for urban water supply systems, which can effectively improve leakage detection efficiency, reduce leakage rate, reduce misjudgment and missed detection, and at the same time save water resources, reduce economic losses, and enhance the stability and reliability of the water supply system, making a certain contribution to the sustainable development of cities. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0023] Figure 1 It is a flow chart of the intelligent monitoring method for water supply network leakage based on drone vision and voiceprint analysis of the present invention;

[0024] Figure 2 This is a schematic diagram of the intelligent inspection of water supply pipelines based on drone vision of the present invention;

[0025] Figure 3 It is a technical roadmap for pipeline hidden leak detection based on voiceprint analysis in the present invention;

[0026] Figure 4 It is a structural schematic diagram of the intelligent monitoring device for water supply network leakage based on drone vision and voiceprint analysis of the present invention;

[0027] Figure 5 It is a schematic structural diagram of the electronic device of the present invention. DETAILED DESCRIPTION

[0028] In order to more clearly illustrate the overall concept of the present invention, a detailed description is given below in an exemplary manner in conjunction with the accompanying drawings.

[0029] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited to the specific embodiments disclosed below.

[0030] In response to the high leakage rate problem of urban water supply systems, the present invention proposes an intelligent monitoring method for water supply network leakage based on drone vision and voiceprint analysis. For the obvious leakage and hidden leakage problems of the water supply network, intelligent inspection of water supply pipelines and pipeline acoustic detection and leak detection are studied respectively.

[0031] On the one hand, the present invention provides a method for intelligent monitoring of water supply network leakage based on drone vision and voiceprint analysis, such as Figure 1-2 As shown, including:

[0032] Step S101: Acquire characteristic data and attribute data of the water supply pipeline to be inspected, and connect the characteristic data and attribute data to a geographic information system integrated with a water supply pipeline map;

[0033] In this step, the characteristic data includes: the location, length, diameter, material and layout of the water supply pipeline; the attribute data includes: the pressure level, water flow rate, installation time and service life of the water supply pipeline and other related information.

[0034] During the specific implementation, various data such as the location, length, diameter, material, layout, etc. of the water supply pipeline are collected, and the collected data are linked to the specific location of the GIS system. The water supply pipeline is represented as a line feature, and other relevant information such as pressure level, water flow velocity, installation time and service life of the pipeline are attached to these line features as attribute data. Not only can the leakage risk points be inferred based on the design pressure level of the water supply pipeline, the actual operating pressure and the water flow velocity of different pipe sections, but also the information such as installation time and service life can be used to assist in predicting possible aging and corrosion problems of the pipeline, and prompt the pipe section areas that need to be inspected first; at the same time, the drone is represented as a point feature, and the pipelines that need to be inspected are marked on the GIS system.

[0035] As an optional embodiment, Figure 3 As shown, step S101 may include:

[0036] Step A1: In the geographic information system, creating a layer for representing a water supply pipeline;

[0037] Step A2: according to the characteristic data, drawing the direction of the pipeline on the layer, and connecting the pipeline as a line element through nodes;

[0038] Step A3: adding the attribute data to the line element;

[0039] Step A4: In the geographic information system, creating another layer for representing the drone;

[0040] Step A5: Using the position of the UAV as a point element through a patrol point mark, and updating the coordinates of the patrol point in real time to reflect the actual flight position of the UAV;

[0041] Step A6: Writing the flight status information of the UAV into the geographic information system in real time to perform optimal path planning.

[0042] By completing the above steps A1-A6, the map modeling of the water supply pipeline to be inspected can be completed in the geographic information system.

[0043] It should be noted that, based on the above scheme, those skilled in the art may also adopt a scheme of creating only one layer, that is, adding line elements and point elements in the same layer at the same time. The above optional schemes all fall within the protection scope of this application.

[0044] Step S102: performing optimal path planning according to the feature data and attribute data;

[0045] In this step, the inspection path of the UAV is optimally planned based on the Dijkstra algorithm.

[0046] In the specific implementation, the inspection area of ​​the drone is first divided into multiple nodes, which can be key locations that need to be inspected (such as poles, equipment, etc.) or specific navigation points; then the weight of the edge is determined according to factors such as the distance between the nodes and the traffic conditions. For example, if the distance between two nodes is far or there are obstacles that make it difficult to fly, the weight of the corresponding edge is larger; then the starting point is set as the source point, the distance from the source point to itself is set to 0, and the distance to all other nodes is set to infinity, and a set is established to store the nodes for which the shortest path has been found. Initially, the set only contains the source point; for each node not in the set, the distance from the source point to the node through the nodes in the set is calculated, that is, for each node, the nodes in the set are traversed to see if the distance to the node can be updated through these nodes and the corresponding edges, and the node with the smallest distance is selected to be added to the set; repeat the previous step until the target node is added to the set, and finally start from the target node, according to the shortest path information recorded by each node (such as which node makes the distance the shortest), reverse backtracking is performed to obtain the shortest path from the source point to the target point.

[0047] It should be noted that, based on the above scheme, technical personnel in this field can also adopt the Bellman-Ford algorithm to optimally plan the inspection path of the UAV, that is, update the shortest distance from each node to the starting point through multiple iterations; adopt the bidirectional Dijkstra algorithm to optimally plan the inspection path of the UAV, that is, start searching from the starting point and the target point at the same time, and maintain the distance table and priority queue in the two directions respectively. The above optional schemes all fall within the scope of protection of this application.

[0048] Step S103: enabling the drone to perform inspection according to the optimal path, so that the drone collects pipeline laying road surface video data and audio signals of the water supply pipeline to be inspected during the inspection process;

[0049] In this step, the focus of the intelligent inspection of water supply pipelines based on drone vision is on the intelligent identification of abnormal conditions on the road surface where the water supply pipelines are laid, mainly including automatic obstacle avoidance of drones based on binocular vision technology, reconstruction of district-level water supply pipeline laying maps based on geographic information systems, and drone mission path planning and automatic cruising of drones based on reinforcement learning in dynamic environments.

[0050] In the specific implementation, the drone collects video data of the pipeline laying road surface of the water supply pipeline through the binocular camera carried by it, and collects audio signals through the audio collector carried by it. At the same time, during the intelligent inspection process of the drone, the drone also uses the binocular camera to collect stereo images of the environment around the pipeline in real time, processes the collected stereo images, and calculates the relative distance, three-dimensional position, shape and motion state of the obstacles in front according to the parallax between the images obtained by the two cameras. This process is similar to human binocular vision. Subsequently, the drone identifies and classifies the natural and artificial obstacles (such as trees, wires, street lights, etc.) that may exist in front, and determines their types and locations. After that, the drone uses the path planning algorithm to re-plan the path, calculates an optimal inspection path that avoids obstacles, and continues the inspection flight.

[0051] As an optional embodiment, step S103 may include:

[0052] Step B1: acquiring in real time stereoscopic image information of the environment surrounding the pipeline during the inspection and monitoring process of the drone, wherein the stereoscopic image information includes static obstacles and dynamic obstacles;

[0053] In this step, static obstacles include trees, wires, street lights, etc., and dynamic obstacles include birds, drones, etc.

[0054] Step B2: Processing the collected stereo images, and determining the relative distance, three-dimensional position, shape and motion state of the obstacle ahead according to the parallax between the images collected by the two cameras of the drone;

[0055] In this step, for static obstacles, only the relative distance, three-dimensional position and shape need to be confirmed, while for dynamic obstacles, the motion state needs to be additionally confirmed.

[0056] Step B3: Perform optimal path planning based on the relative distance, three-dimensional position, shape and motion state of the obstacle ahead, and determine the optimal inspection path to avoid the obstacle so that the UAV can continue the inspection flight.

[0057] In the present invention, a one-machine-multiple-collection method is adopted, so that the UAV can effectively carry out intelligent inspection and avoid obstacles while comprehensively collecting the status of the water supply pipeline, so as to timely discover and deal with leakage problems.

[0058] Step S104: Based on the multi-scale hybrid attention network and the pipeline acoustic detection network, the pipeline laying pavement video data and audio signal are processed and analyzed to obtain the location information of the water supply network leakage.

[0059] In this step, the location of the visible leak in the water supply network is obtained by processing and analyzing the video data of the pipeline laying road surface, and the location of the hidden leak in the water supply network is obtained by processing and analyzing the audio signal.

[0060] As an optional embodiment, the types of water supply network leakage include visible leakage and hidden leakage, the multi-scale hybrid attention network includes a multi-scale hybrid attention network module (Multi-scaleAttention, MsA), a foreground enhancement module (Foreground Enhancement Module, FEM), and a deep separable channel attention module (Deep Separable Channel Attention, DSCA), the pipeline acoustic detection network includes a multi-frequency signal separation and analysis network and a recurrent neural network model, and the step S104 includes:

[0061] Step C1: Processing the pipeline laying pavement video data based on a multi-scale hybrid attention network module, a foreground enhancement module, and a depth-separable channel attention module to obtain the location of the water supply network leak;

[0062] Since the height of the drone is not fixed during the inspection process, the scales of the target objects in the collected video data are inconsistent. This paper proposes a multi-scale hybrid attention network to enhance the feature representation of multi-scale objects.

[0063] Preferably, the step C1 may include:

[0064] Step C11: based on the multi-scale hybrid attention network module, the pipeline laying pavement video data collected when the drone is flying at a relatively high altitude is marked as a large object, and the pipeline laying pavement video data collected when the drone is flying at a relatively low altitude is marked as a small object;

[0065] In this step, since the large object is photographed from a high altitude, the details are not rich enough, and the small object is photographed from a low altitude, so the features are insufficient.

[0066] Step C12: adjusting the attention weights of the small object and the large object, and establishing a connection between the large object and the corresponding small object, wherein the small object is used to supplement the corresponding large object to enrich its details, and the large object is used to set off the corresponding small object to make its features sufficient;

[0067] In this step, the multi-scale hybrid attention network module can adjust the attention weight of the target object according to the multi-scale image data collected by the drone at different flight altitudes, so that the features of large objects and small objects are fully characterized, and the connection between large objects with rich details and small objects with insufficient features is established from the perspective of multi-scale features. This multi-scale analysis capability helps to detect leakage risks of different sizes in the panoramic view of the captured image. For example, in the images taken by drones, the image information in the full field of view scene is covered, and there is a problem of uneven distribution of ground cracks. Reflected in the real scene, there are long-distance ground cracks and close-range ground cracks, which are manifested as fine linear structures and wider crack morphology, that is, cracks of different scales. This module can fully capture the characteristic information of cracks of different scales and enhance the representation ability of the network.

[0068] Step C13: focusing on the foreground features of the associated large objects and small objects based on the foreground enhancement module, focusing on the large objects and small objects that are directly related to the risk of pipeline leakage.

[0069] In this step, during the inspection process, there will be redundant and interfering information such as trees, pedestrians or vehicles. In order to reduce the interference of redundant information, the foreground enhancement module focuses on the foreground features of the multi-scale features, focusing on objects directly related to the risk of pipeline leakage (such as water marks, ground crack areas, etc.), ignoring other irrelevant background information (such as road markings or plants), and further improving the accuracy of abnormal situation detection.

[0070] Step C14: extracting multidimensional channel information from multiple large objects and small objects based on the deeply separable channel attention module to obtain the location of the visible leak in the water supply network.

[0071] In this step, the deeply separable channel attention module is used to effectively extract multi-dimensional channel information to ensure that the extracted features are more accurate. For example, after using the foreground enhancement module to focus on the ground crack area, the deeply separable channel attention module can effectively extract the tiny wet marks or local crack features in the area, and separate the subtle feature information related to leakage, thereby improving the accuracy of leakage risk identification.

[0072] By completing the above steps C11-C14, the location of the obvious leakage of the water supply network can be obtained. In addition, due to the variety of abnormal conditions on the road surface where the pipeline is laid, the marked sample data cannot cover all abnormal conditions. For example, during the inspection process, a new type of leakage risk that was previously unmarked is encountered, such as an unconventional ground settlement pattern or a new type of crack feature. The present invention adopts a generative adversarial network (GAN) to learn the difference between the distribution characteristics of normal data and abnormal data, and detects abnormal samples or unmarked abnormal conditions that are different from normal samples, thereby improving the recognition ability and adaptability to unknown risks.

[0073] Step C2: Process the audio signal based on a multi-frequency signal separation and analysis network and a recurrent neural network model to locate the location of the hidden leak in the water supply network.

[0074] In terms of pipeline acoustic detection and leak detection, since the sound of water leakage is composed of three different noises: pipeline vibration sound, the sound wave is loud; the sound of water passing through the cracks of the pipeline, the sound color is related to the material of the impact object; the gurgling sound of water flowing through the soil gap. Therefore, the present invention applies audio intelligent processing technology to the task of water supply pipeline hidden leakage detection, which is used to detect whether the pipeline is leaking, the number of leaking points, the precise location of the leaking points, and the water loss rate. Specifically, the present invention proposes a pipeline hidden leakage detection method based on soundprint analysis, which mainly includes multi-frequency signal separation and analysis and high-precision sound source positioning based on recurrent neural network. When water flows out of the cracks or holes in the pipeline, the leakage causes vibration and sound waves. The captured sound waves are first separated and analyzed by multi-frequency signals using soundprint analysis technology to eliminate environmental noise, and then the three kinds of pipe network hidden leakage sounds, namely, leakage friction sound, water head impact sound, and medium friction sound, are located with high precision based on recurrent neural network to help operators accurately determine the location of the leakage point.

[0075] Preferably, if Figure 3 As shown, step C2 may include:

[0076] Step C21: converting the audio signal into a frequency domain representation using a short-time Fourier transform, separating the mixed signal into independent components based on an independent component analysis (ICA) method, removing environmental noise, and modeling the time series characteristics of the remaining components to obtain a time series signal;

[0077] This step is the separation and analysis of multi-frequency signals. Since the hidden leakage sound of the pipe network includes leakage friction sound, water head impact sound, medium friction sound and environmental noise, the first three sounds have different frequencies. Leakage friction sound refers to the sound produced by the friction between the water spraying out of the pipe and the leakage, and its frequency is usually 300-2500Hz, and it propagates far away along the pipe; water head impact sound refers to the sound produced by the collision between the water spraying out of the pipe and the surrounding medium, and it spreads to the ground through the soil in the form of a funnel, and its frequency is usually between 100-800 Hz; medium friction sound refers to the sound produced by the collision and friction between the surrounding particles (such as soil particles, sand particles, etc.) driven by the water spraying out of the pipe, and its frequency is relatively low. When multiple audio signals are mixed together, their frequencies and amplitudes will affect each other, making them present complex interlacing and overlapping in the time domain and frequency domain.

[0078] In the specific implementation, multiple independent components are separated into independent components based on the principle of maximizing independence based on the independent component analysis method. That is to say, through this step, the mixed audio signal (the collected audio signal) is separated and analyzed into leakage friction sound, water head impact sound, medium friction sound and environmental noise. Then, the environmental noise is removed, and the model is used to further analyze the leakage friction sound, water head impact sound and medium friction sound.

[0079] Step C22: Input the time series signal into the trained recurrent neural network model to locate the position of the hidden leak in the water supply network.

[0080] This step is a high-precision sound source localization based on a recurrent neural network. The sound source localization algorithm based on a recurrent neural network mainly models the temporal characteristics of the audio signal and uses the recurrent neural network to establish its association with the sound source position to achieve accurate sound source localization.

[0081] In terms of intelligent inspection of water supply pipelines, since drones are flexible and maneuverable, leaks are visually visible, and water supply pipelines are fixed, the present invention applies drone visual recognition to the intelligent inspection of water supply pipelines, automatically identifying safety hazards that affect pipeline leakage, such as pipeline laying road collapse, road construction, human excavation, and ground dry and wet conditions, as well as abnormal conditions such as large-scale leakage. It can discover pipeline leakage risk factors in advance, identify and locate burst or damaged areas in time, and minimize leakage as much as possible, thereby realizing intelligent unmanned inspections and greatly improving inspection efficiency. By shortening the inspection cycle, risk items that may cause obvious leakage in the pipeline network can be warned and dealt with in a timely manner, effectively reducing water supply leakage caused by external factors.

[0082] Preferably, the training method of the recurrent neural network model may include:

[0083] Step D1: Input the time series features of the labeled audio data into the recurrent neural network model;

[0084] In this step, the timing features of the audio data are divided into normal timing features and dark leakage timing features, and marked with corresponding labels, and then input into the recurrent neural network model.

[0085] Step D2: Obtain the model prediction result based on the input audio data;

[0086] Step D3: Compare the model prediction result with the label to determine the loss function value;

[0087] In this step, by selecting a suitable loss function and substituting the model prediction results and labels into the corresponding formula for calculation, the loss function value can be obtained. This loss function value reflects the degree of difference between the model prediction results and the true labels, and is the goal of optimization during the model training process. By continuously adjusting the model parameters to minimize the loss function value, the model performance can be improved.

[0088] Step D4: Adjust the parameters of the model according to the loss function value so that the prediction result gradually approaches the label.

[0089] In this method, in the modeling and training stage of sound source localization, the labeled speech signal is input into the model, and the recurrent neural network model is trained through the back propagation algorithm; in the model inference stage of sound source localization, the trained deep neural network model is used to predict the corresponding sound source given a new unknown speech signal, thereby achieving high-precision sound source localization.

[0090] In summary, the present invention has the following beneficial effects:

[0091] Through drone visual recognition technology and voiceprint analysis technology, the present invention provides an efficient and accurate intelligent monitoring solution for pipe network leakage for urban water supply systems, which can effectively improve leakage detection efficiency, reduce leakage rate, reduce misjudgment and missed detection, and at the same time save water resources, reduce economic losses, and enhance the stability and reliability of the water supply system, making a certain contribution to the sustainable development of cities.

[0092] On the other hand, the present invention provides an intelligent monitoring device for water supply network leakage based on visual and voiceprint analysis, such as Figure 4 As shown, including:

[0093] An acquisition module 11 is used to acquire characteristic data and attribute data of the water supply pipeline to be inspected, and connect the characteristic data and attribute data to a geographic information system integrated with a water supply pipeline map;

[0094] A planning module 12, used for performing optimal path planning according to the feature data and attribute data;

[0095] A collection module 13 is used to enable the drone to perform inspection according to the optimal path, and collect pipeline laying road surface video data and audio signals of the water supply pipeline to be inspected during the inspection process;

[0096] The analysis module 14 is used to process and analyze the pipeline laying pavement video data and audio signals based on a multi-scale hybrid attention network and a pipeline acoustic detection network to obtain the location information of the water supply network leakage.

[0097] The device of this embodiment can be used to perform Figure 1The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0098] Preferably, the types of water supply network leakage include visible leakage and hidden leakage, the multi-scale hybrid attention network includes a multi-scale hybrid attention network module, a foreground enhancement module, and a deep separable channel attention module, the pipeline acoustic detection network includes a multi-frequency signal separation and analysis network and a recurrent neural network model, and the analysis module 14 includes:

[0099] An open leak detection submodule is used to process the pipeline laying pavement video data based on a multi-scale hybrid attention network module, a foreground enhancement module, and a depth-separable channel attention module to obtain the location of the open leak in the water supply network;

[0100] The hidden leak detection submodule is used to process the audio signal based on a multi-frequency signal separation and analysis network and a recurrent neural network model to locate the hidden leak in the water supply network.

[0101] Preferably, the visible leakage detection submodule includes:

[0102] a marking unit, configured to mark the pipeline paving road video data collected when the drone is flying at a relatively high altitude as a large object, and mark the pipeline paving road video data collected when the drone is flying at a relatively low altitude as a small object based on the multi-scale hybrid attention network module;

[0103] An adjustment unit, used to adjust the attention weights of the small object and the large object, and to establish a connection between the large object and the corresponding small object, wherein the small object is used to supplement the corresponding large object to enrich its details, and the large object is used to set off the corresponding small object to make its features sufficient;

[0104] A focusing unit, used for focusing on the foreground features of the associated large objects and small objects based on the foreground enhancement module, focusing on the large objects and small objects directly related to the pipeline leakage risk;

[0105] An extraction unit is used to extract multidimensional channel information from multiple large objects and small objects based on the deep separable channel attention module to obtain the location of the visible leak in the water supply network.

[0106] Preferably, the dark leakage detection submodule includes:

[0107] A conversion unit, configured to convert the audio data into a frequency domain representation using a short-time Fourier transform, separate the mixed signal into independent components based on an independent component analysis method, and after removing environmental noise, model the time series characteristics of the remaining components to obtain a time series signal;

[0108] The positioning unit is used to input the time series signal into the trained recurrent neural network model to locate the position of the hidden leak in the water supply network.

[0109] Preferably, the collection module 13 includes:

[0110] An acquisition submodule is used to acquire in real time the stereoscopic image information of the environment around the pipeline during the inspection and monitoring process of the drone, wherein the stereoscopic image information includes static obstacles and dynamic obstacles;

[0111] A processing submodule, for processing the collected stereo images, and determining the relative distance, three-dimensional position, shape and motion state of the obstacle ahead according to the parallax between the images collected by the two cameras of the drone;

[0112] The first planning submodule performs optimal path planning according to the relative distance, three-dimensional position, shape and motion state of the obstacle ahead, and determines the optimal inspection path to avoid the obstacle so that the UAV can continue the inspection flight.

[0113] Preferably, the acquisition module 11 includes:

[0114] A first creation submodule is used to create a layer for representing a water supply pipeline in the geographic information system;

[0115] A connection submodule, for drawing the direction of the pipeline on the layer according to the characteristic data, and connecting the pipeline as a line element through nodes;

[0116] An adding submodule, used for adding the attribute data to the line feature;

[0117] A second creation submodule is used to create another layer for representing the drone in the geographic information system;

[0118] An identification submodule, used to identify the position of the UAV as a point element through a patrol point, and update the coordinates of the patrol point in real time to reflect the actual flight position of the UAV;

[0119] The second planning submodule is used to write the flight status information of the UAV into the geographic information system in real time to perform optimal path planning.

[0120] Preferably, the training method of the recurrent neural network model includes:

[0121] Input the time series features of the labeled audio data into the recurrent neural network model;

[0122] According to the input audio data, the model prediction results are obtained;

[0123] Compare the model prediction result with the label and calculate the loss function value;

[0124] The parameters of the model are adjusted according to the loss function value so that the prediction result gradually approaches the label.

[0125] The present invention also provides an electronic device, Figure 5 The schematic diagram of the structure of the electronic device of the present invention can realize the present invention Figure 1 The process shown is as follows Figure 5 As shown, the electronic device may include: a housing 41, a processor 42, a memory 43, a circuit board 44 and a power supply circuit 45, wherein the circuit board 44 is placed inside the space enclosed by the housing 41, and the processor 42 and the memory 43 are arranged on the circuit board 44; the power supply circuit 45 is used to supply power to various circuits or devices of the electronic device; the memory 43 is used to store executable program codes; the processor 42 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 43, so as to execute the method described in any of the aforementioned method embodiments.

[0126] For details on the specific execution process of the above steps by the processor 42 and the steps further executed by the processor 42 by running the executable program code, please refer to the present invention. Figure 1 The description of the illustrated embodiment will not be repeated here.

[0127] This electronic device exists in many forms, including but not limited to:

[0128] (1) Mobile communication devices: These devices are characterized by their mobile communication functions and their main purpose is to provide voice and data communications. These terminals include: smart phones (such as iPhone), multimedia phones, functional phones, and low-end phones.

[0129] (2) Ultra-mobile personal computer devices: These devices fall into the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. These terminals include: PDA, MID and UMPC devices, such as iPad.

[0130] (3) Portable entertainment devices: These devices can display and play multimedia content. They include audio and video players (such as iPods), handheld game consoles, e-books, smart toys, and portable car navigation devices.

[0131] (4) Server: A device that provides computing services. The server consists of a processor, hard disk, memory, system bus, etc. The server is similar to a general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0132] (5) Other electronic devices with data interaction functions.

[0133] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any of the above method embodiments are implemented.

[0134] The present invention also provides an application program, which is executed to implement the method provided by any method embodiment of the present invention.

[0135] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0136] Each embodiment in this specification is described in a related manner, and the same and similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. For the convenience of description, the description of the above device is divided into various units / modules according to their functions and described separately. Of course, when implementing the present invention, the functions of each unit / module can be implemented in the same or one or more software and / or hardware.

[0137] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. An intelligent monitoring method for water supply network leakage based on drone vision and voiceprint analysis, characterized in that: include: Acquire characteristic data and attribute data of the water supply pipeline to be inspected, and connect the characteristic data and attribute data to a geographic information system integrated with a water supply pipeline map; Performing optimal path planning according to the feature data and attribute data; The drone is made to perform inspection according to the optimal path, so that the drone collects pipeline laying road surface video data and audio signals of the water supply pipeline to be inspected during the inspection process; Based on a multi-scale hybrid attention network and a pipeline acoustic detection network, the pipeline laying pavement video data and audio signals are processed and analyzed to obtain the location information of the water supply network leakage; The pipeline laying pavement video data is processed based on a multi-scale hybrid attention network module, a foreground enhancement module, and a depth-separable channel attention module to obtain the location of the visible leak in the water supply network; Processing the audio signal based on a multi-frequency signal separation and analysis network and a recurrent neural network model to locate the location of the hidden leak in the water supply network; The types of water supply network leakage include obvious leakage and hidden leakage. The multi-scale hybrid attention network includes the multi-scale hybrid attention network module, the foreground enhancement module, and the deep separable channel attention module. The pipeline acoustic detection network includes the multi-frequency signal separation and analysis network and the recurrent neural network model.

2. The method according to claim 1, characterized in that The multi-scale hybrid attention network module, the foreground enhancement module, and the depth-separable channel attention module are used to process the pipeline laying pavement video data to obtain the location of the water supply network leak, including: Based on the multi-scale hybrid attention network module, the pipeline laying pavement video data collected when the drone is flying at a relatively high altitude is marked as a large object, and the pipeline laying pavement video data collected when the drone is flying at a relatively low altitude is marked as a small object; Adjust the attention weights of the small object and the large object, and establish a connection between the large object and the corresponding small object, wherein the small object is used to supplement the corresponding large object to enrich its details, and the large object is used to set off the corresponding small object to make its features sufficient; Focusing on the foreground features of the large and small objects that are connected based on the foreground enhancement module, focusing on the large and small objects that are directly related to the risk of pipeline leakage; Based on the deeply separable channel attention module, multi-dimensional channel information is extracted from multiple large objects and small objects to obtain the location of the visible leak in the water supply network.

3. The method according to claim 1, characterized in that Processing the audio signal based on a multi-frequency signal separation and analysis network and a recurrent neural network model to locate the location of the hidden leak in the water supply network includes: The audio signal is converted into a frequency domain representation using a short-time Fourier transform, the mixed signal is separated into independent components based on an independent component analysis method, and after removing environmental noise, the time series characteristics of the remaining components are modeled to obtain a time series signal; The time series signal is input into a trained recurrent neural network model to locate the position of the hidden leak in the water supply network.

4. The method according to any one of claims 1 to 3, characterized in that: The method of causing the drone to perform inspection according to the optimal path so that the drone collects pipeline laying road surface video data and audio signals of the water supply pipeline to be inspected during the inspection process includes: Acquire in real time the stereoscopic image information of the environment around the pipeline during the inspection and monitoring process of the drone, wherein the stereoscopic image information includes static obstacles and dynamic obstacles; Processing the collected stereo images, and determining the relative distance, three-dimensional position, shape and motion state of the obstacle ahead according to the parallax between the images collected by the two cameras of the drone; The optimal path planning is performed according to the relative distance, three-dimensional position, shape and motion state of the obstacle ahead, and the optimal inspection path avoiding the obstacle is determined so that the UAV can continue the inspection flight.

5. The method according to claim 4, characterized in that The step of obtaining characteristic data and attribute data of the water supply pipeline to be inspected and connecting the characteristic data and attribute data to a geographic information system includes: In the geographic information system, creating a layer for representing a water supply pipeline; According to the characteristic data, the direction of the pipeline is drawn on the layer, and the pipeline is connected through nodes as a line element; adding the attribute data to the line feature; In the geographic information system, creating another layer for representing the drone; The position of the UAV is used as a point element through a patrol point mark, and the coordinates of the patrol point are updated in real time to reflect the actual flight position of the UAV; The flight status information of the UAV is written into the geographic information system in real time to perform optimal path planning.

6. The method according to claim 4, characterized in that The training method of the recurrent neural network model includes: Input the time series features of the labeled audio data into the recurrent neural network model; According to the input audio data, the model prediction results are obtained; Compare the model prediction result with the label to determine the loss function value; The parameters of the model are adjusted according to the loss function value so that the model prediction result gradually approaches the label.

7. An intelligent monitoring device for water supply network leakage based on vision and voiceprint analysis, characterized in that: include: An acquisition module, used to acquire characteristic data and attribute data of the water supply pipeline to be inspected, and connect the characteristic data and attribute data to a geographic information system integrated with a water supply pipeline map; A planning module, used for performing optimal path planning according to the feature data and attribute data; A collection module is used to enable the drone to perform inspections according to the optimal path, and collect pipeline laying road surface video data and audio signals of the water supply pipeline to be inspected during the inspection process; An analysis module is used to process and analyze the pipeline laying pavement video data and audio signals based on a multi-scale hybrid attention network and a pipeline acoustic detection network to obtain location information of water supply network leakage; The pipeline laying pavement video data is processed based on a multi-scale hybrid attention network module, a foreground enhancement module, and a depth-separable channel attention module to obtain the location of the visible leak in the water supply network; Processing the audio signal based on a multi-frequency signal separation and analysis network and a recurrent neural network model to locate the location of the hidden leak in the water supply network; The types of water supply network leakage include obvious leakage and hidden leakage. The multi-scale hybrid attention network includes the multi-scale hybrid attention network module, the foreground enhancement module, and the neglected deep separable channel attention module. The pipeline acoustic detection network includes the multi-frequency signal separation and analysis network and the recurrent neural network model.

8. An electronic device, characterized in that: The electronic device comprises: a housing, a processor, a memory, a circuit board and a power supply circuit, wherein the circuit board is placed inside the space enclosed by the housing, and the processor and the memory are arranged on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the above-mentioned electronic device; the memory is used to store executable program codes; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute any of the methods described in claims 1-6 above.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any of the methods described in claims 1-6.

Images