Water supply network leakage identification method and device based on data enhancement and hybrid neural network architecture, equipment and medium

Abstract

The application relates to a water supply network leakage identification method and device based on data enhancement and a hybrid neural network architecture, equipment and a medium. The method comprises the following steps: generating a simulated pipeline vibration audio signal by using a generative adversarial network, combining the simulated pipeline vibration audio signal with a real pipeline vibration audio signal, and constructing a pipeline vibration audio signal dataset; fusing a mel-frequency cepstral coefficient feature matrix and a first-order difference matrix of the mel-frequency cepstral coefficient to determine a feature fusion matrix of each pipeline vibration audio signal frame; taking the feature fusion matrix as a training sample, taking a leakage state of the pipeline vibration audio signal as a sample label, training a CNN-BiLSTM hybrid neural network model, and determining a water supply network leakage detection model; and inputting a to-be-identified pipeline vibration audio signal into the water supply network leakage detection model to determine whether the to-be-identified pipeline vibration audio signal is in a leakage state or a non-leakage state. The application significantly improves the precision, robustness and generalization ability of leakage detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of water supply network leakage detection, and in particular to a water supply network leakage identification method based on data enhancement and a hybrid neural network architecture, a corresponding device, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] The water supply network is a crucial part of urban infrastructure, and its operating state is directly related to the quality of life of urban residents and economic development. However, the leakage problem of the water supply network has always been a major problem in urban water supply management. Network leakage not only leads to water resource waste and increases the cost of water supply, but also causes environmental pollution and waste of social resources.

[0003] Traditional leakage detection methods (such as the sound bar method) rely on manual experience and are inefficient, making it difficult to meet real-time monitoring needs. In the existing technology, although the leakage identification method based on pipe vibration audio signals has made some progress, there are problems such as insufficient feature extraction and insufficient model generalization ability. For example, existing methods mostly use a single static acoustic feature (such as MFCC), ignoring the distinguishing ability of dynamic features for leakage signals, resulting in limited recognition accuracy.

[0004] With the rapid development of artificial intelligence (AI) technology, especially the wide application prospect of deep learning (DL) technology in system abnormal condition diagnosis, such methods have played a positive role in water supply network leakage detection. However, when using machine learning methods for network leakage detection, a large amount of labeled data is often relied on. In the process of collecting pipe vibration audio signals of water supply pipelines in a leakage state of the water supply network, it is likely to encounter a situation of few signal data samples.

[0005] In summary, due to the lack of labeled data of pipe vibration audio signals of water supply pipelines in a leakage state of the existing technology, the existing machine learning methods have problems such as low recognition accuracy, poor robustness, and weak generalization ability in identifying water supply network leakage. The present application makes corresponding explorations to solve this problem. SUMMARY

[0006] The purpose of the present application is to solve the above problems and provide a water supply network leakage identification method based on data enhancement and a hybrid neural network architecture, a corresponding device, an electronic device, and a computer readable storage medium.

[0007] To achieve the various purposes of the present application, the present application adopts the following technical solutions:

[0008] A water supply network leakage identification method based on data enhancement and hybrid neural network architecture is proposed to adapt to one of the purposes of the present application, comprising:

[0009] Collect real pipe vibration audio signals in the water supply network, input the real pipe vibration audio signals into the preset generative adversarial network, and output through different rounds to generate simulated pipe vibration audio signals, combine the real pipe vibration audio signals with the simulated pipe vibration audio signals to construct a pipe vibration audio signal dataset, wherein the pipe vibration audio signal dataset includes a plurality of pipe vibration audio signals with a certain audio duration, and each pipe vibration audio signal includes a plurality of pipe vibration audio signal frames;

[0010] After the pipe vibration audio signals are processed in turn by using fast Fourier transform and a mel filter bank, the mel frequency cepstral coefficients corresponding to each pipe vibration audio signal frame in the pipe vibration audio signal are extracted by using discrete cosine transform to construct a mel frequency cepstral coefficient feature matrix of the pipe vibration audio signal frame.

[0011] The first difference matrix of the mel frequency cepstral coefficients is calculated, and the mel frequency cepstral coefficient feature matrix and the first difference matrix are feature fused to determine the feature fusion matrix of each pipe vibration audio signal frame.

[0012] The feature fusion matrix of each pipe vibration audio signal frame in the pipe vibration audio signal is used as a training sample, and the corresponding leakage state of the pipe vibration audio signal is used as a sample label to train a preset CNN-BiLSTM hybrid neural network model to determine a water supply network leakage detection model trained to a convergence state.

[0013] The pipe vibration audio signal to be identified is input into the water supply network leakage detection model trained to a convergence state to determine whether the pipe corresponding to the pipe vibration audio signal to be identified is in a leakage state or a non-leakage state, thereby completing the water supply network leakage identification based on data enhancement and hybrid neural network architecture.

[0014] Optionally, the step of determining a plurality of pipe vibration audio signal frames comprises:

[0015] The audio duration, frame duration, overlap duration and sampling rate of each pipe vibration audio signal are obtained.

[0016] The first product between the audio duration and the sampling rate is calculated, the second product between the frame duration and the sampling rate is calculated, and the first difference between the first product and the second product is calculated.

[0017] calculating a second difference between the frame length and the overlap length to determine a hop length, and calculating a third product between the hop length and the sampling rate;

[0018] determining a frame number of each pipeline vibration audio signal according to a first ratio between the first difference and the third product, and framing the pipeline vibration audio signal according to the frame number to determine a plurality of pipeline vibration audio signal frames in each pipeline vibration audio signal.

[0019] Optionally, the step of training the generative adversarial network comprises:

[0020] obtaining key parameters, initializing the network structure of the generator and the discriminator in the preset generative adversarial network, initializing the network weight, and using the Adam optimizer, wherein the key parameters include a data set path, a batch size, a random noise dimension, a training round number, a learning rate, and a sampling rate;

[0021] obtaining the path of all real pipeline vibration audio signals in the audio data set, creating a data set object, and generating a data loader for the generator and the discriminator, respectively;

[0022] In each training round, the data loader is traversed for training, the discrimination results of the real pipeline vibration audio signal and the simulated pipeline vibration audio signal are calculated, a gradient penalty term is calculated to ensure the continuity of the discriminator, the parameters of the discriminator are updated to minimize the discrimination ability of the discriminator on the generated signal, and the loss value of the discriminator is recorded;

[0023] the simulated pipeline vibration audio signal is generated and the output of the discriminator is calculated, the parameters of the generator are updated to maximize the discrimination result of the discriminator on the simulated pipeline vibration audio signal, and the loss value of the generator is recorded;

[0024] Every certain training round, the simulated pipeline vibration audio signal is generated and saved to a specified directory until the generator can generate a high-quality simulated pipeline vibration audio signal, the trained generator is saved, the trained generator is used to generate a new simulated pipeline vibration audio signal, and the simulated pipeline vibration audio signal is combined with the real pipeline vibration audio signal for data enhancement.

[0025] Optionally, after the pipeline vibration audio signal is processed in sequence by using the fast Fourier transform and the mel filter bank, the step of extracting the mel cepstral coefficient corresponding to each pipeline vibration audio signal frame in the pipeline vibration audio signal by using the discrete cosine transform to construct the mel cepstral coefficient feature matrix of the pipeline vibration audio signal frame comprises:

[0026] performing fast Fourier transform on the pipeline vibration audio signals to convert time domain signals of each pipeline vibration audio signal frame into frequency domain data to obtain spectral information, wherein the spectral information reflects energy distribution of the pipeline vibration audio signals at different frequencies;

[0027] filtering the pipeline vibration audio signals according to the spectral information by using each mel filter in a mel filter bank to determine output energy on each mel filter channel, performing logarithmic operation on the output energy on each mel filter channel to obtain logarithmic energy of each mel filter channel, and converting the logarithmic energy into a log-mel spectrum;

[0028] extracting mel cepstral coefficients according to the logarithmic energy of each mel filter channel by using discrete cosine transform to obtain a mel cepstral coefficient feature matrix corresponding to each pipeline vibration audio signal frame in the pipeline vibration audio signals.

[0029] Optionally, a feature fusion matrix of each pipeline vibration audio signal frame in the pipeline vibration audio signals is taken as a training sample, a leakage state corresponding to the pipeline vibration audio signals is taken as a sample label, a pre-set CNN-BiLSTM hybrid neural network model is trained to determine a water supply network leakage detection model trained to a convergent state, and the step includes:

[0030] a pre-set Bayesian optimization algorithm is used to perform model hyperparameter optimization on the CNN-BiLSTM hybrid neural network model to determine an optimal hyperparameter combination of the CNN-BiLSTM hybrid neural network model, wherein the model hyperparameters include a learning rate, a batch size, and an LSTM hidden unit number;

[0031] the CNN-BiLSTM hybrid neural network model with the optimal hyperparameter combination is taken as the water supply network leakage detection model.

[0032] Optionally, a basic network architecture of the water supply network leakage detection model is a CNN-BiLSTM hybrid neural network model, wherein the CNN-BiLSTM hybrid neural network model is composed of two convolutional neural network layers and one bidirectional long short-term memory network layer, and the convolutional neural network is composed of two convolutional layers and two pooling layers.

[0033] Optionally, the feature fusion matrix represents joint features of the pipeline vibration audio signals in the time domain and the frequency domain, and not only contains static spectral information of the signals, but also combines dynamic characteristics of the signals changing over time.

[0034] Another object of the present application is to provide a water supply network leakage identification device based on data enhancement and a hybrid neural network architecture, comprising:

[0035] An audio data set construction module is configured to collect real pipe vibration audio signals in a water supply network, input the real pipe vibration audio signals into a preset generative adversarial network, generate simulation pipe vibration audio signals through different rounds of output, and combine the real pipe vibration audio signals with the simulation pipe vibration audio signals to construct a pipe vibration audio signal data set, wherein the pipe vibration audio signal data set includes a plurality of pipe vibration audio signals with a certain audio duration, and each pipe vibration audio signal includes a plurality of pipe vibration audio signal frames.

[0036] A mel-frequency cepstral coefficient extraction module is configured to sequentially process the pipe vibration audio signals by using fast Fourier transform and a mel filter bank, and extract mel-frequency cepstral coefficients corresponding to each pipe vibration audio signal frame in the pipe vibration audio signal by using discrete cosine transform to construct a mel-frequency cepstral coefficient feature matrix of the pipe vibration audio signal frame.

[0037] A feature fusion matrix construction module is configured to calculate and determine a first-order difference matrix of the mel-frequency cepstral coefficients, and perform feature fusion on the mel-frequency cepstral coefficient feature matrix and the first-order difference matrix to determine a feature fusion matrix of each pipe vibration audio signal frame.

[0038] A detection model training module is configured to take the feature fusion matrix of each pipe vibration audio signal frame in the pipe vibration audio signal as a training sample, take a leakage and loss state corresponding to the pipe vibration audio signal as a sample label, train a preset CNN-BiLSTM hybrid neural network model, and determine a water supply network leakage and loss detection model trained to a convergent state.

[0039] A water supply network leakage and loss identification module is configured to input a pipe vibration audio signal to be identified into the water supply network leakage and loss detection model trained to the convergent state, determine whether a pipe corresponding to the pipe vibration audio signal to be identified is in a leakage and loss state or a non-leakage and loss state, and complete water supply network leakage and loss identification based on data enhancement and a hybrid neural network architecture.

[0040] Another object of the present application is to provide an electronic device including a central processing unit and a memory, wherein the central processing unit is configured to invoke a computer program stored in the memory to execute the steps of the water supply network leakage and loss identification method based on data enhancement and a hybrid neural network architecture.

[0041] Another object of the present application is to provide a computer readable storage medium storing a computer program implemented according to the water supply network leakage identification method based on data enhancement and hybrid neural network architecture in the form of computer readable instructions, which, when invoked and run by a computer, performs the steps included in the corresponding method.

[0042] With respect to the prior art, the present application is directed to the problem of the scarcity of pipeline vibration audio signal annotation data for water supply pipelines in a leakage state in the prior art, and the problems of low recognition accuracy, poor robustness, and weak generalization ability of existing machine learning methods in identifying water supply network leakage. The present application includes but is not limited to the following

[0043] Advantages:

[0044] First, traditional leakage detection methods rely on a large amount of labeled data, while the collection of pipeline vibration audio signals for water supply network leakage often suffers from a lack of samples. By using a generative adversarial network (WaveGAN), the training data set can be effectively expanded to solve the problem of data scarcity, thereby improving the effectiveness of the training model. By synthesizing real and simulated pipeline vibration audio signals, the robustness of the model can be enhanced in different signal and noise environments, ensuring the adaptability of the leakage detection in diverse environments.

[0045] Second, existing technologies often rely on a single acoustic feature to identify leakage signals, but often ignore the influence of dynamic features. By fusing the Mel cepstrum coefficient with its first-order difference feature matrix, the present application extracts more rich time and frequency domain features, which can more accurately reflect the change rule of the leakage signal and enhance the model's ability to distinguish leakage signals.

[0046] Third, the Mel cepstrum coefficient is used to process the spectral feature, and BiLSTM is used to capture the time series feature, so that the model can consider both the instantaneous spectral feature and the time variation rule of the pipeline vibration audio signal of the water supply pipeline in a leakage state, thereby improving the accuracy of leakage identification.

[0047] Fourth, the present application is based on a CNN-BiLSTM hybrid architecture. Convolutional neural networks (CNN) are good at extracting spatial features, especially when dealing with high-dimensional data, they have strong feature learning ability. The bidirectional long short-term memory network (BiLSTM) can fully utilize the bidirectional information of the time series and capture the long-term dependence of the signal. By combining the advantages of the two, both local and global feature information can be considered when processing the vibration audio signals of the water supply network leakage, improving the recognition effect. After extracting the spatial features by CNN, BiLSTM further mines the time series information, and finally realizes the high-precision output of the leakage identification model. This hybrid architecture introduces a more complex network structure based on traditional methods, making the leakage identification more accurate.

[0048] Fifthly, the application significantly improves the automation and intelligence level of leakage detection, reducing the vulnerability of traditional manual experience dependence. In particular, based on the automatic detection of the neural network model, the leakage state of the pipeline can be identified in real time, avoiding the neglect or misjudgment of leakage. Compared with the traditional manual detection method, the technology can identify leakage without the need for a large amount of manual intervention, reducing labor costs and improving detection efficiency.

[0049] Sixthly, the application can accurately identify the leakage state, and can perform pipeline maintenance and repair in advance to avoid sudden accidents such as water pipe burst, thereby improving the stability and safety of the water supply system. Especially in large-scale water supply pipe network, systematic leakage detection can effectively prevent water supply interruption and the resulting social and economic losses.

[0050] Further, the application effectively solves the problems of data scarcity, insufficient feature extraction, and low model precision in water supply pipe network leakage detection by combining data enhancement with hybrid neural network architecture, significantly improving the precision, robustness, and generalization ability of leakage detection. This technical solution not only improves the efficiency of water supply pipe network leakage detection and reduces the economic and environmental losses caused by leakage, but also further promotes the development of intelligent management of water supply systems, and has a wide application prospect. BRIEF DESCRIPTION OF DRAWINGS

[0051] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

[0052] Figure 1 FIG. 1 is a flowchart of a water supply pipe network leakage identification method based on data enhancement and hybrid neural network architecture in an embodiment of the present application;

[0053] Figure 2 FIG. 4 is a flowchart of MFCC feature matrix extraction in an embodiment of the present application;

[0054] Figure 3 FIG. 6 is a flowchart of the feature extraction step of the convolutional layer in an embodiment of the present application;

[0055] Figure 4 FIG. 8 is a schematic diagram of the Bi LSTM unit structure in an embodiment of the present application;

[0056] Figure 5 FIG. 10 is a schematic diagram of the pipeline vibration audio signal processing and CNN-Bi LSTM hybrid neural network structure in an embodiment of the present application;

[0057] Figure 6 FIG. 12 is a principle block diagram of a water supply pipe network leakage identification device based on data enhancement and hybrid neural network architecture in an embodiment of the present application;

[0058] Figure 7 FIG. 1 is a structural schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0059] Embodiments of the present application are described in detail below with reference to the attached drawing figures, wherein the embodiments of the application are shown as examples only, and are not intended to be limiting of the present application. The following detailed description is presented in the context of a computer device, and is presented by way of example only.

[0060] As will be understood by those skilled in the art, the singular forms "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. It should be further understood that the terms "comprise," "comprises," "comprising," "include," "includes," "including," "contain," "contains," "containing," and the like, when used herein, mean "including but not limited to." It is further understood that when a particular element is indicated to be "connected" or "coupled" to another element, it can be directly connected or coupled to the other element, or intervening elements can be present. In addition, the use of "connection" or "coupling" herein includes wireless connection or wireless coupling. The use of the term "and / or" includes all combinations of one or more of the associated listed items.

[0061] As will be understood by those skilled in the art, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains, unless otherwise defined. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0062] Those skilled in the art will understand that, as used herein, the terms "client," "terminal," and "terminal device" include both devices that are solely wireless signal receivers and devices that have both receiving and transmitting hardware that can communicate bi-directionally over a bi-directional communication link. Such devices can include cellular or other communication devices with single-line or multiple-line displays, or no display, Personal Communications Service (PCS) devices that can combine a voice and / or data processor, a PDA that can include a radio frequency receiver and a pager, Internet and / or Intranet access, a Web browser, a calendar, and / or a GPS receiver, a conventional laptop and / or palmtop computer and / or other devices that have a radio frequency receiver. As used herein, the terms "client," "terminal," and "terminal device" can be portable, transportable, mounted in a vehicle (aeronautical, maritime, and / or land), or adapted and / or configured for local and / or distributed operation on Earth and / or any other location in space. As used herein, the terms "client," "terminal," and "terminal device" can also be a communication terminal, an Internet terminal, a music / video playing terminal, such as a PDA, a Mobile Internet Device (MID), and / or a mobile phone with music / video playing function, a smart television, a set-top box, and / or the like.

[0063] As used herein, the terms "server," "client," "service node," and the like refer to hardware that has the equivalent capability of a personal computer, i.e., an electronic device having a central processing unit (including an arithmetic unit and a controller), a memory, an input device, and an output device, and the like necessary components disclosed by the Von Neumann principle. A computer program is stored in the memory, the central processing unit loads the program stored in the external memory into the memory and runs it, executes the instructions in the program, and interacts with the input and output devices, thereby completing a specific function.

[0064] It should be noted that the concept of "server" in the present application can also be extended to the case of a server cluster. According to the principle of network deployment understood by those skilled in the art, the servers should be logically divided, and in physical space, these servers can be independent of each other but can be called through an interface, or can be integrated into a physical computer or a computer cluster. Those skilled in the art should understand this variation and should not be restricted by the implementation of the network deployment of the present application.

[0065] One or more technical features of the present application, unless explicitly specified, can be deployed on a server and accessed by remotely calling the online service interface provided by the server, or can be directly deployed and run on a client to implement access.

[0066] The neural network model referred to or possibly referred to in the present application, unless explicitly specified, can be deployed on a remote server and remotely called at the client, or can be deployed on a client with sufficient device capability for direct calling. In some embodiments, when it runs on a client, its corresponding intelligence can be obtained through transfer learning to reduce the requirement for client hardware running resources and avoid excessive occupation of client hardware running resources.

[0067] The various data involved in the present application, unless explicitly specified, can be remotely stored on a server or stored on a local terminal device, as long as it is suitable for being called by the technical solutions of the present application.

[0068] Those skilled in the art should know that the various methods of the present application, although based on the same concept and described to present commonality between them, are independently executable unless otherwise specified. Similarly, for each embodiment disclosed in the present application, it is based on the same inventive concept, so the same concept is understood to be equivalent, and although the concept is expressed differently, it is only for convenience and is appropriately transformed.

[0069] Unless it is explicitly stated that the embodiments disclosed in the present application are mutually exclusive, the technical features involved in each embodiment can be combined flexibly to construct new embodiments, as long as such combination does not deviate from the spirit of the present application and can meet the needs of the prior art or solve some deficiencies in the prior art. For this variation, those skilled in the art should know.

[0070] Please refer to Figure 1 The water supply network leakage identification method based on data enhancement and hybrid neural network architecture of the present application includes, in one embodiment thereof:

[0071] Step S10, collecting a real pipe vibration audio signal in the water supply network, inputting the real pipe vibration audio signal into a preset generative adversarial network, outputting through different rounds to generate a simulated pipe vibration audio signal, combining the real pipe vibration audio signal with the simulated pipe vibration audio signal to construct a pipe vibration audio signal dataset, wherein the pipe vibration audio signal dataset includes a plurality of pipe vibration audio signals with a certain audio duration, and each pipe vibration audio signal includes a plurality of pipe vibration audio signal frames.

[0072] The water supply network leakage identification system in the terminal device can collect a real pipe vibration audio signal in the water supply network, input the real pipe vibration audio signal into a preset generative adversarial network, output through different rounds to generate a simulated pipe vibration audio signal, and combine the real pipe vibration audio signal with the simulated pipe vibration audio signal to construct a pipe vibration audio signal dataset, wherein the pipe vibration audio signal dataset includes a plurality of pipe vibration audio signals with a certain audio duration, and each pipe vibration audio signal includes a plurality of pipe vibration audio signal frames.

[0073] In some embodiments, the real pipe vibration audio signal in the water supply network is collected, a proper number of noise recorders and water sound sensors are arranged in the water supply pipe of the target area, the pipe vibration audio signals of the water supply pipe leakage of the ductile iron pipe and the PE pipe are collected in the target area under the condition of a relatively quiet period (after 22:00 every day), the pressure regulating valve is used for pressure regulation on the main pipe during the process of collecting the real pipe vibration audio signal, and the pressure regulating range is 0.15 Mpa to 0.32 Mpa. Each time, a small amplitude pressure regulation (for example, 0.05 MPa) is performed to simulate different sizes of leakage. The duration of each pipe vibration audio signal is 5.46 seconds, the frequency of the pipe vibration audio signal is 8000 Hz, and finally the real pipe vibration audio signal is saved in the waveform audio file format (WAV). These real pipe vibration audio signals will be marked as having leakage or no leakage. In this embodiment, a total of 788 real pipe vibration audio signals are collected, of which 612 pipe vibration audio signals are marked as having leakage, and 176 pipe vibration audio signals are marked as having no leakage.

[0074] In some embodiments, all pipe vibration audio signals are cropped, and in order to improve the operation speed and save the calculation time cost, a 1-second segment of all pipe vibration audio signals is reserved. In order to avoid the problem of data imbalance, only the small sample problem is focused on, and finally 500 pipe vibration audio signals of the water supply pipe in the leakage state and 500 pipe vibration audio signals of the water supply pipe in the non-leakage state are selected.

[0075] In some embodiments, the pipeline vibration audio signal is denoised using a wavelet denoising method. The wavelet denoising method decomposes the pipeline vibration audio signal into coefficients of different scales through wavelet transform, removes noise from the coefficients through threshold processing based on the characteristic differences between the pipeline vibration audio signal and noise at each scale, and finally reconstructs the signal through inverse wavelet transform. The calculation formula of the wavelet transform is represented as:

[0076]

[0077] where s represents the original pipeline vibration audio signal. is an approximation wavelet coefficient, which represents the coefficient at scale j0, and represents the low-frequency part (smooth part) of the pipeline vibration audio signal, which usually retains a large amplitude signal. j (k) is a detail wavelet coefficient, which represents the coefficient at scale j, and represents the high-frequency part (detail part) of the signal, which usually contains noise. is a scaling function, also known as a parent wavelet function, which controls the smooth part of the signal, and is a wavelet function based on a specific scale j0 and position k. j，k (t) is a wavelet function based on scale j and position k, which is used to extract the high-frequency detail part of the signal.

[0078] After wavelet decomposition, the wavelet coefficient signal related part has a large amplitude, while the noise related part has a small amplitude. Therefore, small values in the wavelet coefficient are considered as noise and removed or reduced through a soft threshold method, and the calculation formula is represented as:

[0079]

[0080] where denotes the denoised detail wavelet coefficient, which is the wavelet coefficient after soft threshold processing; sign(d j (k)) represents the sign of d j (k) that returns +1 or -1, which is used to maintain the direction of the wavelet coefficient |d j (k) | represents the absolute value of d j (k), which represents the amplitude of the wavelet coefficient, and the threshold value represents the threshold value used to distinguish between signal and noise, which can be set by a person skilled in the art according to the characteristics of the noise.

[0081] Finally, the denoised wavelet coefficient is reconstructed through inverse wavelet transform, and the calculation formula of the inverse wavelet transform is represented as:

[0082]

[0083] where s denoiseddenotes the de-noised pipeline vibration audio signal, which is reconstructed from the de-noised coefficients by inverse wavelet transform. denotes the de-noised approximation wavelet coefficients, which usually do not change much during the de-noising process because they represent the low frequency part of the signal. denotes the de-noised detail wavelet coefficients, which are obtained by removing noise through the soft threshold method. denotes the scaling function, also known as the parent wavelet function, which controls the smooth part of the signal, and is a wavelet function based on a specific scale j0 and position k; ψ j，k (t) is a wavelet function based on scale j and position k, which is used to extract the high frequency detail part of the signal.

[0084] In some embodiments, the real pipeline vibration audio signal is input into a preset generative adversarial network, and different rounds of output are performed to generate a simulated pipeline vibration audio signal. The real pipeline vibration audio signal and the simulated pipeline vibration audio signal are combined to construct a pipeline vibration audio signal dataset for data augmentation and expansion of the database. The basic network architecture of the generative adversarial network includes a WaveGAN network, which is a generative adversarial network specially used for generating one-dimensional time series data (such as audio signals) and can be implemented through Python;

[0085] The generative adversarial network (GAN) is composed of a generator and a discriminator. The generator generates new samples by learning the distribution of real data, and the discriminator is responsible for distinguishing real samples from generated samples. Through continuous adversarial training, the generator can generate more and more realistic samples. The steps for training the generative adversarial network include:

[0086] Step S101, obtain key parameters, initialize the network structure of the generator and the discriminator in the preset generative adversarial network, initialize the network weight, and use the Adam optimizer, wherein the key parameters include the dataset path, the batch size, the random noise dimension, the training round number, the learning rate, and the sampling rate;

[0087] Step S102, obtain the path of all real pipeline vibration audio signals in the audio data set, create a data set object, and generate data loaders for the generator and the discriminator, respectively;

[0088] Step S103, in each training round, the data loader is traversed for training, the discrimination results of the real pipeline vibration audio signal and the simulated pipeline vibration audio signal are calculated, the gradient penalty term is calculated to ensure the continuity of the discriminator, the parameters of the discriminator are updated to minimize the discrimination ability of the discriminator to the generated signal, and the loss value of the discriminator is recorded;

[0089] Step S104, the simulated pipeline vibration audio signal is generated and the output of the discriminator is calculated, the parameters of the generator are updated to maximize the discrimination result of the discriminator to the simulated pipeline vibration audio signal, and the loss value of the generator is recorded;

[0090] Step S105, every certain training round, the simulated pipeline vibration audio signal is saved to the specified directory until the generator can generate high-quality simulated pipeline vibration audio signal, the trained generator is saved, the trained generator is used to generate new simulated pipeline vibration audio signal, and the simulated pipeline vibration audio signal is combined with the real pipeline vibration audio signal for data enhancement.

[0091] In some embodiments, the step of determining a plurality of pipeline vibration audio signal frames comprises:

[0092] Step S1001, obtaining the audio duration, the frame duration, the overlap duration and the sampling rate of each pipeline vibration audio signal;

[0093] Step S1002, calculating a first product between the audio duration and the sampling rate, calculating a second product between the frame duration and the sampling rate, and calculating a first difference between the first product and the second product;

[0094] Step S1003, calculating a second difference between the frame duration and the overlap duration to determine a skip duration, and calculating a third product between the skip duration and the sampling rate;

[0095] Step S1004, determining the frame number of each pipeline vibration audio signal according to a first ratio between the first difference and the third product, and framing the pipeline vibration audio signal according to the frame number to determine a plurality of pipeline vibration audio signal frames in each pipeline vibration audio signal.

[0096] Specifically, the pipeline vibration audio signal of a certain audio duration is framed to determine a plurality of pipeline vibration audio signal frames, wherein the calculation formula for determining the frame number of the pipeline vibration audio signal is:

[0097]

[0098] wherein, Frames represents the number of frames of each pipeline vibration audio signal, nRows represents the number of sampling points contained in the input pipeline vibration audio signal, that is, the product of the audio duration (s) and the sampling rate (Hz); winLen represents the number of sampling points contained in each frame of the pipeline vibration audio signal, which is expressed as the product of the frame duration and the sampling rate; hopLen is the number of sampling points contained in each hop of the pipeline vibration audio signal, and the hop length is the difference between the frame duration and the overlap duration, which is expressed as: hop length = frame duration - overlap duration.

[0099] In some embodiments, the audio duration can be 1s, the sampling rate can be 8000Hz, which represents the number of sampling points contained in the pipeline vibration audio signal per second; the frame duration can be 30ms, that is, 0.03s; and the overlap duration can be 12.5ms, that is, 0.0125.

[0100] Based on the above calculation formula and the above data, the number of frames of the pipeline vibration audio signal can be calculated as 56, that is, the time step of the bidirectional long short-term memory network (BiLSTM) in the CNN-BiLSTM hybrid neural network model is 56.

[0101] Step S20, after sequentially processing the pipeline vibration audio signal using fast Fourier transform and a mel filter bank, the mel cepstral coefficients corresponding to each frame of the pipeline vibration audio signal are extracted using discrete cosine transform to construct a mel cepstral coefficient feature matrix of the pipeline vibration audio signal frame;

[0102] Step S30, a first-order difference matrix of the mel cepstral coefficients is calculated, and the mel cepstral coefficient feature matrix and the first-order difference matrix are fused to determine a feature fusion matrix of each pipeline vibration audio signal frame;

[0103] The real pipe vibration audio signal in the water supply network is collected, the real pipe vibration audio signal is input into a preset generative adversarial network, and the output of different rounds is used to generate a simulation pipe vibration audio signal, the real pipe vibration audio signal and the simulation pipe vibration audio signal are combined to construct a pipe vibration audio signal dataset, then the pipe vibration audio signal is processed in sequence by using fast Fourier transform and a mel filter bank, the mel cepstrum coefficients corresponding to each pipe vibration audio signal frame in the pipe vibration audio signal are extracted by using discrete cosine transform to construct a mel cepstrum coefficient feature matrix of the pipe vibration audio signal frame, a first difference matrix of the mel cepstrum coefficients is calculated and determined, and the mel cepstrum coefficient feature matrix and the first difference matrix are fused to determine a feature fusion matrix of each pipe vibration audio signal frame; wherein the feature fusion matrix represents the joint features of the pipe vibration audio signal in the time domain and the frequency domain, and not only contains the static frequency spectrum information of the signal, but also combines the dynamic characteristics of the signal changing with time.

[0104] In some embodiments, referring to Figure 2 , Figure 2 is a flowchart for extracting the MFCC feature matrix, before the pipe vibration audio signal is processed in sequence by using fast Fourier transform and a mel filter bank, the pipe vibration audio signal is pre-emphasized and windowed, which includes:

[0105] S301, pre-emphasize the pipe vibration audio signal, enhance the high-frequency component through a high-pass filter, compensate for the high-frequency attenuation in the speech signal caused by the sound production system and the propagation medium, and make the frequency spectrum flat, wherein the calculation formula of the pre-emphasis is:

[0106] y(n) = x(n) - αx(n-1),

[0107] wherein x(n) is the sampling value of the original pipe vibration audio signal at time point n, y(n) is the pre-emphasized pipe vibration audio signal, and α is the pre-emphasis coefficient, generally α ∈ [0.95, 0.97], and in this embodiment, α = 0.97.

[0108] S303, window the pipe vibration audio signal to reduce frame edge discontinuity and suppress spectral leakage; the Hamming window is a commonly used window function and is widely used in the field of signal processing, especially in the processing of pipe vibration audio signals, and the calculation formula of the Hamming window is:

[0109]

[0110] Wherein, w(n) represents the window function value of the nth sampling point under the window function. Windowing is to reduce the edge effect of the signal when framing, and to prevent spectral leakage. The cosine function is applied to each sampling point to generate a smooth window function curve. The window function reduces the sudden change of the signal at the frame edge and reduces spectral leakage; n represents the sampling point index in the current frame, and the value range is from 0 to N-1, n corresponds to each sampling point in the current frame, and the value of n is discrete, representing different positions in the frame; N is the frame length sample number, which represents the number of sampling points contained in each pipe vibration audio signal frame. For example, if the framing length is 30ms and the sampling rate is 8000Hz, then N=240 (number of sampling points).

[0111] In some embodiments, after the pipe vibration audio signal is processed in turn by the fast Fourier transform and the mel filter bank, the step of extracting the mel cepstral coefficient corresponding to each pipe vibration audio signal frame in the pipe vibration audio signal by using the discrete cosine transform to construct the mel cepstral coefficient feature matrix of the pipe vibration audio signal frame includes:

[0112] Step S201, performing fast Fourier transform on the pipe vibration audio signal to convert the time domain signal of each pipe vibration audio signal frame in the pipe vibration audio signal into frequency domain data to obtain spectral information, wherein the spectral information reflects the energy distribution of the pipe vibration audio signal at different frequencies;

[0113] Step S202, filtering the pipe vibration audio signal according to the spectral information by using each mel filter in the mel filter bank to determine the output energy on each mel filter channel, performing logarithmic operation on the output energy on each mel filter channel to obtain the logarithmic energy of each mel filter channel, and converting it into logarithmic mel spectrum;

[0114] Step S203, extracting the mel cepstral coefficient according to the logarithmic energy of each mel filter channel by using the discrete cosine transform to obtain the mel cepstral coefficient feature matrix corresponding to each pipe vibration audio signal frame in the pipe vibration audio signal.

[0115] Specifically, step S2001, performing fast Fourier transform (FFT) on the pipe vibration audio signal to convert the time domain signal into frequency domain and obtain spectral information; the formula of fast Fourier transform is as follows:

[0116]

[0117] Wherein, X(k) represents the complex frequency domain signal after fast Fourier transform (FFT); |X(k)| 2represents the energy spectrum, which measures the energy distribution of the signal at different frequencies; represents the complex exponential, represents the kernel function of the fast Fourier transform (FFT). x(n) represents the time-domain signal; w(n) represents the window function, which is usually the Hamming window or the Hanning window, used to reduce the spectral leakage caused by signal discontinuity; N FFT represents the number of FFT points, which is usually a power of 2, such as 1024, 2048, etc.

[0118] Step S2002, filter the pipeline vibration audio signal using a mel filter bank, logarithmically operate the energy on each mel filter channel to obtain the logarithmic energy, and convert it into a log mel spectrum;

[0119] Specifically, M triangular filters are uniformly divided in the mel domain, where M can be 40, etc., the mel frequency is converted back to linear frequency to determine the center frequency f c , the start frequency f l , and the cutoff frequency f h of each mel filter.

[0120]

[0121] where f represents the actual frequency, with the unit of Hz; F Mel represents the mel frequency, with the unit of mel.

[0122] Further, the calculation formula of the weight H m (k) of the mth mel filter at the frequency point k is as follows:

[0123]

[0124] where f c represents the center frequency of the mel filter; f l represents the start frequency of the mel filter, f h represents the cutoff frequency of the mel filter; H m (k) represents the weight of the mth mel filter at the frequency point k.

[0125] Further, the output energy calculation formula of each filter channel is as follows:

[0126]

[0127] where E(m) represents the output energy of the mth mel filter channel; X(k) represents the complex frequency domain signal after the fast Fourier transform (FFT); H m (k) represents the weight of the mth mel filter at the frequency point k.

[0128] Subsequently, the output energy of each mel filter channel is logarithmized log(E(m) to obtain the logarithmic energy of each mel filter channel, so as to compress the dynamic range of the pipeline vibration audio signal and make it consistent with the perceptual characteristics of the logarithmic response of the human ear.

[0129] Step S2003, the mel frequency cepstral coefficient (MFCC coefficient) is extracted by using the discrete cosine transform, and a mel frequency cepstral coefficient (MFCC coefficient) feature matrix is obtained, wherein the calculation formula of the discrete cosine transform is represented as:

[0130]

[0131] Wherein C(n) represents the nth mel frequency cepstral coefficient (MFCC coefficient); E(m) represents the output energy of the mth mel filter channel; M represents the number of mel filter channels; L represents the number of MFCC coefficients, which can be 13, representing a 13-dimensional mel frequency cepstral coefficient feature matrix.

[0132] The feature parameter dimension output by the transformed mel frequency cepstral coefficient feature matrix is 13, including the logarithmic energy of the pipeline vibration audio signal frame and 12 mel frequency cepstral coefficients, and the first-order difference matrix of the mel frequency cepstral coefficient (MFCC coefficient) mentioned below is the same.

[0133] Step S2004, the first-order difference matrix of the mel frequency cepstral coefficient (MFCC coefficient) is calculated, wherein the calculation formula of the first-order difference matrix of the mel frequency cepstral coefficient (MFCC coefficient) is represented as:

[0134] ΔC(j) = C(j+1) - C(j),

[0135] Wherein j is the number of frames of the pipeline vibration audio signal; C(j) represents the mel frequency cepstral coefficient corresponding to the jth pipeline vibration audio signal frame; and ΔC(j) is the first-order difference matrix of the mel frequency cepstral coefficient corresponding to the jth pipeline vibration audio signal frame.

[0136] Step S2005, the mel frequency cepstral coefficient feature matrix and the first-order difference matrix of the mel frequency cepstral coefficient obtained are fused to determine the feature fusion matrix of each pipeline vibration audio signal frame; wherein the step of feature fusion comprises:

[0137] The obtained mel-frequency cepstral coefficient feature matrix and the first-order difference matrix ΔMFCC of the mel-frequency cepstral coefficient are transversely spliced, that is, the two feature matrices with a dimension of 13 are spliced into a feature matrix with a dimension of 26 to determine the feature fusion matrix of each pipe vibration audio signal frame, and then the feature fusion matrix of each pipe vibration audio signal frame in the pipe vibration audio signal is input into the CNN-BiLSTM hybrid neural network model for training. After being trained to a convergent state, a water supply network leakage detection model trained to the convergent state is determined, which is used for leakage detection in the water supply pipeline in the water supply network.

[0138] In step S40, the CNN-BiLSTM hybrid neural network model is trained by taking the feature fusion matrix of each pipe vibration audio signal frame in the pipe vibration audio signal as a training sample and taking the leakage state corresponding to the pipe vibration audio signal as a sample label, so as to determine the water supply network leakage detection model trained to the convergent state.

[0139] In step S40, the CNN-BiLSTM hybrid neural network model is trained by taking the feature fusion matrix of each pipe vibration audio signal frame in the pipe vibration audio signal as a training sample and taking the leakage state corresponding to the pipe vibration audio signal as a sample label, so as to determine the water supply network leakage detection model trained to the convergent state.

[0140] In some embodiments, the step of training the preset CNN-BiLSTM hybrid neural network model by taking the feature fusion matrix of each pipe vibration audio signal frame in the pipe vibration audio signal as a training sample and taking the leakage state corresponding to the pipe vibration audio signal as a sample label to determine the water supply network leakage detection model trained to the convergent state includes:

[0141] In step S401, a preset Bayesian optimization algorithm is used to optimize the model hyperparameters of the CNN-BiLSTM hybrid neural network model to determine the optimal hyperparameter combination of the CNN-BiLSTM hybrid neural network model, wherein the model hyperparameters include a learning rate, a batch size, and an LSTM hidden unit number.

[0142] Step S402, the optimal hyperparameter combination CNN-BiLSTM hybrid neural network model as the water supply network leakage detection model.

[0143] The hybrid neural network model in this embodiment is composed of two layers of CNN (convolutional neural network) and one layer of BiLSTM (bidirectional long short-term memory network) in series, connected by a flattening layer. Among them, CNN (convolutional neural network) is used to extract the local spatial features of MFCC related features, and BiLSTM (bidirectional long short-term memory network) is used to capture the time sequence characteristics of the CNN (convolutional neural network) output features. For specific CNN-BiLSTM hybrid neural network model structure and related parameters, please refer to Table 1, which is the CNN-BiLSTM hybrid neural network model structure parameter table.

[0144] Table 1 CNN-Bi LSTM hybrid neural network model structure parameter table

[0145]

[0146] Build a CNN (convolutional neural network), which is composed of 2 convolutional layers and 2 pooling layers. Specifically, the feature fusion matrix of each pipeline vibration audio signal frame in the pipeline vibration audio signal is extracted by a CNN (convolutional neural network) composed of two convolutional layers, ReLU activation function and maximum pooling layer, wherein the feature extraction steps of the convolutional layer are as shown in Figure 3

[0147] The output of the convolutional layer can be represented by the following formula:

[0148]

[0149] Where I is the input feature matrix, K is the convolution kernel, and b is the bias term. In this embodiment, the size of the convolution kernel 1 is 5, the number of convolution kernel 1 is 32, the size of the convolution kernel 2 is 5, and the number of convolution kernel 2 is 64.

[0150] The activation function uses the ReLU activation function, whose formula is as follows:

[0151] O ReLU = max (0, O BN ),

[0152] ReLU function will set the negative values in the output feature matrix to 0 and retain the positive values.

[0153] The maximum pooling layer down-samples the feature matrix, and the formula is:

[0154] O pool (i, j) = O ReLU = max[O​ReLU (i+k, j+l)] ; (k, l) e window,

[0155] wherein, window is the size of the pooling window (2x2); (i, j) is the position of the output feature matrix; (k, l) is the position within the pooling window.

[0156] In a further embodiment, a flattening layer is built to expand the feature matrix extracted by the two-layer CNN network into a one-dimensional vector. The feature matrix output by the convolutional layer is converted into a one-dimensional vector by the flattening layer, which is convenient for being sent to the subsequent BiLSTM (bidirectional long short-term memory network) for processing.

[0157] In a further embodiment, please refer to Figure 4 , a BiLSTM (bidirectional long short-term memory network) is built to process the feature vector after the CNN (convolutional neural network) feature extraction and flattening. The BiLSTM (bidirectional long short-term memory network) is used to process the flattened feature vector and capture the time sequence characteristics of the signal. This layer processes the time dependence of the audio signal through the bidirectional LSTM, thereby capturing the forward and reverse information of the signal at the same time. In this way, the model can better understand the time dynamic characteristics of the missing signal;

[0158] Specifically, please refer to Table 1, which contains the structure parameters of the BiLSTM (bidirectional long short-term memory network). Generally speaking, the LSTM algorithm uses three gates (forget gate, input gate and output gate) to adjust the output state of the current unit and the previous unit.

[0159] Specifically, please refer to Figure 5 , wherein, h t-1 represents the state at t-1, x t represents the input information. The forget gate adjusts the last time input data retained by the hidden layer node, and calculates according to the current hidden layer input and the last time state to determine whether the current time state needs to be discarded or retained, and outputs by the sigmoid activation function, wherein, the formula of the sigmoid activation function is as follows:

[0160]

[0161] In an LSTM unit, x t and h t-1 are input information, x t and h t-1 output 0 or 1 values through the forget gate, wherein 0 represents discard and 1 represents retain, wherein the expression of the forget gate is:

[0162] f t = σ (W f · [ht-1 , x t ]+b f ),

[0163] where σ denotes sigmoid activation function, W f denotes forget gate weight, b f denotes forget gate bias.

[0164] The input gate adjusts the input state of the current hidden layer memory cell, and judges whether the input data is updated with the current memory cell state or not, wherein the expression of the input gate is:

[0165] i t = σ (W f · [h t-1 , x t ]+b i ),

[0166] where σ denotes sigmoid activation function, W f denotes input gate weight; b i denotes input gate bias, i t denotes the output value of the input gate.

[0167] x t and h t-1 first use the sigmoid activation function to judge whether the input data is saved or updated, and then x t and h t-1 obtain the new candidate memory cell value C′ t through the tanh function, and the expression is:

[0168] C′ t = tanh (W c · [h t-1 , x t ]+b c ),

[0169] where tanh denotes the activation function; W c denotes the candidate memory cell weight; b c denotes the candidate memory cell bias.

[0170] The output value at t-1 time is multiplied by the output value of the forget gate, and the candidate memory cell value is multiplied by the output value of the input gate, and the two are added to update the current time state to obtain the current time memory cell, wherein the expression of the forget gate is:

[0171] C t = f t *C t-1 +i t *C′ t ;

[0172] wherein f t represents the forget gate output, C t-1 represents the cell output value at time t-1, i t represents the input gate output value, C' t represents the candidate memory cell value.

[0173] The output gate adjusts the output state of the current hidden layer node, and determines whether the data is output to the next unit hidden layer or the output layer. According to the current time state updated by the forget gate and the input gate, the output gate is determined by x t and h t-1 The output signal feature is identified, and the output gate information is determined by a sigmoid function, and the expression is as follows:

[0174] O t = σ(W o · [h t-1 , x t ] + b o ),

[0175] wherein σ represents a sigmoid activation function, W o represents an output gate weight, and b o represents an output gate bias.

[0176] The output y t of the LSTM unit finally obtained is output by a tanh function, and the expression is as follows:

[0177] y t = O t · tanh(C t )

[0178] In this embodiment, a bidirectional LSTM (i.e., BiLSTM) is used, that is, two independent LSTMs are run at each time step, one from the beginning to the end of the sequence (forward LSTM), and the other from the end to the beginning of the sequence (backward LSTM). As can be seen from the above embodiment, the BILSTM time step is 56, the forward LSTM sequentially processes the input sequence from time step 1 to 56, and finally outputs the hidden state of the last time step (step 56) (dimension [batch size, hidden unit number]); at the same time, the backward LSTM processes the same sequence in reverse order from time step 56 to 1, and finally outputs the hidden state of the last time step of its processing flow (corresponding to the first step of the original sequence) (dimension [batch size, hidden unit number]). By concatenating the final outputs of the two direction LSTMs in the feature dimension, the merged bidirectional feature representation [forward output, backward output] is obtained, and the dimension is [batch size, 2xhidden unit number]. This structure enables the model to capture both forward and backward dependencies of the sequence.

[0179] In a further embodiment, a Softmax output layer is built; the task of this embodiment is a binary classification problem (i.e., determining whether the pipeline vibration audio signal is a pipeline vibration audio signal of a water supply pipeline in a leakage state), and after the Softmax layer outputs the probability that the sample belongs to the positive class (i.e., leakage occurs), the classification output layer will calculate the binary cross-entropy loss, which is calculated as follows:

[0180] Loss = -(y·lg(p) + (1-y)·lg(1-p))

[0181] In the formula, y represents the true label (0 or 1), where 0 represents no leakage state; 1 represents a leakage state; and p represents the probability output by the Softmax layer.

[0182] The classification layer finally converts the probability p output by the Softmax layer into the final classification result and provides training feedback. If p≥0.5, it is predicted as a positive class, indicating that the water supply pipeline is in a leakage state, otherwise it is predicted as a negative class, indicating that the water supply pipeline is in a non-leakage state.

[0183] Further, Bayesian optimization is used for neural network hyperparameter selection and optimization;

[0184] Bayesian optimization is particularly suitable for solving complex objective functions with high computational cost, non-convex characteristics or difficulty in derivation. This algorithm can effectively explore the parameter space by constructing a proxy model and collecting functions, and is mainly used in the field of deep learning for automatic tuning of neural network hyperparameters, significantly improving the efficiency of parameter optimization.

[0185] In this embodiment, Bayesian optimization is used to optimize learning rate, batch size, LSTM hidden unit number (HiddenUnits) and other hyperparameters. By constructing a probability model of the objective function and balancing exploration and utilization, Bayesian optimization can find the optimal combination of hyperparameters in fewer iterations. In MATLAB, Bayesian optimization can be implemented using the bayesopt function, using accuracy or F1 score as the objective function to improve the performance of the CNN-BiLSTM model.

[0186] Further, during training, the accuracy, precision, recall, and F1 score are used to evaluate the detection accuracy and performance of the model to ensure that the model can accurately identify the leakage state;

[0187] The performance evaluation index formula is as follows:

[0188]

[0189]

[0190]

[0191] Wherein, TP is true positive; TN is true negative; FP is false positive; FN is false negative.

[0192] Step S50, input the to-be-identified pipeline vibration audio signal into the water supply network leakage detection model trained to the convergent state, to determine whether the pipeline corresponding to the to-be-identified pipeline vibration audio signal is in a leakage state or a non-leakage state, to complete the water supply network leakage identification based on data enhancement and hybrid neural network architecture.

[0193] With the feature fusion matrix of each pipeline vibration audio signal frame in the pipeline vibration audio signal as a training sample, and the leakage state corresponding to the pipeline vibration audio signal as a sample label, a preset CNN-BiLSTM hybrid neural network model is trained to determine the water supply network leakage detection model trained to the convergent state, and then the to-be-identified pipeline vibration audio signal is input into the water supply network leakage detection model trained to the convergent state, to determine whether the pipeline corresponding to the to-be-identified pipeline vibration audio signal is in a leakage state or a non-leakage state, to complete the water supply network leakage identification based on data enhancement and hybrid neural network architecture.

[0194] From the above embodiments, compared with the prior art, the present application is aimed at the problems of the prior art, such as the scarcity of pipeline vibration audio signal labeling data of water supply pipelines in a leakage state in the water supply network, the low recognition accuracy, poor robustness, and weak generalization ability of existing machine learning methods in identifying water supply network leakage, etc. The present application includes but is not limited to the following beneficial effects:

[0195] Firstly, the traditional leakage detection method relies on a large amount of labeled data, while the pipeline vibration audio signal collection of water supply network leakage often has a sample scarcity situation. Through the generative adversarial network (WaveGAN), the training data set can be effectively expanded to solve the data scarcity problem, thereby improving the effect of the training model. By synthesizing real and simulated pipeline vibration audio signals, the robustness of the model can be enhanced in different signal and noise environments, ensuring the adaptability of the leakage detection in diversified environments.

[0196] Secondly, the existing technology often relies on a single acoustic feature to identify leakage signals, but often ignores the influence of dynamic features. By fusing the mel-frequency cepstral coefficient with its first-order difference feature matrix, the present application extracts more rich time and frequency domain features, which can more accurately reflect the change rule of the leakage signal and enhance the model's ability to distinguish leakage signals.

[0197] Thirdly, the Mel-frequency cepstral coefficient is used to process the spectral features, and BiLSTM is combined to capture the time sequence characteristics, so that the model can consider the instantaneous spectral features and time variation rules of the pipeline vibration audio signals of the water supply pipeline in the leakage state at the same time, thereby improving the accuracy of leakage identification.

[0198] Fourthly, the application is based on the CNN-Bi LSTM hybrid architecture. Convolutional neural network (CNN) is good at extracting spatial features, especially in processing high-dimensional data with strong feature learning ability. The bidirectional long short-term memory network (Bi LSTM) can fully utilize the bidirectional information of time series and capture the long-time dependence of signals. By combining the advantages of the two, the local and global feature information can be considered when processing the vibration audio signals of the water supply network leakage, and the recognition effect is improved. After extracting the spatial features by CNN, Bi LSTM further excavates the time sequence information, and finally realizes the high-precision output of the leakage identification model. The hybrid architecture introduces a more complex network structure based on the traditional method, making the leakage identification more accurate.

[0199] Fifthly, the application significantly improves the automation and intelligence level of leakage detection, reduces the vulnerability of traditional manual experience dependence. Especially based on the automatic detection of neural network model, the leakage state of the pipeline can be identified in real time, avoiding the leakage being ignored or misjudged. Compared with the traditional manual detection method, the technology can identify the leakage without the need for a large amount of manual intervention, reducing the labor cost and improving the detection efficiency.

[0200] Sixthly, by accurately identifying the leakage state, the application can perform pipeline maintenance and repair in advance to avoid the occurrence of water pipe burst and other emergencies, thereby improving the stability and safety of the water supply system. Especially in large-scale water supply network, systematic leakage detection can effectively prevent water supply interruption and the resulting social and economic losses.

[0201] Further, the application effectively solves the problems of data scarcity, insufficient feature extraction, and low model accuracy in water supply network leakage detection by combining data enhancement with hybrid neural network architecture, significantly improving the precision, robustness, and generalization ability of leakage detection. This technical solution not only improves the efficiency of water supply network leakage detection and reduces the economic and environmental losses caused by leakage, but also further promotes the development of intelligent management of water supply systems, and has a wide application prospect.

[0202] Please refer to Figure 6, provided by one of the purposes of the application, a water supply network leakage identification device based on data enhancement and hybrid neural network architecture, comprising an audio data set construction module 1100, a mel frequency cepstral coefficient extraction module 1200, a feature fusion matrix construction module 1300, a detection model training module 1400 and a water supply network leakage identification module 1500. Among them, the audio data set construction module 1100 is set to collect the real pipe vibration audio signal in the water supply network, input the real pipe vibration audio signal into the preset generative adversarial network, and output through different rounds to generate the simulation pipe vibration audio signal, combine the real pipe vibration audio signal with the simulation pipe vibration audio signal to construct the pipe vibration audio signal data set, wherein the pipe vibration audio signal data set includes a plurality of pipe vibration audio signals with a certain audio time length, and each pipe vibration audio signal includes a plurality of pipe vibration audio signal frames; the mel frequency cepstral coefficient extraction module 1200 is set to sequentially process the pipe vibration audio signal by using fast Fourier transform and mel filter bank, and then extract the mel frequency cepstral coefficient corresponding to each pipe vibration audio signal frame in the pipe vibration audio signal by using discrete cosine transform to construct the mel frequency cepstral coefficient feature matrix of the pipe vibration audio signal frame; the feature fusion matrix construction module 1300 is set to calculate and determine the first difference matrix of the mel frequency cepstral coefficient, and fuse the mel frequency cepstral coefficient feature matrix and the first difference matrix to determine the feature fusion matrix of each pipe vibration audio signal frame; the detection model training module 1400 is set to take the feature fusion matrix of each pipe vibration audio signal frame in the pipe vibration audio signal as a training sample, take the leakage state corresponding to the pipe vibration audio signal as a sample label, train a preset CNN-BiLSTM hybrid neural network model to determine a water supply network leakage detection model trained to a convergence state; the water supply network leakage identification module 1500 is set to input the pipe vibration audio signal to be identified into the water supply network leakage detection model trained to a convergence state to determine whether the pipe corresponding to the pipe vibration audio signal to be identified is in a leakage state or a non-leakage state, so as to complete the water supply network leakage identification based on data enhancement and hybrid neural network architecture.

[0203] On the basis of any embodiment of the present application, please refer to Figure 7 Another embodiment of the present application also provides an electronic device, which can be realized by a computer device, such as Figure 7As shown, the internal structure diagram of the computer device is shown. The computer device includes a processor, a computer readable storage medium, a memory and a network interface connected by a system bus. Among them, the computer readable storage medium of the computer device stores an operating system, a database and computer readable instructions, the database can store control information sequence, and the computer readable instructions are executed by the processor to enable the processor to implement a water supply network leakage identification method based on data enhancement and hybrid neural network architecture. The processor of the computer device is used to provide computing and control capability to support the operation of the entire computer device. The memory of the computer device can store computer readable instructions, and the computer readable instructions are executed by the processor to enable the processor to execute the water supply network leakage identification method based on data enhancement and hybrid neural network architecture of the present application. The network interface of the computer device is used to connect and communicate with the terminal. Those skilled in the art can understand, Figure 7 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0204] The processor in the embodiment is used to execute the specific functions of each module in Figure 6 The memory stores the program codes and various data required for executing the above-mentioned modules. The network interface is used for data transmission between the user terminal or the server. The memory in the embodiment stores the program codes and data required for executing all modules in the water supply network leakage identification device based on data enhancement and hybrid neural network architecture of the present application, and the server can call the program codes and data of the server to execute the functions of all modules.

[0205] The present application also provides a storage medium storing computer readable instructions, which are executed by one or more processors to enable the one or more processors to execute the steps of the water supply network leakage identification method based on data enhancement and hybrid neural network architecture described in any embodiment of the present application.

[0206] The present application also provides a computer program product including computer programs / instructions, which are executed by one or more processors to implement the steps of the water supply network leakage identification method based on data enhancement and hybrid neural network architecture described in any embodiment of the present application.

[0207] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the application can be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer readable storage medium. When the program is executed, the processes of the above-mentioned embodiments of the methods can be included. The storage medium can be a computer readable storage medium such as a magnetic disc, an optical disc, a read-only memory (ROM), or a random access memory (RAM).

[0208] The above only describes some embodiments of the application. It should be pointed out that those skilled in the art can make some improvements and refinements without departing from the principles of the application. These improvements and refinements should also be considered as the protection scope of the application.

Claims

1. A water distribution network leakage identification method based on data augmentation and hybrid neural network architecture, characterized in that, The method comprises the following steps: Collecting real pipe vibration audio signals in a water supply network, inputting the real pipe vibration audio signals into a preset generative adversarial network, and generating simulation pipe vibration audio signals through different rounds of output, combining the real pipe vibration audio signals with the simulation pipe vibration audio signals to construct a pipe vibration audio signal dataset, wherein the pipe vibration audio signal dataset comprises a plurality of pipe vibration audio signals with a certain audio duration, and each pipe vibration audio signal comprises a plurality of pipe vibration audio signal frames; After the pipe vibration audio signals are processed in sequence by using fast Fourier transform and a mel filter bank, the mel frequency cepstral coefficients corresponding to each pipe vibration audio signal frame in the pipe vibration audio signal are extracted by using discrete cosine transform to construct a mel frequency cepstral coefficient feature matrix of the pipe vibration audio signal frame; A first difference matrix of the mel frequency cepstral coefficients is calculated, and the mel frequency cepstral coefficient feature matrix and the first difference matrix are fused to determine a feature fusion matrix of each pipe vibration audio signal frame; The feature fusion matrix of each pipe vibration audio signal frame in the pipe vibration audio signal is taken as a training sample, and the corresponding leakage state of the pipe vibration audio signal is taken as a sample label, and a preset CNN-BiLSTM hybrid neural network model is trained to determine a water supply network leakage detection model trained to a convergent state; The pipe vibration audio signal to be identified is input into the water supply network leakage detection model trained to the convergent state to determine whether the pipe corresponding to the pipe vibration audio signal to be identified is in a leakage state or a non-leakage state, thereby completing water supply network leakage identification based on data enhancement and a hybrid neural network architecture.

2. The data-augmentation and hybrid neural network architecture based water distribution network leakage identification method according to claim 1, wherein, The step of determining a plurality of pipe vibration audio signal frames comprises: obtaining the audio duration, the frame duration, the overlap duration, and the sampling rate of each pipe vibration audio signal; calculating a first product between the audio duration and the sampling rate, calculating a second product between the frame duration and the sampling rate, and calculating a first difference between the first product and the second product; calculating a second difference between the frame duration and the overlap duration to determine a skip duration, and calculating a third product between the skip duration and the sampling rate; determining the number of frames of each pipe vibration audio signal according to a first ratio between the first difference and the third product, and framing the pipe vibration audio signal according to the number of frames to determine a plurality of pipe vibration audio signal frames in each pipe vibration audio signal.

3. The data-augmentation and hybrid neural network architecture based water distribution network leakage identification method of claim 1, wherein, The step of training the generative adversarial network comprises: obtaining key parameters, initializing the network structure of the generator and the discriminator in the preset generative adversarial network, initializing the network weight, and using an Adam optimizer, wherein the key parameters comprise a dataset path, a batch size, a random noise dimension, a number of training rounds, a learning rate, and a sampling rate; Obtaining the paths of all real pipeline vibration audio signals in the audio data set, creating a data set object, and generating data loaders for the generator and discriminator, respectively; In each training round, the data loader is traversed for training, the discrimination results of the real pipeline vibration audio signals and the simulated pipeline vibration audio signals are calculated, the gradient penalty term is calculated to ensure the continuity of the discriminator, the parameters of the discriminator are updated to minimize the discrimination ability of the discriminator on the generated signals, and the loss value of the discriminator is recorded; The simulated pipeline vibration audio signal is generated and the output of the discriminator is calculated, the parameters of the generator are updated to maximize the discrimination result of the discriminator on the simulated pipeline vibration audio signal, and the loss value of the generator is recorded; Every certain training round, simulate the pipeline vibration audio signal and save it to a specified directory until the generator can generate high-quality simulated pipeline vibration audio signals, save the trained generator, use the trained generator to generate new simulated pipeline vibration audio signals, and combine the simulated pipeline vibration audio signals with the real pipeline vibration audio signals for data augmentation.

4. The data-augmentation and hybrid neural network architecture based water distribution network leakage identification method of claim 1, wherein, After the pipeline vibration audio signals are sequentially processed by fast Fourier transform and mel filter bank, the mel frequency cepstral coefficients corresponding to each pipeline vibration audio signal frame in the pipeline vibration audio signal are extracted by discrete cosine transform to construct a mel frequency cepstral coefficient feature matrix of the pipeline vibration audio signal frame. The time domain signal of each pipeline vibration audio signal frame in the pipeline vibration audio signal is converted into frequency domain data by fast Fourier transform to obtain spectral information, wherein the spectral information reflects the energy distribution of the pipeline vibration audio signal at different frequencies; Each mel filter in the mel filter bank filters the pipeline vibration audio signal according to the spectral information to determine the output energy on each mel filter channel, and performs logarithmic operation on the output energy on each mel filter channel to obtain the logarithmic energy of each mel filter channel and convert it into a log mel spectrum; Mel frequency cepstral coefficients are extracted from the logarithmic energy of each mel filter channel by discrete cosine transform to obtain the mel frequency cepstral coefficient feature matrix corresponding to each pipeline vibration audio signal frame in the pipeline vibration audio signal.

5. The data-augmentation and hybrid neural network architecture based water distribution network leakage identification method of claim 1, wherein, The feature fusion matrix of each pipeline vibration audio signal frame in the pipeline vibration audio signal is used as a training sample, and the corresponding leakage state of the pipeline vibration audio signal is used as a sample label to train a pre-set CNN-BiLSTM hybrid neural network model to determine a trained supply network leakage detection model in a convergent state, and the steps include: The CNN-BiLSTM hybrid neural network model is optimized by a pre-set Bayesian optimization algorithm to determine the optimal hyperparameter combination of the CNN-BiLSTM hybrid neural network model, wherein the model hyperparameters include learning rate, batch size, and LSTM hidden unit number; The CNN-BiLSTM hybrid neural network model combined with the optimal hyperparameter group is used as the water supply network leakage detection model.

6. The data-augmentation and hybrid neural network architecture based water distribution network leakage identification method of claim 1, wherein, The base network architecture of the water supply network leakage detection model is a CNN-BiLSTM hybrid neural network model, wherein the CNN-BiLSTM hybrid neural network model is constructed by two convolutional neural network layers and one bidirectional long short-term memory network layer.

7. The data-augmentation and hybrid neural network architecture based water distribution network leakage identification method according to any one of claims 1 to 6, characterized in that, The feature fusion matrix represents the joint features of the pipeline vibration audio signal in the time domain and the frequency domain, and contains not only the static spectrum information of the signal, but also the dynamic characteristics of the signal changing over time.

8. A water distribution network leakage identification apparatus based on data augmentation and hybrid neural network architecture, characterized in that, The audio data set construction module is configured to collect real pipeline vibration audio signals in the water supply network, input the real pipeline vibration audio signals into a preset generative adversarial network, generate simulated pipeline vibration audio signals through different rounds of output, and combine the real pipeline vibration audio signals with the simulated pipeline vibration audio signals to construct a pipeline vibration audio signal data set, wherein the pipeline vibration audio signal data set includes a plurality of pipeline vibration audio signals with a certain audio duration, and each pipeline vibration audio signal includes a plurality of pipeline vibration audio signal frames. The mel-frequency cepstral coefficient extraction module is configured to sequentially process the pipeline vibration audio signals by using fast Fourier transform and a mel filter bank, and then extract mel-frequency cepstral coefficients corresponding to each pipeline vibration audio signal frame in the pipeline vibration audio signal by using discrete cosine transform to construct a mel-frequency cepstral coefficient feature matrix of the pipeline vibration audio signal frame. The feature fusion matrix construction module is configured to calculate and determine a first-order difference matrix of the mel-frequency cepstral coefficients, and fuse the mel-frequency cepstral coefficient feature matrix and the first-order difference matrix to determine a feature fusion matrix of each pipeline vibration audio signal frame. The detection model training module is configured to use the feature fusion matrix of each pipeline vibration audio signal frame in the pipeline vibration audio signal as a training sample, use the leakage state corresponding to the pipeline vibration audio signal as a sample label, train a preset CNN-BiLSTM hybrid neural network model, and determine a water supply network leakage detection model that has been trained to a convergent state. The water supply network leakage identification module is configured to input a to-be-identified pipeline vibration audio signal into the water supply network leakage detection model that has been trained to a convergent state, determine whether the to-be-identified pipeline vibration audio signal corresponds to a pipeline in a leakage state or a pipeline in a non-leakage state, and complete water supply network leakage identification based on data enhancement and a hybrid neural network architecture. The central processing unit is configured to invoke and run a computer program stored in the memory to perform the steps of the method according to any one of claims 1 to 7.

9. An electronic device comprising a central processing unit and a memory, characterized in that The computer program is stored in the form of computer readable instructions and is implemented according to the method of any one of claims 1 to 7. When the computer program is invoked and run by a computer, the steps included in the corresponding method are performed.

10. A computer-readable storage medium, characterized in that, ​

Images