A method, apparatus, and article of manufacture for identifying acoustic frequencies of a fluid leak in a hydrocarbon pipeline

Abstract

The present disclosure belongs to the technical field of oil and gas pipeline management, and particularly relates to a method, device and equipment for identifying fluid leakage acoustic frequency of an oil and gas pipeline. The method for identifying fluid leakage acoustic frequency of an oil and gas pipeline is applied to an edge computing device of the oil and gas pipeline, and comprises the following steps: acquiring acoustic frequency; when a short-time energy value of the acoustic frequency is greater than a first preset threshold value, controlling the device to enter a first wake-up state; when the device is in the first wake-up state, if a preset condition is met, controlling the device to enter a second wake-up state; when the device is in the second wake-up state, performing noise reduction processing on the acoustic frequency to obtain a frequency spectrum feature map of the noise-reduced acoustic frequency; and when the device is in the second wake-up state, inputting the frequency spectrum feature map into a model established according to any one of claims 1-6 to identify whether the acoustic frequency belongs to fluid leakage acoustic frequency of the oil and gas pipeline.

Description

Technical Field

[0001] This disclosure belongs to the field of oil and gas pipeline management technology, and specifically relates to a method, apparatus and equipment for identifying the sound frequency of fluid leakage in oil and gas pipelines. Background Technology

[0002] Long-distance oil and gas pipelines are crucial energy infrastructure in my country. As the primary means of transporting oil and gas resources, the safe operation and maintenance of these pipelines are vital for protecting the lives and property of the nation. Pipeline failures and leaks could lead to catastrophic consequences, making it imperative to detect and address potential safety hazards as early as possible.

[0003] Acoustic monitoring is widely used in pipeline leak detection. In recent years, the technology of identifying acoustic waves from oil and gas pipeline leaks based on neural networks has been extensively studied. However, existing technologies do not perform noise reduction processing on the original audio frequencies, making them susceptible to noise from wind, rain, and other sources. Furthermore, the neural network models used are large in scale, computationally complex, and consume a lot of power during operation, making them unsuitable for deployment on low-power edge devices and thus unsuitable for the common field environments of long-distance oil and gas pipelines. Summary of the Invention

[0004] This disclosure proposes a scheme for identifying the sound frequency of fluid leakage in oil and gas pipelines, in order to solve the problem that existing technologies for identifying the sound waves of leakage in oil and gas pipelines based on neural networks are susceptible to noise such as rain and wind, and cannot be deployed on low-power edge devices, thus failing to adapt to the field environment.

[0005] The first aspect of this disclosure provides a method for training an acoustic recognition model for fluid leakage in oil and gas pipelines, comprising: The fluid leakage sound and the specific noise of the environment in different stations of the oil and gas pipeline are input into a pre-trained audio noise reduction model to obtain positive audio samples, and the non-specific noise of the environment is input into the audio noise reduction model to obtain negative audio samples. The spectral feature maps of the positive and negative audio samples are obtained. After feature folding, the spectral feature maps are used as positive and negative samples, respectively. The true values ​​of the audio samples are used as labels. The data are input into a deep segregating convolutional neural network optimization model with embedded feature folding attention for training to obtain a pipeline fluid leakage audio monitoring and identification network model. The deep segregating convolutional neural network optimization model with embedded feature folding attention also includes an aggregation-dilation layer between the two deep segregating convolutional kernels. The aggregation-dilation layer is used to decouple the channel feature relationship of the neural network model from the spatial relationship learned by the convolutional kernel. The deep segregating convolutional neural network optimization model with embedded feature folding attention is quantized so that the model can be deployed on a preset edge computing device.

[0006] In some embodiments, the training method for the audio noise reduction model includes: The fluid leakage sound from different stations of the oil and gas pipeline is obtained as the target audio frequency, and the specific noise of the environment is used as the noise audio frequency. The audio frequencies are then standardized and randomly superimposed, and the random gain coefficients are obtained respectively. The audio frequency after random gain superposition is framed, and the audio frequency features of each audio frequency segment are extracted. The true value of sample gain is calculated based on the original audio energy and the random gain coefficient. The audio features are used as samples, and the true value of sample gain is used as labels. These are then input into a three-layer GRU neural network for training to obtain an audio noise reduction model.

[0007] In some embodiments, processing the audio input into a pre-trained audio noise reduction model to obtain audio samples includes: The frequency domain information of the audio frequency is obtained by processing the pre-trained audio noise reduction model. The frequency domain information of the audio frequency is restored to the time domain information.

[0008] In some embodiments, obtaining the spectral feature map of the audio sample includes: The audio samples are divided according to a preset unit length, and audio samples smaller than the preset unit length are discarded. The segmented audio samples are framed with a frame length of L and a step size of S, and then windowed. The time-domain signal of each frame of audio sample is converted into a frequency-domain signal through a fast Fourier transform. The spectral features are extracted using a Mel filter and the fbank features are obtained through logarithmic operations.

[0009] In some embodiments, the feature folding process performed on the spectral feature map of the audio frequency includes: Arrange the spectral feature maps of the 30 frames horizontally in three consecutive rows, with 10 frames in each row. Feature folding is performed on frames that are in the same position in each of the three rows.

[0010] In some embodiments, quantizing the deep segregating convolutional neural network optimization model of the embedded feature folding attention so that the model can be deployed on a preset terminal device includes: The computation graph parameters of the neural network model are frozen and saved as a pb file with a precision of float64; Based on the TF-lite framework, the float64 pb file is converted into an int8 tflite file. A portion of the fluid leakage sound is selected as a calibration dataset to test the accuracy of the tflite model and perform weight fine-tuning. The tflite file is converted into a C++ file for deployment on a preset terminal device.

[0011] A second aspect of this disclosure provides an apparatus for training an acoustic recognition model for fluid leakage in oil and gas pipelines, comprising: The noise reduction module is used to input the fluid leakage sound and the specific noise of the environment in different stations of the oil and gas pipeline into a pre-trained audio noise reduction model to obtain positive audio samples, and input the non-specific noise of the environment into the audio noise reduction model to obtain negative audio samples. The training module is used to acquire the spectral feature maps of the positive and negative audio samples. After feature folding, the spectral feature maps are used as positive and negative samples, respectively. The true values ​​of the audio samples are used as labels. The training module is then used to train a deep segregating convolutional neural network optimization model with embedded feature folding attention to obtain a pipeline fluid leakage audio monitoring and identification network model. The deep segregating convolutional neural network optimization model with embedded feature folding attention further includes an aggregation-dilation layer between the two deep segregating convolutional kernels. The aggregation-dilation layer is used to decouple the channel feature relationship of the neural network model from the spatial relationship learned by the convolutional kernel. A quantization module is used to quantize the deep segregating convolutional neural network optimization model with embedded feature folding attention so that the model can be deployed on a preset edge computing device.

[0012] A third aspect of this disclosure provides a method for identifying the acoustic frequency of fluid leakage in an oil and gas pipeline, applied to an edge computing device for an oil and gas pipeline, comprising: Acquire audio frequency; when the short-time energy value of the audio frequency is detected to be greater than a first preset threshold, control the device to enter a first wake-up state; when the device is in the first wake-up state, if a preset condition is met, control the device to enter a second wake-up state. When the device is in the second wake-up state, the audio frequency is input into the audio frequency noise reduction model to obtain the spectral feature map of the noise-reduced audio frequency; When the device is in a second wake-up state, the spectral feature map is input into a model established according to the method described in the first aspect of this disclosure to identify whether the sound frequency belongs to the sound frequency of fluid leakage in oil and gas pipelines.

[0013] In some embodiments, the preset conditions refer to: The short-time audio energy is not greater than the second preset threshold or the short-time zero-crossing rate is not less than the third preset threshold.

[0014] A fourth aspect of this disclosure provides a device for identifying the acoustic frequency of fluid leakage in an oil and gas pipeline, applied to an edge computing device for an oil and gas pipeline, comprising: The audio acquisition module is used to acquire audio frequencies. When the short-time energy value of the audio frequency is detected to be greater than a first preset threshold, the device is controlled to enter a first wake-up state. When the device is in the first wake-up state, if a preset condition is met, the device is controlled to enter a second wake-up state. The noise reduction module is used to input the audio frequency into the audio noise reduction model to obtain the spectral feature map of the noise-reduced audio frequency when the device is in the second wake-up state. An identification module is used to input the spectral feature map into a model established according to the method described in the first aspect of this disclosure when the device is in a second wake-up state in order to identify whether the sound frequency belongs to the sound frequency of fluid leakage in an oil and gas pipeline.

[0015] A fifth aspect of this disclosure provides a pipeline fluid leakage audio identification device, comprising an audio receiving module, an audio endpoint detection module, a noise reduction module, and a monitoring module connected in sequence. The noise reduction module and the monitoring module are in a sleep state; The audio receiving module is used to receive audio frequencies; The audio frequency endpoint detection module is used to detect and analyze the audio frequency. When the short-time energy value of the audio frequency is detected to be greater than a first preset threshold, the device is controlled to enter a first wake-up state. When the device is in the first wake-up state, if a preset condition is detected, the device is controlled to enter a second wake-up state. When the device is in the second wake-up state, the noise reduction module and the monitoring module are in working state. The noise reduction module is used to input the audio frequency into the audio frequency noise reduction model to obtain the spectral feature map of the noise-reduced audio frequency when the device is in the second wake-up state; The monitoring module is used to input the spectral feature map into a model established according to the method described in the first aspect of this disclosure when the device is in the second wake-up state, so as to identify whether the sound frequency belongs to the sound frequency of fluid leakage in oil and gas pipelines.

[0016] Preferably, the audio receiving module is an electret microphone.

[0017] In summary, the methods, apparatuses, and devices for training audio recognition models of fluid leaks in oil and gas pipelines, and for recognizing audio frequencies of fluid leaks in oil and gas pipelines, provided in the embodiments of this disclosure, avoid the impact of noise such as rain and wind on the recognition effect by first denoising the audio frequencies to be recognized; by employing a neural network model incorporating attention mechanisms and deep segregating convolution operators, the algorithm complexity can be greatly reduced without significantly reducing the recognition effect; at the same time, by using feature folding to improve recognition efficiency and edge computing speed, a smaller neural network model with lower computational complexity and lower power consumption can be obtained; by directly using the frequency domain information obtained from denoising as the input for recognition, the computation of frequency domain-to-time domain conversion is reduced, thus reducing the amount of computation and power consumption; furthermore, the model is quantized to reduce its size, making it more suitable for deployment on edge devices; by setting different sleep and working modes for the edge devices deploying the recognition model, power consumption is further reduced, meeting the needs of the field environment of long-distance oil and gas pipelines. Attached Figure Description

[0018] The features and advantages of this disclosure will be more clearly understood by referring to the accompanying drawings, which are schematic and should not be construed as limiting the scope of this disclosure in any way. Figure 1 This is a flowchart illustrating a method for training an acoustic recognition model for fluid leakage in oil and gas pipelines, based on some embodiments of this disclosure. Figure 2 These are spectral feature maps of target samples shown according to some embodiments of this disclosure; Figure 3 These are spectral characteristic diagrams of noise samples shown according to some embodiments of this disclosure; Figure 4 These are spectral feature maps of negative samples shown according to some embodiments of this disclosure; Figure 5 These are spectrum diagrams of the target sample before and after standardization, as shown in some embodiments of this disclosure; Figure 6 These are spectrum diagrams of noisy samples before and after normalization, as shown in some embodiments of this disclosure; Figure 7 A schematic diagram of a deep segregating convolutional neural network optimization model with embedded feature folding attention, shown according to some embodiments of the present disclosure; Figure 8 This is a schematic diagram of an apparatus for training an acoustic recognition model of fluid leakage in oil and gas pipelines, according to some embodiments of the present disclosure. Figure 9 This is a flowchart illustrating a method for identifying the acoustic frequency of fluid leakage in an oil and gas pipeline according to some embodiments of the present disclosure; Figure 10 This is an embodiment of the present disclosure showing the actual effect of using a two-level wake-up method to detect synthesized audio containing rain noise, interfering speech, and pipe leak sound frequencies; Figure 11 This is an example of an embodiment of the present disclosure showing the original audio of a pipe fluid leak mixed with wind and rain noise; Figure 12 This is an embodiment of the present disclosure showing the pair Figure 10 The actual effect of noise reduction processing on the original audio of pipeline fluid leakage is shown; Figure 13 This is a schematic diagram of a device for identifying the sound frequency of fluid leakage in an oil and gas pipeline, according to some embodiments of the present disclosure. Detailed Implementation

[0019] In the following detailed description, numerous specific details of this disclosure are set forth by way of example in order to provide a thorough understanding of the relevant disclosure. However, it will be apparent to those skilled in the art that this disclosure may be practiced without these details. It should be understood that the terms “system,” “apparatus,” “unit,” and / or “module” used in this disclosure are a method of distinguishing different parts, elements, sections, or components at different levels in a sequential arrangement. However, these terms may be replaced by other expressions if they can achieve the same purpose.

[0020] It should be understood that when a device, unit, or module is referred to as being "on," "connected to," or "coupled to" another device, unit, or module, it may be directly connected to or coupled to, or communicate with, other devices, units, or modules, or there may be intermediate devices, units, or modules present, unless the context explicitly indicates otherwise. For example, the term "and / or" as used in this disclosure includes any one and all combinations of one or more of the associated listed items.

[0021] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to limit the scope of this disclosure. As shown in this specification and claims, unless the context clearly indicates otherwise, words such as "a," "an," "an," and / or "the" do not specifically refer to the singular and may include the plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified features, integrals, steps, operations, elements, and / or components, and such expressions do not constitute an exclusive list, in which other features, integrals, steps, operations, elements, and / or components may also be included.

[0022] Referring to the following description and accompanying drawings, these and other features and characteristics, operating methods, functions of related structural elements, combinations of parts, and economics of manufacture of this disclosure can be better understood, wherein the description and drawings form part of the specification. However, it is clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this disclosure. It is understood that the drawings are not drawn to scale.

[0023] Various structural diagrams are used in this disclosure to illustrate various variations of embodiments according to this disclosure. It should be understood that the preceding or following structures are not intended to limit this disclosure. The scope of protection of this disclosure is defined by the claims.

[0024] Figure 1 This is a flowchart illustrating a method for training an acoustic recognition model for fluid leaks in oil and gas pipelines, according to some embodiments of this disclosure. In some embodiments, the method for training the acoustic recognition model for fluid leaks in oil and gas pipelines is executed by a model training server. Figure 1 As shown, the method for training the acoustic recognition model for fluid leakage in oil and gas pipelines includes the following steps: S110, the fluid leakage sound and the specific noise of the environment in different stations of the oil and gas pipeline are input into the pre-trained audio noise reduction model to obtain positive audio samples, and the non-specific noise of the environment is input into the audio noise reduction model to obtain negative audio samples.

[0025] In some embodiments of this disclosure, a target feature library of fluid leakage sound in oil and gas pipelines, a noise feature library under environmental background, and a negative sample feature library are first established. When establishing the target feature library of fluid leakage sound from oil and gas pipelines and the noise feature library under environmental background, the feature library is classified according to the following criteria: based on the different environments of the station, valve chamber, and pipeline burial, the sound of fluid leakage from oil and gas pipelines, noise under environmental background, and other non-specific sounds are collected as data sources. The specific classification criteria for the feature library are as follows: Target feature library: a. Sounds of fluid leakage at different oil and gas pipelines and pipeline connections in the station area; b. Sounds of fluid leakage at different oil and gas pipelines and pipeline connections in the valve chamber; c. The sound of fluid leakage in pipelines under different burial environments; Noise signature library: a. Specific noises from the station area, such as wind and rain noises; b. Specific noises from the valve chamber, such as wind noise and rain noise; c. Specific noises of pipelines and lines in different burial environments, such as wind noise and rain noise; Negative sample feature library: a. Non-specific speech in the station area, excluding target feature database and noise feature database; b. Non-specific speech in the valve chamber excluding the target feature library and noise feature library; c. Non-specific speech data of pipelines in different burial environments, excluding target feature databases and noise feature databases; In some embodiments of this disclosure, the target feature library consists of fluid leakage sounds from different oil and gas pipelines and pipeline connections collected from several stations; the noise feature library consists of wind and rain noises under the same collection environment at the stations; and the negative sample feature library consists of sounds generated by daily activities such as the voices of staff and footsteps during inspections under the same collection environment at the stations. The spectral feature diagrams of the target samples, noise samples, and negative samples are shown below. Figure 2 , Figure 3 , Figure 4 As shown.

[0026] Secondly, an audio noise reduction network model is trained based on a feature combination library; In some embodiments of this disclosure, a feature combination library b is selected as the sample dataset to train the audio noise reduction network model based on the specific characteristics of the station environment. The target feature library (sounds of fluid leakage from different oil and gas pipelines and pipeline connections collected at the station) and the noise feature library (wind sound, rain sound) are sequentially standardized and then subjected to random gain superposition. The spectrum diagrams of the target samples before and after standardization are shown in the reference diagram. Figure 5 ; Spectrum diagrams of noisy samples before and after standardization are referenced Figure 6 Input target feature gain randomly from 0.01 to 10, input noise feature gain randomly from 0.1 to 10, and save the two sets of random gain coefficients as follows: Table 1 Random Coefficients for Target / Noise Characteristic Gain

[0027] After preprocessing the dataset, the audio frequency after random gain superposition is divided into frames and windowed (Hamming window) according to the frame length of 64ms (sampling points: 64 / 1000*16000=1024), frame shift of 32ms (sampling points: 32 / 1000*16000=512), and superposition frame length of 32ms (sampling points: 32 / 1000*16000=512). The approximately 1 second (0.992s) audio frequency is processed into 30 audio frequency segments. Then, the time domain signal is converted into the frequency domain signal using the Fast Fourier Transform (FFT) method. Finally, the frequency domain information is divided into spectra based on the frequency band standard used by the Opus codec.

[0028] The noise reduction network model implemented in this invention is constructed using a three-layer GRU neural network. Based on the original audio frequency energy of the target sample and noise sample, and the sample / noise random gain coefficients recorded in Table 1 above, the model is defined according to the frequency band gain formula: Calculate the ground truth of the sample gain in the training dataset. The band gain is calculated using the formula: Smoothing is performed, where the first parameter is the frequency band gain coefficient of the previous frame, and λ is the attenuation factor. Since the time required for a sound source to attenuate by 60dB after stopping sound in an outdoor environment is shorter than in an indoor environment, λ is calculated based on an empirical value of 0.4. Then, for each frequency band k, the formula is applied: Interpolation gain processing is performed. The loss of the denoising network model is calculated using the formula... The calculation uses two parameters: the first parameter is the bandwidth gain calculated by the algorithm model, and the second parameter is the ground truth of the training samples calculated based on the random gain coefficients of the target feature library and the noise feature library. The exponent γ is a perceptual parameter used to control the strength of noise suppression. In this embodiment of the invention, γ is set to 1 / 2, which is equivalent to minimizing the mean square error of the 1 / 4 power energy.

[0029] After the algorithm model was built, the neural network model was trained according to the series of hyperparameters in Table 2. When epoch=120, the algorithm model loss tended to stabilize, and the model training ended. Validated on the test set, the denoising network model parameters improved the Mean Opinion Score (MOS) by approximately 0.7 on a dataset with SNR=0dB~10dB.

[0030] Table 2. A series of hyperparameters for training the model

[0031] Finally, the audio frequencies selected from the feature combination library and the noise feature library are input into the audio frequency denoising network model for denoising processing to obtain the frequency domain information of the audio frequencies, and then restored to the time domain information to obtain positive audio frequency samples and negative audio frequency samples.

[0032] S120, obtain the spectral feature maps of the positive and negative audio samples, and use the spectral feature maps as positive and negative samples respectively after feature folding processing. The real values ​​of the audio samples are used as labels and input into the deep segregating convolutional neural network optimization model with embedded feature folding attention for training to obtain the pipeline fluid leakage audio monitoring and identification network model. The deep segregating convolutional neural network optimization model with embedded feature folding attention also includes a convergence-dilation layer between the two deep segregating convolutional kernels. The convergence-dilation layer is used to decouple the channel feature relationship of the neural network model from the spatial relationship learned by the convolutional kernel.

[0033] In some embodiments of this disclosure, the spectral features of the positive and negative audio samples are first extracted: For the positive / negative samples, they are first segmented according to the minimum unit length, and audio samples smaller than the minimum unit length are discarded. Frame segmentation is performed with a frame length of L and a stride of S. After segmentation, windowing is applied, and then the time-domain signal of each frame is converted into a frequency-domain signal using a Fast Fourier Transform (FFT). Spectral features are extracted using a Mel Filterbank, and the fbank features are obtained through logarithmic operations. After the above processing, the minimum unit yields N frames of spectrograms.

[0034] Then, we build a deep separable convolutional neural network optimization model with embedded feature folding attention (DS-SECNN): Model structure as follows Figure 7 As shown, DS-SECNN adds an SE module to a deep separable convolutional neural network model, including two operations: squeezing and excitation. The main reason for adding the SE module is that the channel feature relationships and the spatial relationships learned by the convolutional kernels in the original deep separable convolutional neural network model are coupled together and cannot be represented separately. The SE module is to decouple the channel and spatial relationships, allowing the model to directly learn the channel feature relationships. Adding the SE module increases the attention of different channels during feature map convolution information extraction. By increasing channel attention, the accuracy of the algorithm model in identifying the sound frequencies of oil and gas pipeline leaks can be improved.

[0035] The first step is the aggregation operation, which compresses the audio features according to the index order generated based on the spatial dimension. That is, the original feature map dimension H*W*C is compressed to 1*1*C, where H is the height, W is the width, and C is the number of channels. Next comes the expansion operation. Using the aggregated 1*1*C feature information, a fully connected layer is trained to obtain channel importance weights. Weights are generated for each feature channel using the parameter w. Finally, the trained weights are applied to the corresponding channels of the feature map. The final step is to train the deep segregating convolutional neural network optimization model with embedded feature folding attention. The extracted audio features are divided into frames by smallest unit, windowed, and processed by Fast Fourier Transform (FFT) to form feature maps. These feature maps are then folded in a continuous frame manner. In some embodiments of this disclosure, a 30-frame continuous vertical frame feature folding process is used. This 30-frame continuous vertical frame feature folding process refers to arranging the 30 frame spectral feature maps horizontally into three rows of 10 frames each, and folding the frames at the same position in each row. Feature folding improves the efficiency of convolutional kernels in extracting feature map information, thereby increasing the model inference speed when the algorithm model is deployed to edge devices, and thus improving edge computing speed. In practical terms, this can accelerate the monitoring and feedback speed of oil and gas pipeline leak accidents.

[0036] Table 3 illustrates the folding process for 30 consecutive vertical frames. Then, the audio features of the positive / negative samples after feature folding, along with the ground truth of the training samples in the positive / negative sample library, are fed into a deep segregating convolutional neural network optimization model with embedded feature folding attention for training.

[0037] Table 3. Schematic diagram of feature folding (continuous vertical frames)

[0038] The training parameters for training the DS-SECNN algorithm model in this embodiment are shown in Table 4. When epoch=120, the algorithm model loss tends to stabilize, and the model training ends.

[0039] Table 4 Training parameters of DS-SECNN model

[0040] S130, quantize the deep segregated convolutional neural network optimization model with embedded feature folding attention so that the model can be deployed on a preset edge computing device.

[0041] Specifically, in some embodiments of this disclosure, the computational graph parameters of the audio noise reduction network model and the DS-SECNN neural network model are frozen and saved as a pb file with a precision of float64.

[0042] Then, based on the TF-lite framework, the saved float64 pb file is converted into an int8 tflite file. A small number of samples from the target feature library are then used as a calibration dataset to test the accuracy of the tflite model and fine-tune the weights. The test results show that the accuracy of the two fused pb models decreased from 94.1% to 92.6%, which reduces the computational complexity within a 2% accuracy loss range. This is more meaningful for the next step of deployment on the terminal system. After conversion, the algorithm accuracy is slightly lower, but the computational complexity is reduced, resulting in a faster inference response time on the chip.

[0043] Finally, based on the Linux system platform, the tflite file was converted into a C++ file using the command `xxd -i *.tflite>*.cc`, and the C++ file was then used for engineering fusion on an MCU terminal device with an electret microphone array. Figure 8 This is a schematic diagram of an apparatus for training an acoustic recognition model of fluid leakage in oil and gas pipelines, according to some embodiments of this disclosure. Figure 8 As shown, the device 800 for training the acoustic recognition model of fluid leakage in oil and gas pipelines includes a noise reduction module 810, a training module 820, and a quantization module 830. The function of training the acoustic recognition model of fluid leakage in oil and gas pipelines can be executed by a model training server. Wherein: The noise reduction module 810 is used to input the fluid leakage sound and the specific noise of the environment in different stations of the oil and gas pipeline into a pre-trained audio noise reduction model to obtain positive audio samples, and input the non-specific noise of the environment into the audio noise reduction model to obtain negative audio samples. The training module 820 is used to acquire the spectral feature maps of the positive and negative audio samples, and after feature folding processing, the spectral feature maps are used as positive and negative samples respectively. The true values ​​of the audio samples are used as labels. The training module is input into a deep segregating convolutional neural network optimization model with embedded feature folding attention to obtain a pipeline fluid leakage audio monitoring and recognition network model. The deep segregating convolutional neural network optimization model with embedded feature folding attention also includes an aggregation-dilation layer between the two deep segregating convolutional kernels. The aggregation-dilation layer is used to decouple the channel feature relationship of the neural network model from the spatial relationship learned by the convolutional kernel. The quantization module 830 is used to quantize the deep segregating convolutional neural network optimization model with embedded feature folding attention so that the model can be deployed on a preset edge computing device.

[0044] Figure 9This is a flowchart illustrating a method for identifying the acoustic frequencies of fluid leaks in oil and gas pipelines, according to some embodiments of this disclosure. In some embodiments, the method for identifying the acoustic frequencies of fluid leaks in oil and gas pipelines is performed by edge computing devices deployed around the oil and gas pipeline. Figure 9 As shown, the method for identifying the acoustic frequency of fluid leakage in oil and gas pipelines includes the following steps: S910, acquire audio frequency; when the short-time energy value of the audio frequency is detected to be greater than a first preset threshold, control the device to enter a first wake-up state; when the device is in the first wake-up state, if a preset condition is met, control the device to enter a second wake-up state. In some embodiments of this disclosure, the preset conditions refer to: The short-time audio energy is not greater than the second preset threshold or the short-time zero-crossing rate is not less than the third preset threshold.

[0045] Specifically, in some embodiments of this disclosure, the audio detection function is implemented through a two-level wake-up system, AFAD1 and AFAD2. AFAD1 determines whether the system should be woken up at level one by judging whether the currently received short-time energy value of the audio signal is greater than an energy threshold of 0.1. After level one wake-up, AFAD2 remains in a dormant state under the condition of high short-time energy (STE) and low zero-crossing rate (ZCC), and is activated under other conditions. The STE and ZCC thresholds are set to empirical values ​​of 0.05 and 40, respectively. The short-time energy threshold in AFAD1 represents the minimum volume that can be detected in the real environment; the zero-crossing rate monitors the duration of sound in the real environment. We believe that situations where the sound volume is greater than the short-time energy but the duration is extremely short are not within the scope of concern for oil and gas pipeline leak accidents. The audio detection module reduces power consumption and achieves the goal of ultra-low power consumption by filtering out these clearly non-pipeline leak accident time periods.

[0046] S920, when the device is in the second wake-up state, the audio frequency is input into the audio frequency noise reduction model to obtain the spectral feature map of the noise-reduced audio frequency; S930, when the device is in the second wake-up state, the spectral feature map is input. Figure 1 The models established in S110-S130 are used to identify whether the sound frequencies belong to fluid leakage sound frequencies in oil and gas pipelines.

[0047] Unlike the training method of the DS-SECNN model, during recognition, we directly input the frequency domain information obtained by the audio denoising model after denoising the audio to be identified into the DS-SECNN network for recognition, without going through the step of restoring the frequency domain information obtained by the audio denoising model to the time domain information and then obtaining the spectral feature map. By directly using the output of the denoising process as the input of the recognition step, we can save the computational load and corresponding power consumption of restoring the frequency domain information to the time domain information.

[0048] This embodiment of the disclosure takes a segment of audio recording of a leaking oil and gas pipeline at a station, which includes wind, rain, and interfering audio, and processes and tests it. The specific audio data processing flow is as follows: The audio detection module identifies the start and end points of audio signals in the environment, excluding silence and interfering speech. An example is taken: a synthesized audio stream (with silence at the connection points) consisting of interfering speech, pipe leak audio, wind and rain noise, interfering speech, and pipe leak audio. The actual detection effect of the audio detection module is as follows: Figure 10 ; In the secondary wake-up state of the audio detection module, the noise reduction module is used to process the audio frequency, resulting in a noise-reduced audio frequency spectrum feature map. In this embodiment of the invention, the original audio frequency of pipeline fluid leakage mixed with wind and rain noise is used as a reference. Figure 11 The noise reduction effect after noise reduction processing is referenced. Figure 12 ; The monitoring module uses inference to judge the feature map to determine whether the monitored audio frequency belongs to the fluid leakage audio frequency of oil and gas pipeline. In the example, a 60-minute audio segment with a signal-to-noise ratio of 0dB to 10dB containing rain noise, wind noise and interference speech was selected. After processing and testing by the three-level audio processing system, under the condition that the confidence threshold of the monitoring network model is set to 0.85, the accuracy and recall of the three-level audio processing system are 94.1% and 93.2%, respectively.

[0049] Figure 13 This is a schematic diagram of a device for identifying the sound frequency of fluid leakage in an oil and gas pipeline, according to some embodiments of this disclosure. Figure 13 As shown, the device 1300 for identifying the sound frequency of fluid leakage in oil and gas pipelines includes a sound frequency acquisition module 1310, a noise reduction module 1320, and an identification module 1330. The function of identifying the sound frequency of fluid leakage in oil and gas pipelines can be performed by edge computing devices deployed around the oil and gas pipelines. The audio acquisition module 1310 is used to acquire audio frequencies. When the short-time energy value of the audio frequency is detected to be greater than a first preset threshold, the device is controlled to enter a first wake-up state. When the device is in the first wake-up state, if a preset condition is met, the device is controlled to enter a second wake-up state. The noise reduction module 1320 is used to input the audio frequency into the audio frequency noise reduction model to obtain the spectral feature map of the noise-reduced audio frequency when the device is in the second wake-up state. The identification module 1330 is used to input the spectral feature map according to the device when the device is in a second wake-up state. Figure 1 The model established by the method described in S110-S130 is used to identify whether the sound frequency belongs to the sound frequency of fluid leakage in oil and gas pipelines.

[0050] Some embodiments of this disclosure provide a pipeline fluid leakage audio identification device, including an audio receiving module, an audio endpoint detection module, a noise reduction module, and a monitoring module connected in sequence. The noise reduction module and the monitoring module are in a sleep state; The audio receiving module is used to receive audio frequencies; The audio frequency endpoint detection module is used to detect and analyze the audio frequency. When the short-time energy value of the audio frequency is detected to be greater than a first preset threshold, the device is controlled to enter a first wake-up state. When the device is in the first wake-up state, if a preset condition is detected, the device is controlled to enter a second wake-up state. When the device is in the second wake-up state, the noise reduction module and the monitoring module are in working state. The noise reduction module is used to input the audio frequency into the audio frequency noise reduction model to obtain the spectral feature map of the noise-reduced audio frequency when the device is in the second wake-up state; The monitoring module is used to input the spectrum feature map according to the data when the device is in the second wake-up state. Figure 1 The model established by the method described in S110-S130 is used to identify whether the sound frequency belongs to the sound frequency of fluid leakage in oil and gas pipelines.

[0051] Specifically, in some embodiments of this disclosure, the pipeline fluid leakage audio identification device is an edge computing device deployed around the oil and gas pipeline. In particular, the edge computing device is a SOC-level MCU terminal device, such as the STM32F7 SOC-level chip based on Arm M7. The audio noise reduction model and the DS-SECNN network are deployed on the STM32F7, which also has a two-level wake-up system (AFAD1 and AFAD2) and an electret microphone or electret microphone array for audio detection.

[0052] The edge computing device can be deployed indoors or outdoors. It can be connected to mains power or not, equipped with a charging module powered by renewable energy sources such as solar or wind power. The identification results from the edge computing device can be uploaded to a cloud-based server for processing or stored locally. High-risk results can trigger alerts to relevant personnel via phone, SMS, email, or other means.

[0053] In summary, the methods, apparatuses, and devices for training audio recognition models of fluid leaks in oil and gas pipelines, and for recognizing audio frequencies of fluid leaks in oil and gas pipelines, provided in the embodiments of this disclosure, avoid the impact of noise such as rain and wind on the recognition effect by first denoising the audio frequencies to be recognized; by employing a neural network model incorporating attention mechanisms and deep segregating convolution operators, the algorithm complexity can be greatly reduced without significantly reducing the recognition effect; at the same time, by using feature folding to improve recognition efficiency and edge computing speed, a smaller neural network model with lower computational complexity and lower power consumption can be obtained; by directly using the frequency domain information obtained from denoising as the input for recognition, the computation of frequency domain-to-time domain conversion is reduced, thus reducing the amount of computation and power consumption; furthermore, the model is quantized to reduce its size, making it more suitable for deployment on edge devices; by setting different sleep and working modes for the edge devices deploying the recognition model, power consumption is further reduced, meeting the needs of the field environment of long-distance oil and gas pipelines.

[0054] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding descriptions in the foregoing device embodiments, and will not be repeated here.

[0055] Although the subject matter described herein is provided in the general context of execution on a computer system in conjunction with an operating system and applications, those skilled in the art will recognize that other implementations can also be executed in conjunction with other types of program modules. Generally, program modules include routines, programs, components, data structures, and other types of structures that perform specific tasks or implement specific abstract data types. Those skilled in the art will understand that the subject matter described herein can be practiced using other computer system configurations, including handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframes, etc., and can also be used in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules may reside on both local and remote memory storage devices.

[0056] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0057] It should be understood that the specific embodiments described above are merely illustrative or explanatory of the principles of this disclosure and do not constitute a limitation thereof. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of this disclosure should be included within the protection scope of this disclosure. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A method for training an acoustic recognition model for fluid leakage in oil and gas pipelines, characterized in that, include: The fluid leakage sound and the specific noise of the environment in different stations of the oil and gas pipeline are input into a pre-trained audio noise reduction model to obtain positive audio samples, and the non-specific noise of the environment is input into the audio noise reduction model to obtain negative audio samples. The spectral feature maps of the positive and negative audio samples are obtained. After feature folding, the spectral feature maps are used as positive and negative samples, respectively. The true values ​​of the audio samples are used as labels. The data are input into a deep segregating convolutional neural network optimization model with embedded feature folding attention for training to obtain a pipeline fluid leakage audio monitoring and identification network model. The deep segregating convolutional neural network optimization model with embedded feature folding attention also includes an aggregation-dilation layer between the two deep segregating convolutional kernels. The aggregation-dilation layer is used to decouple the channel feature relationship of the neural network optimization model from the spatial relationship learned by the convolutional kernel. The deep segregating convolutional neural network optimization model with embedded feature folding attention is quantized so that the optimized neural network model can be deployed on a preset edge computing device; wherein... The training method for the audio noise reduction model includes: The fluid leakage sound from different stations of the oil and gas pipeline is obtained as the target audio frequency, and the specific noise of the environment is used as the noise audio frequency. The audio frequencies are then standardized and randomly gained, and the random gain coefficients are obtained respectively. The audio frequency after random gain superposition is framed, and the audio frequency features of each audio frequency segment are extracted. The original audio energy and the random gain coefficient are used to calculate the true value of the sample gain. The audio features are used as samples and the true value of the sample gain is used as labels. The samples are then input into a three-layer GRU neural network for training to obtain an audio noise reduction model. The spectral feature map of the audio frequency undergoes feature folding processing, which includes: Arrange the spectral feature maps of the 30 frames horizontally in three consecutive rows, with 10 frames in each row. Feature folding is performed on frames that are in the same position in each of the three rows.

2. The method according to claim 1, characterized in that, The audio samples obtained by processing the pre-trained audio noise reduction model include: The frequency domain information of the audio frequency is obtained by processing the pre-trained audio noise reduction model. The frequency domain information of the audio frequency is restored to the time domain information.

3. The method according to claim 1, characterized in that, Obtaining the spectral feature map of the audio sample includes: The audio samples are divided according to a preset unit length, and audio samples smaller than the preset unit length are discarded. The segmented audio samples are framed with a frame length of L and a step size of S, and then windowed. The time-domain signal of each frame of audio sample is converted into a frequency-domain signal through a fast Fourier transform. The spectral features are extracted using a Mel filter and the fbank features are obtained through logarithmic operations.

4. The method according to claim 1, characterized in that, The optimization model of the deep segregating convolutional neural network that quantifies the embedded feature folding attention to enable deployment of the model on a preset edge computing device includes: The computation graph and parameters of the neural network optimization model are frozen and saved as a pb file with a precision of float64; Based on the TF-lite framework, the float64 pb file is converted into an int8 tflite file. A portion of the fluid leakage sound is selected as a calibration dataset to test the accuracy of the tflite model and perform weight fine-tuning. The tflite file is converted into a C++ file for deployment on a pre-defined edge computing device.

5. An apparatus for training an acoustic recognition model for fluid leakage in oil and gas pipelines, wherein the apparatus is applied to the method described in any one of claims 1 to 4, characterized in that, include: The noise reduction module is used to input the fluid leakage sound and the specific noise of the environment in different stations of the oil and gas pipeline into a pre-trained audio noise reduction model to obtain positive audio samples, and input the non-specific noise of the environment into the audio noise reduction model to obtain negative audio samples. The training module is used to acquire the spectral feature maps of the positive and negative audio samples. After feature folding, the spectral feature maps are used as positive and negative samples, respectively. The true values ​​of the audio samples are used as labels. The data are input into a deep segregating convolutional neural network optimization model with embedded feature folding attention for training, thereby obtaining a pipeline fluid leakage audio monitoring and identification network model. The deep segregating convolutional neural network optimization model with embedded feature folding attention further includes an aggregation-dilation layer between the two deep segregating convolutional kernels. The aggregation-dilation layer is used to decouple the channel feature relationship of the neural network optimization model from the spatial relationship learned by the convolutional kernel. The quantization module is used to quantize the deep segregating convolutional neural network optimization model with embedded feature folding attention so that the neural network optimization model can be deployed on a preset edge computing device.

6. A method for identifying the acoustic frequency of fluid leakage in oil and gas pipelines, applied to edge computing devices for oil and gas pipelines, characterized in that... include: Acquire audio frequency; when the short-time energy value of the audio frequency is detected to be greater than a first preset threshold, control the device to enter a first wake-up state; when the device is in the first wake-up state, if a preset condition is met, control the device to enter a second wake-up state. When the device is in the second wake-up state, the audio frequency is input into the audio frequency noise reduction model to obtain the spectral feature map of the noise-reduced audio frequency; When the device is in the second wake-up state, the spectral feature map is input into the identification model established by the method according to any one of claims 1-4 to identify whether the sound frequency belongs to the sound frequency of fluid leakage in oil and gas pipelines.

7. The method according to claim 6, characterized in that, The preset conditions refer to: The short-time audio energy is not greater than the second preset threshold or the short-time zero-crossing rate is not less than the third preset threshold.

8. A device for identifying the acoustic frequency of fluid leakage in oil and gas pipelines, applied to edge computing equipment for oil and gas pipelines, characterized in that, include: The audio acquisition module is used to acquire audio frequencies. When the short-time energy value of the audio frequency is detected to be greater than a first preset threshold, the device is controlled to enter a first wake-up state. When the device is in the first wake-up state, if a preset condition is met, the device is controlled to enter a second wake-up state. The noise reduction module is used to input the audio frequency into the audio noise reduction model to obtain the spectral feature map of the noise-reduced audio frequency when the device is in the second wake-up state. An identification module is used to input the spectral feature map into the identification model established by the method according to any one of claims 1-4 when the device is in a second wake-up state in order to identify whether the sound frequency belongs to the sound frequency of fluid leakage in oil and gas pipelines.

9. A pipeline fluid leakage audio identification device, characterized in that, It includes an audio receiving module, an audio endpoint detection module, a noise reduction module, and a monitoring module, which are connected in sequence. The noise reduction module and the monitoring module are in a sleep state; The audio receiving module is used to receive audio frequencies; The audio frequency endpoint detection module is used to detect and analyze the audio frequency. When the short-time energy value of the audio frequency is detected to be greater than a first preset threshold, the device is controlled to enter a first wake-up state. When the device is in the first wake-up state, if a preset condition is detected, the device is controlled to enter a second wake-up state. When the device is in the second wake-up state, the noise reduction module and the monitoring module are in working state. The noise reduction module is used to input the audio frequency into the audio noise reduction model to obtain the spectral feature map of the noise-reduced audio frequency when the device is in the second wake-up state; The monitoring module is used to input the spectral feature map into the identification model established by the method according to any one of claims 1-4 when the device is in the second wake-up state to identify whether the sound frequency belongs to the sound frequency of fluid leakage in oil and gas pipelines.

10. The device according to claim 9, characterized in that: The audio receiving module is an electret microphone.

Images