Omnibearing steam leakage detection method based on acousto-optic fusion

Abstract

The invention discloses an omnibearing steam leakage detection method based on acousto-optic fusion. The method comprises the steps that a sound sensor obtains audio data of a steam pipe network area to calculate a sound detection value; the microphone sound source array sensor obtains a sound source orientation of an audio data source in a steam pipe network area, and a horizontal rotating shaft of the visual holder coincides with an orientation axis of the microphone sound source array sensor; when the sound detection value is greater than a preset sound detection threshold value, the visual holder rotates based on the sound source orientation of the audio data greater than the preset sound detection threshold value, and collects a steam pipe network area detection image along the facing sound source direction after rotation to calculate an image detection value; the data processing upper computer calculates a steam leakage detection value according to the sound detection value and the image detection value, when the steam leakage detection value is larger than a preset steam leakage threshold value, steam leakage alarm information is sent, and the pipeline steam leakage detection precision is improved by fusing sound and visual information.

Description

Technical Field

[0001] The present application relates to the technical field of pressure medium leakage detection, and in particular to an all-round steam leakage detection method based on acoustic and optical fusion. Background Art

[0002] Steam, as an important carrier of energy transfer, is widely used in the fields of petroleum, energy, etc., especially in the field of nuclear power, and plays a huge role in the operation of nuclear power plants. Nuclear power plants and their plant buildings have a huge steam pipe network system with complex pipelines. Leakage of steam medium in the pipeline loop will affect the stable operation of nuclear power plants. Therefore, in order to ensure the stable operation of nuclear power plants, nuclear power plants are required to have a high steam leak detection capability, timely discover the location of pipeline leaks, repair or replace the pipelines, and avoid serious accidents. However, steam detection is currently mainly carried out by manual detection, but due to the large number of pipelines and low detection efficiency, it is difficult to achieve detection during operation. Summary of the invention

[0003] In view of this, the present application provides an all-round steam leak detection method based on sound and light fusion, wherein the sound sensor obtains audio data of the steam pipe network area and calculates the sound detection value; the microphone sound source array sensor obtains the sound source orientation of the audio data source in the steam pipe network area, and the horizontal rotation axis of the visual pan-tilt head coincides with the azimuth axis of the microphone sound source array sensor; when the sound detection value is greater than a preset sound detection threshold, the visual pan-tilt head rotates based on the sound source orientation of the audio data greater than the preset sound detection threshold, and collects a detection image of the steam pipe network area along the direction of the sound source facing the rotation to calculate the image detection value; the data processing host computer calculates the steam leakage detection value based on the sound detection value and the image detection value, and sends a steam leakage alarm message when the steam leakage detection value is greater than the preset steam leakage threshold, thereby improving the accuracy of pipeline steam leakage detection by fusing sound and visual information.

[0004] According to one aspect of the present application, a comprehensive steam leak detection method based on acoustic-optical fusion is provided, the method comprising:

[0005] The sound sensor acquires audio data of the steam pipe network area in real time, and acquires a sound detection value based on the audio data;

[0006] The microphone sound source array sensor acquires the sound source orientation of the audio data in the steam pipe network area in real time, wherein the microphone sound source array sensor is connected to a visual pan-tilt platform, and the horizontal rotation axis of the visual pan-tilt platform coincides with the orientation axis of the microphone sound source array sensor;

[0007] When the sound detection value is greater than a preset sound detection threshold, the visual pan / tilt head rotates based on the sound source orientation of the audio data greater than the preset sound detection threshold, and collects a steam pipe network area detection image along the sound source direction facing the rotation, and obtains an image detection value based on the steam pipe network area detection image;

[0008] The data processing host computer calculates a steam leakage detection value according to the sound detection value and the image detection value, and sends a steam leakage alarm message when the steam leakage detection value is greater than a preset steam leakage threshold value, wherein the steam leakage alarm message includes a collected steam pipe network area detection image, and the steam pipe network area detection image includes a detected steam leakage area.

[0009] Optionally, the visual pan head corresponds to a plurality of preset pan head azimuth angles, and the visual pan head rotates based on the sound source azimuth of the audio data greater than a preset sound detection threshold, and collects a steam pipe network area detection image along the sound source direction facing after the rotation, including:

[0010] The visual pan-tilt head determines a pan-tilt target rotation azimuth angle based on the sound source azimuth of the audio data greater than a preset sound detection threshold value in the corresponding plurality of pan-tilt preset azimuth angles, and rotates to the pan-tilt target rotation azimuth angle;

[0011] After the visual pan-tilt head rotates to the pan-tilt head target rotation azimuth angle, the steam pipe network area detection image is collected along the direction of the sound source facing after the rotation, wherein the pan-tilt head preset azimuth angle is obtained by averaging the preset angle based on the horizontal plane where the visual pan-tilt head is located.

[0012] Optionally, the data processing host computer calculates the steam leakage detection value according to the sound detection value and the image detection value, including:

[0013] The data processing host computer calculates the steam leakage detection value based on the steam leakage detection value calculation formula, the sound detection value and the image detection value, wherein the steam leakage detection value calculation formula is:

[0014] R=ωR1+R2,

[0015] R is the steam leakage detection value, ω is the preset fusion coefficient, R1 is the sound detection value, and R2 is the image detection value.

[0016] Optionally, the audio data is two-channel audio data, and the sound sensor acquires the sound detection value based on the audio data, including:

[0017] The sound sensor filters the acquired dual-channel audio data to obtain an initial audio sequence, intercepts the normalized initial audio sequence based on a sliding window, obtains multiple target audio sequences, and obtains sound detection values ​​corresponding to the target audio sequences based on a steam leak sound detection model.

[0018] Optionally, the steam leakage sound detection model is established based on a convolutional neural network and a long-short time neural network. After the target audio sequence is input into the steam leakage sound detection model, the sound detection value is output after passing through a convolutional layer, a pooling layer, a Dropout layer, a Flatten layer, a RepeatVector layer, a LSTM layer and a Dense layer.

[0019] Optionally, the visual PTZ detects an image based on the steam pipe network area and acquires an image detection value, including:

[0020] The visual PTZ divides the steam pipe network area detection image to obtain a plurality of steam pipe network area detection image units;

[0021] The visual PTZ determines the dynamic change area of ​​steam leakage based on the inter-frame difference method and the steam pipe network area detection image unit;

[0022] The visual gimbal crops the minimum continuous area image containing the dynamically changing area of ​​steam leakage in the steam leakage area image, and obtains the image detection value corresponding to the minimum continuous area image based on the image detection model, wherein the image detection model is established based on the YOLOv5 target detection framework.

[0023] Optionally, the minimum continuous area image includes multiple frames, and the visual gimbal obtains an image detection value corresponding to the minimum continuous area image based on an image detection model:

[0024] The visual gimbal obtains the image detection result value of each minimum continuous area image of each frame based on the image detection model;

[0025] The visual gimbal obtains an image detection value based on an average of a preset number of image detection result values ​​in the image detection result values.

[0026] By means of the above technical scheme, the present application provides an all-round steam leak detection method based on sound and light fusion, wherein the sound sensor obtains the audio data of the steam pipe network area and calculates the sound detection value; the microphone sound source array sensor obtains the sound source orientation of the audio data source in the steam pipe network area, and the horizontal rotation axis of the visual pan-tilt unit coincides with the azimuth axis of the microphone sound source array sensor; when the sound detection value is greater than the preset sound detection threshold, the visual pan-tilt unit rotates based on the sound source orientation of the audio data greater than the preset sound detection threshold, and collects the steam pipe network area detection image along the sound source direction facing after the rotation to calculate the image detection value; the data processing host computer calculates the steam leakage detection value according to the sound detection value and the image detection value, and sends a steam leakage alarm message when the steam leakage detection value is greater than the preset steam leakage threshold, thereby improving the accuracy of pipeline steam leakage detection by fusing sound and visual information.

[0027] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0029] Figure 1 A schematic flow chart of a comprehensive steam leak detection method based on acoustic-optical fusion provided in an embodiment of the present application is shown;

[0030] Figure 2 A schematic diagram of the arrangement of a sound sensor, a microphone sound source array sensor and a visual pan-tilt platform provided in an embodiment of the present application is shown;

[0031] Figure 3 A schematic flow chart of another omnidirectional steam leak detection method based on acoustic-optical fusion provided in an embodiment of the present application is shown;

[0032] Figure 4 A schematic diagram of adjusting the orientation of a visual cradle head provided in an embodiment of the present application is shown;

[0033] Figure 5 A schematic flow chart of another all-round steam leak detection method based on sound and light fusion provided in an embodiment of the present application is shown;

[0034] Figure 6 A schematic diagram of image positioning area division provided in an embodiment of the present application is shown;

[0035] Figure 7 A schematic diagram of image positioning and cropping provided in an embodiment of the present application is shown;

[0036] 21 - sound sensor, 22 - microphone sound source array sensor, 23 - visual pan-tilt platform. DETAILED DESCRIPTION

[0037] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0038] In this embodiment, a comprehensive steam leak detection method based on acoustic and optical fusion is provided. Figure 1 As shown, the method includes:

[0039] Step 101: The sound sensor acquires audio data of the steam network area in real time, and acquires a sound detection value based on the audio data.

[0040] Early detection was mainly carried out by manual detection, but due to the large number of pipelines and low detection efficiency, it was difficult to achieve detection during operation. With the development of electronic technology, sensor-based detection methods have developed rapidly. The main methods for steam leak detection currently include sound sensors, infrared thermal imaging, high-definition images and other methods. The above methods mainly use single-source information to detect and locate steam leaks. Single-source information is less effective than multi-dimensional information in terms of information dimension. For example, visual information is easily affected by ambient lighting, and sound information is easily affected by ambient noise.

[0041] In the above embodiments of the present application, the complementarity of sound information and visual information can effectively increase the richness of collected information and provide a data basis for improving the accuracy of steam leak detection. For the method of monitoring using fixed-point sensors, in order to improve the regional monitoring capability and range, it is necessary to use distributed sensors for multi-point monitoring, thereby increasing the operating cost and maintenance requirements of the monitoring system. In addition, for detection using visual images, due to the influence of the camera's field of view, increasing the monitoring range of a single camera will reduce the image acquisition quality, thereby affecting the recognition accuracy. Specifically, in order to obtain on-site sound and image signals, such as Figure 2 As shown, the steam pipe network monitoring system composed of the sound sensor 21, the microphone sound source array sensor 22, and the visual pan-tilt platform 23 can be arranged in the steam pipe network area so that the image field of the visual pan-tilt platform covers the steam pipe network leakage area. Then, the sound sensor obtains the audio data of the steam pipe network area in real time, and obtains the sound detection value based on the audio data.

[0042] Step 102, the microphone sound source array sensor obtains the direction of the sound source that generates audio data in the steam network area in real time, wherein the microphone sound source array sensor is connected to a visual pan-tilt platform, and the horizontal rotation axis of the visual pan-tilt platform coincides with the azimuth axis of the microphone sound source array sensor.

[0043] Step 103, when the sound detection value is greater than a preset sound detection threshold, the visual gimbal rotates based on the sound source orientation of the audio data greater than the preset sound detection threshold, and collects a steam pipe network area detection image along the sound source direction facing after the rotation, and obtains an image detection value based on the steam pipe network area detection image.

[0044] Next, the microphone sound source array sensor obtains the direction of the sound source that generates audio data in the steam network area in real time. Since the sound sensor and the microphone sound source array sensor are arranged near the visual pan-tilt platform, and the azimuth axis of the microphone sound source array sensor coincides with the horizontal rotation axis of the high-definition visual pan-tilt platform, when the sound detection value is greater than the preset sound detection threshold, the visual pan-tilt platform can rotate based on the direction of the sound source, and collect the steam network area detection image along the direction of the rotated sound source, and obtain the image detection value based on the steam network area detection image. By real-time monitoring whether audio data is generated in the steam network area, when audio data is generated, it indicates that steam leakage may occur. Therefore, the direction of the sound source that generates the sound is obtained, and the visual pan-tilt platform is rotated to the aforementioned sound source direction and then the image data is collected. The image can be collected in a targeted manner for the area where leakage may occur, which can further improve the detection accuracy.

[0045] Step 104: the data processing host computer calculates a steam leakage detection value according to the sound detection value and the image detection value, and sends a steam leakage alarm message when the steam leakage detection value is greater than a preset steam leakage threshold value, wherein the steam leakage alarm message includes a collected steam pipe network area detection image, and the steam pipe network area detection image includes a detected steam leakage area.

[0046] Next, the data processing host computer calculates the steam leakage detection value based on the sound detection value and the image detection value. When the steam leakage detection value is greater than the preset steam leakage threshold, a steam leakage alarm message is sent, wherein the steam leakage alarm message includes the collected steam pipe network area detection image, and the steam pipe network area detection image includes the detected steam leakage area. To this end, the automatic judgment of steam leakage is realized by the fusion of sound information and visual information. By increasing the complementarity of sound and visual information, the effect and detection efficiency of steam leakage identification are improved, which can reduce the work intensity of operation and maintenance personnel; the sound source array sensor senses the direction of the leakage sound source, and adjusts the direction of the pan-tilt head so that the field of view of the collected image can cover the steam leakage area, realizing all-round steam leakage detection.

[0047] By applying the technical solution of this embodiment, the sound sensor obtains the audio data of the steam pipe network area and calculates the sound detection value; the microphone sound source array sensor obtains the sound source orientation of the audio data in the steam pipe network area, wherein the horizontal rotation axis of the visual pan-tilt head coincides with the azimuth axis of the microphone sound source array sensor; when the sound detection value is greater than the preset sound detection threshold, the visual pan-tilt head rotates based on the sound source orientation of the audio data greater than the preset sound detection threshold, and collects the steam pipe network area detection image along the direction of the sound source facing the rotation to calculate the image detection value; the data processing host computer calculates the steam leakage detection value based on the sound detection value and the image detection value, and sends a steam leakage alarm message when the steam leakage detection value is greater than the preset steam leakage threshold. By integrating the sound information with the visual information, the accuracy and efficiency of pipeline steam leakage detection can be improved.

[0048] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another omnidirectional steam leak detection method based on acoustic and optical fusion is provided, such as Figure 3 As shown, the method includes:

[0049] Step 301: The sound sensor acquires audio data of the steam network area in real time, and acquires a sound detection value based on the audio data.

[0050] Step 302, the microphone sound source array sensor obtains the direction of the sound source that generates audio data in the steam network area in real time, wherein the microphone sound source array sensor is connected to a visual pan-tilt platform, the horizontal rotation axis of the visual pan-tilt platform coincides with the azimuth axis of the microphone sound source array sensor, and the visual pan-tilt platform corresponds to a plurality of preset azimuth angles of the pan-tilt platform.

[0051] In the above embodiment of the present application, the sound sensor acquires audio data of the steam pipe network area in real time, acquires sound detection values ​​based on the audio data, and the microphone sound source array sensor acquires the sound source orientation of the audio data in the steam pipe network area in real time. The microphone sound source array sensor is connected to a visual pan-tilt platform, and the horizontal rotation axis of the visual pan-tilt platform coincides with the orientation axis of the microphone sound source array sensor. This is to prepare for the subsequent fusion of sound information and image information.

[0052] Step 303, when the sound detection value is greater than the preset sound detection threshold, the visual gimbal determines the gimbal target rotation azimuth angle based on the sound source azimuth of the audio data greater than the preset sound detection threshold in the corresponding multiple gimbal preset azimuth angles, and rotates to the gimbal target rotation azimuth angle.

[0053] Step 304, after the visual pan-tilt head rotates to the pan-tilt head target rotation azimuth angle, it collects the steam pipe network area detection image along the direction of the sound source facing after the rotation, wherein the pan-tilt head preset azimuth angle is obtained by averaging the preset angle based on the horizontal plane where the visual pan-tilt head is located.

[0054] Step 305: The visual PTZ detects images based on the steam pipe network area and obtains image detection values.

[0055] Step 306: The data processing host computer calculates the steam leakage detection value based on the steam leakage detection value calculation formula, the sound detection value and the image detection value, wherein the steam leakage detection value calculation formula is:

[0056] R=ωR1+R2,

[0057] R is the steam leakage detection value, ω is the preset fusion coefficient, R1 is the sound detection value, and R2 is the image detection value.

[0058] Next, a first-level judgment is made based on the sound detection value R1. When R1 is greater than the preset sound detection threshold, the horizontal azimuth of the visual PTZ is adjusted according to the direction of the sound source to collect high-definition images of the scene in real time. Figure 4 As shown, the horizontal plane of the visual PTZ is used as the initial reference azimuth and is divided into n equal parts. The corresponding preset azimuth angles of the PTZ are θ1, θ2, ...θ in the counterclockwise direction. n , compare the sound source azimuth with the preset azimuth of each pan-tilt, take the most adjacent azimuth as the pan-tilt target rotation azimuth of the visual pan-tilt, adjust the pan-tilt field of view for image acquisition, that is, after the visual pan-tilt rotates to the pan-tilt target rotation azimuth, collect the steam pipe network area detection image along the direction of the sound source facing after rotation. In the whole process, only sound sensors, sound source array sensors and visual pan-tilts are used, which are easy to deploy and maintain. In addition, low-power sound recognition information is used as a judgment information once, and the visual image fusion judgment is performed after reaching the threshold, which can reduce invalid recognition and reduce the power consumption of the monitoring system in long-term operation. Then, the visual pan-tilt detects the image based on the steam pipe network area and obtains the image detection value. The data processing host computer calculates the steam leakage detection value based on the steam leakage detection value calculation formula, sound detection value and image detection value. In particular, 0≤ω≤1 and ω is positively correlated with R2.

[0059] Step 307, when the steam leakage detection value is greater than a preset steam leakage threshold, the data processing host computer sends a steam leakage alarm message, wherein the steam leakage alarm message includes a collected steam pipe network area detection image, and the steam pipe network area detection image includes the detected steam leakage area.

[0060] Next, when the steam leakage detection value is greater than the preset steam leakage threshold, the data processing host computer sends a steam leakage alarm message so that the staff can find the steam leakage location in time. To this end, the whole process can perform all-round steam leakage detection based on sound and light fusion.

[0061] By applying the technical solution of this embodiment, the sound sensor obtains the audio data of the steam pipe network area in real time, and obtains the sound detection value based on the audio data. The microphone sound source array sensor obtains the sound source orientation of the audio data in the steam pipe network area in real time. When the sound detection value is greater than the preset sound detection threshold, the visual pan-tilt determines the pan-tilt target rotation azimuth based on the sound source orientation of the audio data greater than the preset sound detection threshold in the corresponding multiple pan-tilt preset azimuths, and rotates to the pan-tilt target rotation azimuth, and collects the steam pipe network area detection image along the sound source direction facing after rotation. The visual pan-tilt obtains the image detection value based on the steam pipe network area detection image, and the data processing host computer calculates the steam leakage detection value based on the steam leakage detection value calculation formula, the sound detection value and the image detection value. When the steam leakage detection value is greater than the preset steam leakage threshold, the data processing host computer sends a steam leakage alarm message, the purpose is to solve the steam leakage detection problem during the operation of the steam pipeline, improve the steam leakage monitoring capability, and provide technical support for the stable operation of the nuclear power plant.

[0062] Further, as a refinement and expansion of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another omnidirectional steam leak detection method based on acoustic and optical fusion is provided, such as Figure 5 As shown, the method includes:

[0063] Step 401: The sound sensor acquires audio data of the steam network area in real time, wherein the audio data is two-channel audio data.

[0064] In the above-mentioned embodiment of the present application, the sound sensor acquires the audio data of the steam network area in real time, and the audio data is two-channel audio data, in preparation for subsequent sound detection.

[0065] In step 402, the sound sensor performs filtering processing on the acquired dual-channel audio data to obtain an initial audio sequence, intercepts the normalized initial audio sequence based on a sliding window, obtains multiple target audio sequences, and obtains sound detection values ​​corresponding to the target audio sequences based on a steam leakage sound detection model, wherein the steam leakage sound detection model is established based on a convolutional neural network and a long-short time neural network, and after the target audio sequence is input into the steam leakage sound detection model, the sound detection value is output after passing through a convolutional layer, a pooling layer, a Dropout layer, a Flatten layer, a RepeatVector layer, a LSTM layer, and a Dense layer.

[0066] Next, the sound sensor filters the acquired two-channel audio data to obtain an initial audio sequence. After normalizing the initial audio sequence, the normalized initial audio sequence is intercepted based on a sliding window to obtain multiple target audio sequences. The target audio sequences are input into the steam leak sound detection model to obtain the sound detection value. Specifically, the sound sensor collects the on-site two-channel audio data v in real time. li With v ri After filtering, we get the initial audio sequence V i =[v li ,v ri ] -1 After normalization, the target audio sequence [V1, V2, ... V n ], input the steam leakage sound detection model, and obtain the sound detection value R1 in real time. The steam leakage sound detection model is established based on convolutional neural network and long short-time neural network. After the target audio sequence is input into the steam leakage sound detection model, the sound detection value is output after the convolution layer, pooling layer, Dropout layer, Flatten layer, RepeatVector layer, LSTM layer and Dense layer. The target audio sequence is 2-dimensional data, and the output sound detection value is 1-dimensional data.

[0067] In particular, when training the steam leak sound detection model, two-channel audio signals with steam leaks and two-channel audio signals without steam leaks are collected, and the audio signals are normalized to [0,1]. The labels of audio signals with steam leaks are set to 1, and the labels of audio signals without steam leaks are set to 0. The audio sequence [V1′, V2′, ... V n ′] to conduct model training.

[0068] In particular, the audio sequence used for training [V1′, V2′, ... V n ′] and the audio sequence used for detection [V1, V2, ... V n ] are of the same length.

[0069] Step 403, the microphone sound source array sensor obtains the direction of the sound source that generates audio data in the steam network area in real time, wherein the microphone sound source array sensor is connected to a visual pan-tilt platform, and the horizontal rotation axis of the visual pan-tilt platform coincides with the azimuth axis of the microphone sound source array sensor.

[0070] Step 404, when the sound detection value is greater than a preset sound detection threshold, the visual pan-tilt head rotates based on the sound source orientation of the audio data greater than the preset sound detection threshold, and collects a steam pipe network area detection image along the sound source direction facing after the rotation, divides the steam pipe network area detection image, and obtains multiple steam pipe network area detection image units.

[0071] Step 405: The visual PTZ determines the steam leakage dynamic change area based on the inter-frame difference method and the steam pipe network area detection image unit.

[0072] Step 406: The visual PTZ cuts out a minimum continuous area image containing a steam leakage dynamically changing area in the steam leakage area image, wherein the minimum continuous area image includes multiple frames.

[0073] Step 407, the visual gimbal obtains the image detection result value of each minimum continuous area image of each frame based on the image detection model, and obtains the image detection value based on the average of the image detection result values ​​of a preset number of images among the image detection result values, wherein the image detection model is established based on the YOLOv5 target detection framework.

[0074] Next, the microphone sound source array sensor obtains the sound source orientation of the audio data in the steam pipe network area in real time. When the sound detection value is greater than the preset sound detection threshold, the visual gimbal rotates based on the sound source orientation of the audio data greater than the preset sound detection threshold, and collects the steam pipe network area detection image along the sound source direction after rotation.

[0075] The visual PTZ divides the steam network area detection image to obtain multiple steam network area detection image units, such as Figure 6 As shown, Figure 6 The steam pipe network area detection image M collected in the middle i Divide into m×n parts, and count each area as M i,j ′, using the inter-frame difference method of image processing to calculate the dynamic change area of ​​steam leakage in the image;

[0076] like Figure 7 As shown, the minimum continuous u×v area covering the dynamically changing area in the cropped steam network area detection image is:

[0077] M i,j ′~M i+u-1,j+v-1 ′,

[0078] Get the minimum continuous region image sequence after multi-frame cropping [M1,M2,...M n ].

[0079] The framework of the image detection model adopts the YOLOv5 object detection framework, and inputs an image sequence of n frames of the smallest continuous region images [M1, M2,... M n , and obtains n detection results r i (i = 1, 2, 3... n), calculates the m largest values (m < n) of the r i values, and sorts them in ascending order to obtain r max,i (i = 1, 2, 3... m).

[0080] Finally, the detection result

[0081] That is, the vision pan-tilt obtains the image detection result values of each frame of the smallest continuous region image based on the image detection model. Among the image detection result values, based on the mean value of the previous preset number of image detection result values, the image detection value is obtained.

[0082] Step 408, the data processing host computer calculates the steam leakage detection value according to the sound detection value and the image detection value. When the steam leakage detection value is greater than the preset steam leakage threshold, it sends a steam leakage alarm message, where the steam leakage alarm message includes the detected image of the steam pipe network area, and the detected image of the steam pipe network area includes the detected steam leakage area.

[0083] Next, the data processing host computer calculates the steam leakage detection value according to the sound detection value and the image detection value. When the steam leakage detection value is greater than the preset steam leakage threshold, it sends a steam leakage alarm message.

[0084] By applying the technical solution of this embodiment, the sound sensor acquires the two-channel audio data of the steam pipe network area in real time for filtering processing to obtain an initial audio sequence, and the initial audio sequence after normalization processing is intercepted based on the sliding window to obtain multiple target audio sequences. Based on the steam leakage sound detection model, the sound detection value corresponding to the target audio sequence is obtained, and the microphone sound source array sensor acquires the sound source orientation of the audio data in the steam pipe network area in real time. When the sound detection value is greater than the preset sound detection threshold, the visual pan-tilt head rotates based on the sound source orientation of the audio data greater than the preset sound detection threshold, and collects the steam pipe network area detection image along the sound source direction facing after the rotation, and divides the steam pipe network area detection image to obtain multiple steam pipe network area detection image units. The visual pan-tilt unit determines the dynamic change area of ​​steam leakage based on the inter-frame difference method and the steam pipe network area detection image unit; the visual pan-tilt unit crops the minimum continuous area image containing the dynamic change area of ​​steam leakage in the steam leakage area image, and obtains the image detection result value of each frame of the minimum continuous area image based on the image detection model. In the image detection result value, the image detection value is obtained based on the mean of the previously preset number of image detection result values. The data processing host computer calculates the steam leakage detection value according to the sound detection value and the image detection value. When the steam leakage detection value is greater than the preset steam leakage threshold, a steam leakage alarm information is sent. By fusing the sound information with the visual information, the accuracy and efficiency of pipeline steam leakage detection can be improved.

[0085] Through the description of the above implementation methods, technical personnel in this field can clearly understand that the present application can be implemented by means of software plus necessary general hardware platforms, or by hardware to implement a sound sensor to obtain audio data of the steam pipe network area and calculate a sound detection value; a microphone sound source array sensor obtains the sound source orientation of the audio data source in the steam pipe network area, and the horizontal rotation axis of the visual pan-tilt head coincides with the azimuth axis of the microphone sound source array sensor; when the sound detection value is greater than a preset sound detection threshold, the visual pan-tilt head rotates based on the sound source orientation of the audio data greater than the preset sound detection threshold, and collects a detection image of the steam pipe network area along the direction of the sound source facing the rotation to calculate an image detection value; the data processing host computer calculates the steam leakage detection value based on the sound detection value and the image detection value, and sends a steam leakage alarm message when the steam leakage detection value is greater than the preset steam leakage threshold, thereby improving the accuracy of pipeline steam leakage detection by fusing sound and visual information.

[0086] Those skilled in the art will appreciate that the accompanying drawings are only schematic diagrams of a preferred implementation scenario, and the modules or processes in the accompanying drawings are not necessarily necessary for implementing the present application. Those skilled in the art will appreciate that the modules in the devices in the implementation scenario can be distributed in the devices of the implementation scenario according to the description of the implementation scenario, or can be changed accordingly and located in one or more devices different from the present implementation scenario. The modules of the above-mentioned implementation scenario can be combined into one module, or can be further split into multiple submodules.

[0087] The above serial numbers of this application are only for description and do not represent the advantages and disadvantages of the implementation scenarios. The above disclosure is only a few specific implementation scenarios of this application, but this application is not limited to them, and any changes that can be thought of by technicians in this field should fall within the scope of protection of this application.

Claims

1. A comprehensive steam leak detection method based on acoustic and optical fusion, characterized in that: The method comprises: The sound sensor acquires audio data of the steam pipe network area in real time, and acquires a sound detection value based on the audio data; The microphone sound source array sensor acquires the sound source orientation of the audio data in the steam pipe network area in real time, wherein the microphone sound source array sensor is connected to a visual pan-tilt platform, and the horizontal rotation axis of the visual pan-tilt platform coincides with the orientation axis of the microphone sound source array sensor; When the sound detection value is greater than a preset sound detection threshold, the visual pan / tilt head rotates based on the sound source orientation of the audio data greater than the preset sound detection threshold, and collects a steam pipe network area detection image along the sound source direction facing the rotation, and obtains an image detection value based on the steam pipe network area detection image; The data processing host computer calculates a steam leakage detection value according to the sound detection value and the image detection value, and sends a steam leakage alarm message when the steam leakage detection value is greater than a preset steam leakage threshold value, wherein the steam leakage alarm message includes a collected steam pipe network area detection image, and the steam pipe network area detection image includes a detected steam leakage area.

2. The method according to claim 1, characterized in that The visual pan-tilt platform corresponds to a plurality of preset pan-tilt azimuth angles. The visual pan-tilt platform rotates based on the sound source azimuth of the audio data greater than the preset sound detection threshold, and collects a steam pipe network area detection image along the sound source direction facing after the rotation, including: The visual pan-tilt head determines a pan-tilt target rotation azimuth angle based on the sound source azimuth of the audio data greater than a preset sound detection threshold value in the corresponding plurality of pan-tilt preset azimuth angles, and rotates to the pan-tilt target rotation azimuth angle; After the visual pan-tilt head rotates to the pan-tilt head target rotation azimuth angle, the steam pipe network area detection image is collected along the direction of the sound source facing after the rotation, wherein the pan-tilt head preset azimuth angle is obtained by averaging the preset angle based on the horizontal plane where the visual pan-tilt head is located.

3. The method according to claim 1, characterized in that The data processing host computer calculates the steam leakage detection value according to the sound detection value and the image detection value, including: The data processing host computer calculates the steam leakage detection value based on the steam leakage detection value calculation formula, the sound detection value and the image detection value, wherein the steam leakage detection value calculation formula is: R=ωR1+R2, R is the steam leakage detection value, ω is the preset fusion coefficient, R1 is the sound detection value, and R2 is the image detection value.

4. The method according to claim 1, characterized in that The audio data is two-channel audio data, and the sound sensor obtains a sound detection value based on the audio data, including: The sound sensor filters the acquired dual-channel audio data to obtain an initial audio sequence, intercepts the normalized initial audio sequence based on a sliding window, obtains multiple target audio sequences, and obtains sound detection values ​​corresponding to the target audio sequences based on a steam leak sound detection model.

5. The method according to claim 4, characterized in that The steam leakage sound detection model is established based on a convolutional neural network and a long-short time neural network. After the target audio sequence is input into the steam leakage sound detection model, the sound detection value is output after passing through a convolutional layer, a pooling layer, a Dropout layer, a Flatten layer, a RepeatVect or layer, a LSTM layer and a Dense layer.

6. The method according to any one of claims 1 to 5, characterized in that The visual PTZ detects an image based on the steam pipe network area and obtains an image detection value, including: The visual PTZ divides the steam pipe network area detection image to obtain a plurality of steam pipe network area detection image units; The visual PTZ determines the dynamic change area of ​​steam leakage based on the inter-frame difference method and the steam pipe network area detection image unit; The visual gimbal crops the minimum continuous area image containing the dynamically changing area of ​​steam leakage in the steam leakage area image, and obtains the image detection value corresponding to the minimum continuous area image based on the image detection model, wherein the image detection model is established based on the YOLOv5 target detection framework.

7. The method according to claim 6, characterized in that The minimum continuous area image includes multiple frames, and the visual gimbal obtains an image detection value corresponding to the minimum continuous area image based on an image detection model: The visual gimbal obtains the image detection result value of each minimum continuous area image of each frame based on the image detection model; The visual gimbal obtains an image detection value based on an average of a preset number of image detection result values ​​in the image detection result values.

Images