Steam pipeline leakage detection method and device based on high-definition infrared, and storage medium

Abstract

The invention discloses a steam pipeline leakage detection method and device based on high-definition infrared, a storage medium and computer equipment. The method comprises the steps that a high-definition infrared dual-mode video corresponding to a to-be-detected steam pipeline is acquired; determining multiple frames of target high-definition images from the high-definition infrared dual-mode video, inputting the multiple frames of target high-definition images into a preset leakage detection model, and determining a first leakage area according to a detection result; determining multiple frames of target infrared images from the high-definition infrared dual-mode video, constructing a target background image based on a first frame in the multiple frames of target infrared images, and determining a second leakage area according to the target background image and other frames in the multiple frames of target infrared images; and performing registration processing on the first leakage area and the second leakage area of the to-be-detected steam pipeline, and determining a target leakage area corresponding to the to-be-detected steam pipeline according to a registered overlapping area.

Description

Technical Field

[0001] The present application relates to the technical field of pipeline leakage detection, and in particular to a steam pipeline leakage detection method and device based on high-definition infrared, a storage medium, and a computer device. Background Art

[0002] The growing demand for energy in modern society has promoted the use of thermal power plants, nuclear power plants and various heat conversion devices. Steam pipes are necessary equipment for large energy facilities. In daily operation, steam leakage accidents are difficult to avoid due to factors such as condensation, vibration and sealing. Leakage in steam pipes will inevitably cause huge energy waste and even cause safety problems such as scalding, explosion and radiation leakage. Therefore, leak detection of steam pipes is of great significance.

[0003] In the prior art, when leak detection is performed on steam pipelines, either manual on-site inspections are performed, or fixed-point sensor groups such as pressure flow meters and acoustic emission leak detection equipment are installed on the steam pipelines for leak detection. For manual on-site inspections, the leak detection efficiency is low and the risk is relatively high; for the method of installing fixed-point sensor groups on steam pipelines, it is difficult to accurately locate the leak position, and the detection cost per unit area is relatively high. Summary of the invention

[0004] In view of this, the present application provides a steam pipeline leakage detection method and device based on high-definition infrared, a storage medium, and a computer device. The leakage area of ​​the steam pipeline is determined by high-definition images and infrared images. Whether there is a leak in the steam pipeline can be determined without manual on-site inspection. At the same time, the leakage area of ​​the steam pipeline is determined by the two methods, which can further improve the accuracy of the leakage area determination. In addition, the leakage area can be accurately determined by a device capable of collecting high-definition infrared dual-mode video and a computing device, which can greatly reduce the detection cost.

[0005] According to one aspect of the present application, a steam pipeline leakage detection method based on high-definition infrared is provided, comprising:

[0006] Obtain high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected;

[0007] Determine a plurality of target high-definition images from the high-definition infrared dual-mode video, input the plurality of target high-definition images into a preset leakage detection model, and determine a first leakage area according to the detection result;

[0008] Determine a plurality of target infrared images from the high-definition infrared dual-mode video, construct a target background image based on a first frame of the plurality of target infrared images, and determine a second leakage area according to the target background image and the remaining frames of the plurality of target infrared images;

[0009] The first leakage area and the second leakage area of ​​the steam pipe to be detected are registered, and the target leakage area corresponding to the steam pipe to be detected is determined according to the registered overlapping area.

[0010] According to another aspect of the present application, a steam pipeline leakage detection device based on high-definition infrared is provided, comprising:

[0011] A video acquisition module, used to acquire high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected;

[0012] A first leakage area determination module is used to determine multiple frames of target high-definition images from the high-definition infrared dual-mode video, and input the multiple frames of target high-definition images into a preset leakage detection model, and determine a first leakage area according to the detection results;

[0013] A second leakage area determination module is used to determine multiple frames of target infrared images from the high-definition infrared dual-mode video, and construct a target background image based on the first frame of the multiple frames of target infrared images, and determine a second leakage area according to the target background image and the remaining frames of the multiple frames of target infrared images;

[0014] The registration module is used to perform registration processing on the first leakage area and the second leakage area of ​​the steam pipe to be detected, and determine the target leakage area corresponding to the steam pipe to be detected according to the overlapped area after registration.

[0015] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned steam pipe leakage detection method based on high-definition infrared is implemented.

[0016] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the above-mentioned high-definition infrared-based steam pipe leakage detection method when executing the program.

[0017] By means of the above technical scheme, the present application provides a steam pipe leakage detection method and device based on high-definition infrared, a storage medium, and a computer device. First, a high-definition infrared dual-mode video corresponding to the steam pipe to be detected is obtained. Multiple frames of high-quality, representative images are selected from the high-definition infrared dual-mode video as target high-definition images, and then these target high-definition images are sequentially input into the preset leakage detection model to obtain the detection results corresponding to each frame of the target high-definition image, and the first leakage area is determined according to these detection results. Next, multiple frames of target infrared images are extracted from the high-definition infrared dual-mode video, and a target background image is constructed using the first frame of these target infrared images as a reference. Then, based on the target background image, the infrared radiation changes caused by the leakage are identified from the remaining frames except the first frame, thereby determining the second leakage area. The two areas can be further registered so that they can accurately correspond to the same physical position. Finally, the target leakage area corresponding to the steam pipe to be detected is determined according to the overlapped area after registration. The embodiment of the present application determines the leakage area of ​​the steam pipeline by using high-definition images and infrared images, and can determine whether the steam pipeline has a leak without manual on-site inspection. At the same time, the leakage area of ​​the steam pipeline is determined by using two methods to further improve the accuracy of the leakage area determination. In addition, the leakage area can be accurately determined by using equipment that can collect high-definition infrared dual-mode video and computing equipment, which can greatly reduce the detection cost.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A schematic diagram of a process of a steam pipeline leakage detection method based on high-definition infrared provided in an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of a process of another steam pipeline leakage detection method based on high-definition infrared provided in an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram showing a dual-mode video acquisition process provided by an embodiment of the present application is shown;

[0023] Figure 4A schematic diagram of a registered image provided by an embodiment of the present application is shown;

[0024] Figure 5 A schematic structural diagram of a steam pipe leakage detection device based on high-definition infrared provided in an embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0027] In this embodiment, a steam pipeline leakage detection method based on high-definition infrared is provided. Figure 1 As shown, the method includes:

[0028] Step 101, obtaining a high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected.

[0029] The embodiment of the present application provides a steam pipe leakage detection method based on high-definition infrared, which can detect the leakage area on the steam pipe from both high-definition images and infrared images. Before performing steam pipe leakage detection, first, a high-definition infrared dual-mode video corresponding to the steam pipe to be detected can be obtained. This video not only contains traditional visible light video information, but also contains infrared radiation information, so that the detection can use two different physical properties at the same time. The high-definition infrared dual-mode video can be obtained by shooting with a high-definition infrared dual-mode camera device.

[0030] Step 102, determining multiple frames of target high-definition images from the high-definition infrared dual-mode video, and inputting the multiple frames of target high-definition images into a preset leakage detection model, and determining a first leakage area according to the detection results.

[0031] In this embodiment, multiple frames of high-quality, representative images are selected from the high-definition infrared dual-mode video as target high-definition images. These target high-definition images are then sequentially input into a preset leakage detection model. The model can be a model trained based on a machine learning or deep learning algorithm that can analyze features in a high-definition image and determine the first leakage area based on these features.

[0032] Step 103, determine multiple frames of target infrared images from the high-definition infrared dual-mode video, and construct a target background image based on the first frame of the multiple frames of target infrared images, and determine the second leakage area based on the target background image and the remaining frames of the multiple frames of target infrared images.

[0033] In this embodiment, multiple frames of target infrared images are extracted from the high-definition infrared dual-mode video. A target background image is constructed using the first frame of these target infrared images as a reference. This target background image represents the infrared radiation state of the pipeline when there is no leakage. Then, based on the target background image, the infrared radiation changes caused by the leakage are identified from the remaining frames except the first frame, thereby determining the second leakage area.

[0034] Step 104 , registering the first leakage region and the second leakage region of the steam pipe to be detected, and determining the target leakage region corresponding to the steam pipe to be detected according to the registered overlapping region.

[0035] In this embodiment, since the first leakage area and the second leakage area are determined based on the high-definition image and the infrared image respectively, there may be a positional deviation or inconsistency between the two. Therefore, the two areas can be further registered, that is, their positions and sizes can be adjusted so that they can accurately correspond to the same physical position. Finally, according to the overlapped area after registration, the target leakage area corresponding to the steam pipe to be detected is determined. This area is the leakage point confirmed by the two methods, which has high reliability and accuracy.

[0036] By applying the technical solution of this embodiment, first, a high-definition infrared dual-mode video corresponding to the steam pipe to be detected is obtained. Multiple frames of high-quality and representative images are selected from the high-definition infrared dual-mode video as target high-definition images, and then these target high-definition images are sequentially input into the preset leakage detection model to obtain the detection results corresponding to each frame of the target high-definition image, and the first leakage area is determined according to these detection results. Next, multiple frames of target infrared images are extracted from the high-definition infrared dual-mode video, and a target background image is constructed using the first frame of these target infrared images as a reference. Then, based on the target background image, the infrared radiation changes caused by the leakage are identified from the remaining frames except the first frame, so as to determine the second leakage area. The two areas can be further registered so that they can accurately correspond to the same physical position. Finally, according to the overlapped area after registration, the target leakage area corresponding to the steam pipe to be detected is determined. The embodiment of the present application jointly determines the leakage area of ​​the steam pipe by high-definition images and infrared images, and can determine whether the steam pipe has a leak without manual on-site inspection. At the same time, the leakage area of ​​the steam pipe is jointly determined by two methods, which can further improve the accuracy of the leakage area determination. In addition, the leakage area can be accurately determined by using equipment that can capture high-definition infrared dual-mode video and computing equipment, which can greatly reduce the detection cost.

[0037] Further, as a refinement and extension of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another steam pipeline leakage detection method based on high-definition infrared is provided, such as Figure 2 As shown, the method includes:

[0038] Step 201, obtaining a high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected.

[0039] In an embodiment of the present application, optionally, the steam pipe to be inspected is located in a target area, a detection track and a mobile detection device are provided in the target area, the mobile detection device is configured to move along the detection track, a plurality of detection points are provided on the detection track, the mobile detection device is equipped with an infrared camera and a high-definition camera, and the high-definition infrared dual-mode video corresponding to the steam pipe to be inspected is captured by the mobile detection device at the detection points on the detection track.

[0040] In this embodiment, the target area may be a specific area where steam pipe leakage detection is required, and one or more steam pipes may be included in this area. The detection track may be one or more paths set in the target area, and the mobile detection device may move along these paths for comprehensive detection. The setting of the detection track may be determined according to factors such as the layout of the steam pipe and the detection efficiency. The mobile detection device is a device that can move autonomously along the detection track, has high flexibility and adaptability, and can work under different environmental conditions. The mobile detection device may be equipped with an infrared camera and a high-definition camera, which are used to capture infrared radiation information and visible light information of the steam pipe, respectively. The infrared camera is very sensitive to detecting temperature anomalies caused by leakage, while the high-definition camera can provide detailed visual information of the pipeline. Among them, the infrared camera collects infrared band images, and the infrared band image collection method may be to use an infrared lamp to emit infrared rays of multiple frequency bands, and after being absorbed by the leaking steam and reflected by the environment, the infrared light passes through a specific narrow-band filter to reach the broadband refrigeration infrared core to obtain the corresponding infrared spectrum grayscale image. In addition, a plurality of detection points are set on the detection track, and these detection points are the positions where the mobile detection device needs to stay and shoot videos during the movement. The setting of the detection points should be based on the actual situation of the steam pipeline and the detection requirements to ensure that possible leakage points can be fully covered. When the mobile detection device moves to each detection point along the detection track, it can use an infrared camera and a high-definition camera to shoot a few seconds of dual-mode video of the steam pipeline at the detection point based on the set posture. The shooting time can be determined according to the needs. In this way, each detection point can correspond to a high-definition infrared dual-mode video containing infrared radiation information and visible light information. These dual-mode videos are then used for subsequent leak detection analysis to identify and locate the leakage points of the steam pipeline by comparing information such as changes in infrared radiation and pipeline status under visible light. The embodiment of the present application sets the target area, detection track and detection point, and uses a mobile detection device equipped with an infrared camera and a high-definition camera for shooting, thereby achieving comprehensive, efficient and real-time leakage detection of the steam pipeline. This method not only improves the efficiency, accuracy and reliability of detection, but also reduces the labor intensity and cost of manual detection.

[0041] In addition, a communication device may be installed in the mobile detection device, which may be a multi-frequency WiFi module, etc. The multi-frequency WiFi module may adopt multi-band wireless communication technology, adopt high-frequency radio wave communication in areas with greater fog interference, and adopt low-frequency communication in narrow areas with more obstacles. Specifically, different communication frequency bands may be set for different detection points in advance. When the mobile detection device moves along the detection track, when it reaches a certain detection point, the preset communication frequency band of the point is first determined, and then the high-definition infrared dual-mode video is sent to the server using the preset communication frequency band. The server executes the subsequent leakage area detection steps.

[0042] In one embodiment, if Figure 3 As shown, the mobile detection device can be composed of a four-arm crawler mobile robot, on which a dual-spectrum monitoring platform is installed. The dual-spectrum monitoring platform consists of a high-definition camera and an infrared camera, and an infrared lamp is installed on the infrared camera, and the infrared lamp is coaxial with the infrared camera lens. The infrared camera in the dual-spectrum monitoring platform collects images of the leaking heat source, and the high-definition camera obtains high-definition images of the leaking site. The mobile robot can move along the detection track, such as Figure 3 As shown, the robot can first move to the detection point corresponding to device 1, shoot a dual-mode video, and then move to the detection point corresponding to device 2 to shoot another dual-mode video. After shooting a dual-mode video each time, the mobile robot can send the dual-mode video to the server through the wireless base station via the communication device, and the server will perform subsequent data analysis operations.

[0043] Step 202, determining multiple frames of target high-definition images from the high-definition infrared dual-mode video, and inputting the multiple frames of target high-definition images into a preset leakage detection model, and determining a first leakage area according to the detection results.

[0044] In this embodiment, after each frame of the target high-definition image is input into the preset leakage detection model, the leakage area corresponding to the frame of the target high-definition image can be identified. Finally, from the leakage areas corresponding to these target high-definition images, the target high-definition image with the largest leakage area is found, and the leakage area corresponding to the target high-definition image is used as the first leakage area.

[0045] Step 203, determining multiple frames of target infrared images from the high-definition infrared dual-mode video, and obtaining a leak-free background image corresponding to the steam pipe to be detected.

[0046] In this embodiment, after determining multiple frames of target infrared images, a non-leakage background image corresponding to the steam pipeline to be detected can also be obtained, that is, an infrared image of the steam pipeline without leakage is obtained. Usually, this image is taken when the pipeline is operating normally and no leakage is detected. This non-leakage background image can be used as a benchmark for subsequent analysis to compare and identify changes in infrared radiation caused by leakage.

[0047] Step 204 , constructing a target background image through the first frame target infrared image and the non-leakage background image, and fine-tuning the preset hybrid model based on the target background image to obtain a target hybrid model.

[0048] In this embodiment, further, the first frame of the target infrared image in the multiple frames of the target infrared image (usually a frame taken at the beginning of the detection) is combined with the leakage-free background image to construct a target background image. This target background image can be a fusion of the information of the first frame of the target infrared image and the leakage-free background image in some way (such as weighted averaging, image fusion, etc.) to more accurately reflect the infrared radiation state of the pipeline under the current environmental conditions. Then, based on this target background image, the preset hybrid model is fine-tuned. Specifically, the preset hybrid model can be a mathematical model for analyzing the grayscale distribution of pixels in an infrared image, which can distinguish normal infrared radiation from abnormal radiation caused by leakage by learning the features of the target background image and the current frame target infrared image. The target hybrid model obtained by fine-tuning can better adapt to the current detection scene and improve the accuracy of detection.

[0049] In one embodiment, the target mixture model can be expressed as follows:

[0050]

[0051] Where K is the number of Gaussian distributions included in the target mixture model, and its size is determined by the number of modes of the background and foreground in the image and the computing power used, and is generally 3 to 5; ω i,t is the weight of the Gaussian distribution corresponding to the target infrared image of the tth frame of the i-th model in the target mixture model, X t is the pixel in the target infrared image of the tth frame, μ i,t is the average value of the Gaussian distribution corresponding to the target infrared image of the tth frame in the target mixture model, Σ i,t is the covariance matrix of the Gaussian distribution corresponding to the target infrared image of the tth frame in the target mixture model, and η is the probability density function of the Gaussian distribution.

[0052] Step 205, for each frame of the remaining target infrared images, based on any pixel in the image, calculate the first grayscale distribution corresponding to the any pixel through the target mixing model, and based on the first grayscale distribution corresponding to the any pixel and the second grayscale distribution determined for the any pixel based on the image, calculate the distribution difference between the first grayscale distribution and the second grayscale distribution, and determine whether the any pixel is a target pixel based on the distribution difference.

[0053] In this embodiment, for each frame of the remaining target infrared images, any pixel in the image is selected for analysis. Specifically, the fine-tuned target hybrid model can be used to calculate the first grayscale distribution corresponding to the pixel. This distribution reflects the grayscale value prediction of the target hybrid model for the pixel in a normal (or background) state. At the same time, based on the actual grayscale value of the pixel in the current image, its second grayscale distribution is determined. This distribution can be obtained by statistically analyzing the grayscale values ​​of the pixels around the pixel, reflecting the actual grayscale characteristics of the pixel in the current image. Afterwards, the distribution difference between the first grayscale distribution and the second grayscale distribution is calculated. This distribution difference quantifies the inconsistency between the target hybrid model prediction and the actual observation, which may be caused by changes in infrared radiation caused by leakage. According to the size of the distribution difference, it can be determined whether the pixel is a target pixel (i.e., an abnormal pixel that may be caused by leakage). For example, if the distribution difference exceeds a preset threshold, it is considered that the pixel may be a target pixel. Specifically, the distribution difference between the first grayscale distribution and the second grayscale distribution can be calculated using the Wasserstein distance formula.

[0054] Step 206 , obtaining a second leakage area according to the target pixel points identified from each frame of the remaining target infrared images.

[0055] In this embodiment, by performing the above analysis on all pixels in each frame of the target infrared image, the target pixels in each frame can be identified. Then, the target pixels identified in all frames are summarized and marked, and finally a second leakage area is obtained. This area may contain one or more infrared radiation abnormal areas caused by leakage, and is the focus of subsequent further inspection and repair.

[0056] The embodiment of the present application uses a non-leakage background image as a benchmark and compares and analyzes the target infrared image taken in real time, so that the infrared radiation change caused by the leakage can be more accurately identified. This method based on background contrast can reduce the interference of environmental factors (such as temperature fluctuations, light changes, etc.) on the detection results. In addition, the embodiment of the present application introduces a hybrid model and adapts it to the current detection scene by fine-tuning. This adaptive mechanism enables the detection method to be adjusted according to different environmental conditions (such as pipeline layout, ambient temperature, etc.), thereby improving the flexibility and accuracy of detection.

[0057] Step 207: performing registration processing on the first leakage area and the second leakage area of ​​the steam pipeline to be detected.

[0058] In this embodiment, the VoxelMorph algorithm can be used to register the first leakage area and the second leakage area, with the target high-definition image corresponding to the first leakage area as the fixed image and the target infrared image corresponding to the second leakage area as the image to be registered. The fixed image and the image to be registered are transformed into 3D vectors through dimensional expansion, and then the fixed image and the image to be registered are sent to U-net for encoding and decoding to generate a deformation field. According to the deformation field, the image to be registered is reshaped and interpolated by the spatial transformation network to obtain the registered image. Figure 4 As shown, a schematic diagram of the image after registration is given. Figure 4 In the figure, the leakage region C and the leakage region A are the first leakage region, and the leakage region B and the leakage region D are the second leakage region.

[0059] Step 208: for any overlapped region after registration, determine a first sub-region to which the overlapped region belongs from the first leakage region, and determine a second sub-region to which the overlapped region belongs from the second leakage region.

[0060] In this embodiment, there may be some overlapping areas in the registered image, and these overlapping areas are covered by both detection methods (i.e., high-definition image acquisition and infrared image acquisition). For any overlapping area, the first sub-area to which the area belongs is first determined from the first leakage area, and similarly, the second sub-area to which the area belongs is also determined from the second leakage area. Figure 4 As shown, the overlapping area is A∩B, then the first sub-area is the leakage area A, and the second sub-area is the leakage area B.

[0061] Step 209, calculating the total area of ​​the first sub-region and the second sub-region, and calculating the area intersection-over-combination ratio between the first sub-region and the second sub-region according to the area corresponding to any overlapping region and the total area.

[0062] In this embodiment, the total area of ​​the region refers to the sum of the areas of the first sub-region and the second sub-region. Figure 4 In , the total area of ​​the region can be the area of ​​A∪B. The intersection over union (IoU) is an indicator to measure the degree of overlap between two regions, and the calculation formula is the ratio of the intersection area of ​​the two regions to the area of ​​their union. In this step, the intersection over union ratio of the first sub-region and the second sub-region can be calculated to evaluate the degree of overlap between them.

[0063] Step 210, when the area intersection-over-union ratio is greater than a preset intersection-over-union ratio threshold, determine the proportion of any overlapping area in the registered image, and when the proportion is greater than a preset proportion threshold, use any overlapping area as the target leakage area corresponding to all steam pipes to be detected.

[0064] In this embodiment, when the region IoU is greater than the preset IoU threshold, it indicates that there is a high degree of overlap between the two sub-regions, which may be a real leakage region. However, IoU alone is not enough to determine the target leakage region, because there may be some small, unimportant overlapping regions. Therefore, the proportion of any overlapping region in the registered image can be further determined. If this proportion is greater than the preset proportion threshold, then the overlapping region can be considered important and used as the target leakage region corresponding to the steam pipe to be detected.

[0065] The embodiment of the present application determines the leakage area of ​​the steam pipe more accurately by combining the results of multiple detection methods. By calculating the intersection and union ratio and judging the ratio, some unimportant or false overlapping areas can be excluded, thereby improving the accuracy and reliability of the detection.

[0066] In the embodiment of the present application, optionally, step 206 includes:

[0067] Step 206 - 1 : Based on a target high-definition image frame used to determine a first leakage area among the multiple frames of target high-definition images, determine a first acquisition time corresponding to the target high-definition image frame.

[0068] Step 206-2, determining the second acquisition time corresponding to each frame in the multiple frames of target infrared images, and finding the target second acquisition time having the smallest time difference with the first acquisition time from the second acquisition time.

[0069] Step 206-3, taking the target infrared image frame corresponding to the second acquisition time of the target as the reference image, and marking the leakage area indicated by the target pixel points on the reference image according to the target pixel points identified from each frame of the remaining target infrared images, performing morphological opening and closing operations on the reference image with the leakage area marked, and obtaining the second leakage area.

[0070] In this embodiment, among multiple frames of target high-definition images, the frame of high-definition image used to determine the first leakage area is first found, which is called the "target high-definition image frame", and the first acquisition time corresponding to the target high-definition image frame is determined. It should be noted that after each frame of target high-definition image is input into the preset leakage detection model, a corresponding recognition result can be obtained. Here, the target high-definition image with the largest leakage area in the recognition result can be used as the target high-definition image frame. Next, consider multiple frames of target infrared images. Each frame of target infrared image has a corresponding second acquisition time. From the acquisition time of these target infrared images, find the target second acquisition time with the smallest time difference with the first acquisition time of the target high-definition image frame, and use the target infrared image frame corresponding to the target second acquisition time as the reference image. Since the target high-definition image frame is closest to the target infrared image frame in time, the steam pipe states observed by them are also the most similar. Afterwards, the target pixel points identified from other target infrared images are mapped to the reference image, and the leakage areas indicated by these pixel points are marked accordingly on the reference image. Perform morphological opening and closing operations on the reference image marked with the leakage area. Morphological opening operations are usually used to remove small foreground objects (such as noise) and smooth the boundaries of larger objects while keeping their areas unchanged. Morphological closing operations are used to fill small holes or black areas in foreground objects and smooth their boundaries. By combining opening operations and closing operations (i.e., morphological opening and closing operations), the leakage area in the image can be further cleaned and enhanced to make it clearer and more accurate. After the reference image processed by the morphological opening and closing operations, the leakage area will be clearer and more prominent. This processed area is the second leakage area, which provides a more reliable basis for the detection of steam pipe leakage. The embodiment of the present application can ensure that the steam pipe state observed by both is as consistent as possible by finding the target infrared image frame closest to the target high-definition image frame time as the reference image, so that when the first leakage area and the second leakage area are subsequently registered, the registration result can be made more accurate, thereby improving the robustness of the detection result. In addition, the morphological opening and closing operations further clean up and enhance the leakage area in the image, so that it is not affected by interference factors such as noise and small holes, and further improves the detection accuracy.

[0071] In an embodiment of the present application, optionally, before "inputting the multi-frame target high-definition images into a preset leakage detection model" in step 202, the method also includes: obtaining a smoke sample image, and performing model training on an initial detection model based on the smoke sample image to obtain a target pre-trained detection model; obtaining a steam pipe leakage sample image, and fine-tuning the target pre-trained detection model based on the steam pipe leakage sample image through a freezing training strategy to obtain the preset leakage detection model.

[0072] In this embodiment, smoke sample images can be obtained first. Smoke sample images are a set of images containing smoke, which can come from various sources, such as public data sets, laboratory simulations, or actual field shooting. The selection of smoke sample images is as diverse as possible to cover different smoke types, concentrations, lighting conditions, etc., so that the trained model has better generalization ability. Then, the initial detection model is trained based on the smoke sample images. Specifically, the initial detection model can be a deep learning model, such as a convolutional neural network (CNN), YOLO, SSD, etc., which perform well in image detection tasks. After that, the initial detection model is trained using the collected smoke sample images. During the training process, the initial detection model can learn the characteristics of the smoke, such as shape, texture, color, etc., so as to be able to distinguish between smoke and non-smog areas. After multiple iterative training, the initial detection model gradually converges and has good smoke detection capabilities. The model obtained at this time is called a target pre-training detection model.

[0073] Secondly, steam pipe leakage sample images can be obtained. The steam pipe leakage sample images are a set of images containing steam pipe leakage areas, and the leakage areas on these images are marked. The steam pipe leakage sample images include images covering different leakage types, pipeline layouts, lighting conditions, etc. After that, the target pre-trained detection model is fine-tuned using the steam pipe leakage sample images through the frozen training strategy. Fine-tuning is a commonly used model optimization method that uses a model that has been trained on a similar task as a starting point and fine-tunes it on a dataset of a new task. In this process, most of the layers of the target pre-trained detection model (such as the previous convolutional layers) can be frozen first. These layers have learned general image features and are still useful for new tasks. Then, only the latter layers of the model (such as fully connected layers or specific detection layers) are adjusted, which are more focused on specific detection tasks. The model is fine-tuned using the steam pipe leakage sample images to allow the model to learn the features of the leakage area. Since most of the layers have been frozen, the fine-tuning process is usually much faster than training from scratch, and only requires fewer steam pipe leakage sample images to achieve it. After fine-tuning, the model will be more suitable for the detection task of steam pipe leakage, and the model obtained at this time is the preset leakage detection model. The embodiment of the present application first trains a general smoke detection model, and then uses the steam pipe leakage sample image for fine-tuning, and finally obtains a model specifically for steam pipe leakage detection. This method makes full use of the idea of ​​transfer learning, which not only saves training time but also improves the detection performance of the model. In addition, by using the smoke sample image to train the initial detection model, the number of leakage images required to be collected for the training model is reduced, and only a small number of steam pipe leakage sample images are required to achieve the determination of the final preset leakage detection model, and the determined preset leakage detection model has a good ability to identify leakage areas under smoke, making the preset leakage detection model more suitable for the application scenario of the present application.

[0074] In an embodiment of the present application, optionally, the method also includes: after detecting the first leakage area and / or the second leakage area, generating alarm information based on the first leakage area and / or the second leakage area, and the corresponding detection points, and sending the alarm information to a preset terminal.

[0075] In this embodiment, if the first leakage area and / or the second leakage area is detected in the dual-mode video collected at a certain detection point, then at this time, an alarm message can be generated based on the detected first leakage area, the second leakage area and the corresponding detection point, and the generated alarm message is sent to the user of the preset terminal. These preset terminal users may be engineers, operators or managers responsible for maintaining the steam pipeline. The alarm information can be sent in various ways, including but not limited to: SMS notifications, e-mails, etc. The embodiment of the present application uses automated image processing and machine learning technology to quickly and accurately detect leakage areas on steam pipelines, and generates alarm information in the form of alarms to promptly communicate to relevant personnel so that they can quickly take measures to deal with them, thereby ensuring the safe operation of the steam pipeline.

[0076] In an embodiment of the present application, optionally, after step 203, the method further includes: constructing a target background image through a first frame target infrared image and the leakage-free background image; for each frame of the remaining target infrared images, based on each pixel in the image, calculating a first grayscale value corresponding to the pixel, and calculating a second grayscale value of the pixel in the target background image, and judging whether the pixel is a target pixel based on a grayscale difference between the first grayscale value and the second grayscale value; and obtaining a second leakage area based on the target pixel identified from each frame of the remaining target infrared images.

[0077] In this embodiment, another method for determining the second leakage area is provided. First, the first frame of the target infrared image (usually a frame taken at the beginning of the detection) in the multiple frames of the target infrared image is combined with the non-leakage background image to construct a target background image. This target background image can be a fusion of the information of the first frame of the target infrared image and the non-leakage background image in some way (such as weighted average, image fusion, etc.) to more accurately reflect the infrared radiation state of the pipeline under the current environmental conditions. Then, for each subsequent frame of the target infrared image (i.e., all frames except the first frame), the following operations are performed: for each pixel point in each frame of the target infrared image, its grayscale value in the current frame is calculated, and this value is called the "first grayscale value". Similarly, the pixel point at the corresponding position is found in the previously constructed target background image, and its grayscale value is calculated, and this value is called the "second grayscale value". Then, by comparing the difference between its first grayscale value and the second grayscale value (i.e., grayscale difference), it is determined whether the pixel point belongs to the target (i.e., the possible leakage area). If the grayscale difference exceeds a preset difference threshold, it can be considered that the pixel has changed significantly, which may be caused by leakage, so it is marked as a target pixel. By performing the above analysis on all pixels in each frame of the target infrared image, the target pixel in each frame can be identified. Then, the target pixels identified in all frames are summarized and marked, and finally the second leakage area is obtained. This area may contain one or more abnormal infrared radiation areas caused by leakage, which is the focus of further inspection and repair.

[0078] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a steam pipeline leakage detection device based on high-definition infrared, such as Figure 5 As shown, the device comprises:

[0079] A video acquisition module, used to acquire high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected;

[0080] A first leakage area determination module is used to determine multiple frames of target high-definition images from the high-definition infrared dual-mode video, and input the multiple frames of target high-definition images into a preset leakage detection model, and determine a first leakage area according to the detection results;

[0081] A second leakage area determination module is used to determine multiple frames of target infrared images from the high-definition infrared dual-mode video, and construct a target background image based on the first frame of the multiple frames of target infrared images, and determine a second leakage area according to the target background image and the remaining frames of the multiple frames of target infrared images;

[0082] The registration module is used to perform registration processing on the first leakage area and the second leakage area of ​​the steam pipe to be detected, and determine the target leakage area corresponding to the steam pipe to be detected according to the overlapped area after registration.

[0083] Optionally, the second leakage area determination module is used to:

[0084] Acquire a non-leakage background image corresponding to the steam pipeline to be detected;

[0085] A target background image is constructed through the first frame target infrared image and the non-leakage background image, and a preset hybrid model is fine-tuned based on the target background image to obtain a target hybrid model;

[0086] For each frame of the remaining target infrared images, based on any pixel in the image, a first grayscale distribution corresponding to the any pixel is calculated by the target mixture model, and based on the first grayscale distribution corresponding to the any pixel and a second grayscale distribution determined for the any pixel based on the image, a distribution difference between the first grayscale distribution and the second grayscale distribution is calculated, and whether the any pixel is a target pixel is determined based on the distribution difference;

[0087] A second leakage area is obtained according to the target pixel points identified from each frame of the remaining frames of target infrared images.

[0088] Optionally, the second leakage area determination module is further used to:

[0089] Based on a target high-definition image frame used to determine a first leakage area among the multiple frames of target high-definition images, determining a first acquisition time corresponding to the target high-definition image frame;

[0090] Determine a second acquisition time corresponding to each frame in the multiple frames of target infrared images, and find out the target second acquisition time with the smallest time difference with the first acquisition time from the second acquisition time;

[0091] The target infrared image frame corresponding to the second acquisition time of the target is used as the reference image, and according to the target pixel points identified from each frame of the remaining target infrared images, the leakage area indicated by the target pixel points is marked on the reference image, and a morphological opening and closing operation is performed on the reference image with the leakage area marked to obtain the second leakage area.

[0092] Optionally, the device further comprises:

[0093] A pre-training module, used for obtaining a smoke sample image before inputting the multiple frames of target high-definition images into a preset leakage detection model, and performing model training on an initial detection model based on the smoke sample image to obtain a target pre-training detection model;

[0094] The fine-tuning module is used to obtain a steam pipe leakage sample image, and fine-tune the target pre-trained detection model based on the steam pipe leakage sample image through a freezing training strategy to obtain the preset leakage detection model.

[0095] Optionally, the registration module is used to:

[0096] For any overlapped area after registration, determining a first sub-area to which the overlapped area belongs from the first leakage area, and determining a second sub-area to which the overlapped area belongs from the second leakage area;

[0097] Calculating the total area of ​​the first sub-region and the second sub-region, and calculating the area intersection-over-combination ratio between the first sub-region and the second sub-region according to the area corresponding to any overlapping region and the total area;

[0098] When the area intersection-and-union ratio is greater than a preset intersection-and-union ratio threshold, the proportion of any overlapping area in the registered image is determined, and when the proportion is greater than a preset proportion threshold, any overlapping area is used as the target leakage area corresponding to all steam pipes to be detected.

[0099] Optionally, the steam pipe to be inspected is located in a target area, a detection track and a mobile detection device are provided in the target area, the mobile detection device is configured to move along the detection track, a plurality of detection points are provided on the detection track, the mobile detection device is equipped with an infrared camera and a high-definition camera, and the high-definition infrared dual-mode video corresponding to the steam pipe to be inspected is captured by the mobile detection device at the detection points on the detection track.

[0100] Optionally, the device further comprises:

[0101] The alarm module is used to generate alarm information based on the first leakage area and / or the second leakage area and the corresponding detection points after detecting the first leakage area and / or the second leakage area, and send the alarm information to a preset terminal.

[0102] It should be noted that for other corresponding descriptions of the functional units involved in the steam pipeline leakage detection device based on high-definition infrared provided in the embodiment of the present application, reference can be made to Figures 1 to 4 The corresponding description in the method will not be repeated here.

[0103] The present application also provides a computer device, which may be a personal computer, a server, a network device, etc. Figure 6 As shown, the computer device includes a bus, a processor, a memory and a communication interface, and may also include an input and output interface and a display device. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in each method embodiment are implemented.

[0104] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0105] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0106] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0107] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0108] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0109] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A steam pipeline leakage detection method based on high-definition infrared, characterized in that: include: Obtain high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected; Determine a plurality of frames of target high-definition images from the high-definition infrared dual-mode video, input the plurality of frames of target high-definition images into a preset leakage detection model, and determine a first leakage area according to the detection results; Determine a plurality of target infrared images from the high-definition infrared dual-mode video, construct a target background image based on a first frame of the plurality of target infrared images, and determine a second leakage area according to the target background image and the remaining frames of the plurality of target infrared images; The first leakage area and the second leakage area of ​​the steam pipe to be detected are registered, and the target leakage area corresponding to the steam pipe to be detected is determined according to the registered overlapping area.

2. The method according to claim 1, characterized in that The step of constructing a target background image based on a first frame of the multiple target infrared images, and determining a second leakage area according to the target background image and the remaining frames of the multiple target infrared images includes: Acquire a leak-free background image corresponding to the steam pipeline to be detected; A target background image is constructed through the first frame target infrared image and the non-leakage background image, and a preset hybrid model is fine-tuned based on the target background image to obtain a target hybrid model; For each frame of the remaining target infrared images, based on any pixel in the image, a first grayscale distribution corresponding to the any pixel is calculated by the target mixture model, and based on the first grayscale distribution corresponding to the any pixel and a second grayscale distribution determined for the any pixel based on the image, a distribution difference between the first grayscale distribution and the second grayscale distribution is calculated, and whether the any pixel is a target pixel is determined based on the distribution difference; A second leakage area is obtained according to the target pixel points identified from each frame of the remaining frames of target infrared images.

3. The method according to claim 2, characterized in that The step of obtaining a second leakage area according to the target pixel points identified from each frame of the remaining target infrared images comprises: Based on a target high-definition image frame used to determine a first leakage area among the multiple frames of target high-definition images, determining a first acquisition time corresponding to the target high-definition image frame; Determine a second acquisition time corresponding to each frame in the multiple frames of target infrared images, and find out the target second acquisition time with the smallest time difference with the first acquisition time from the second acquisition time; The target infrared image frame corresponding to the second acquisition time of the target is used as the reference image, and according to the target pixel points identified from each frame of the remaining target infrared images, the leakage area indicated by the target pixel points is marked on the reference image, and a morphological opening and closing operation is performed on the reference image with the leakage area marked to obtain the second leakage area.

4. The method according to claim 1, characterized in that Before inputting the multiple frames of target high-definition images into a preset leakage detection model, the method further includes: Acquire a smoke sample image, and perform model training on an initial detection model based on the smoke sample image to obtain a target pre-trained detection model; A steam pipe leakage sample image is obtained, and the target pre-trained detection model is fine-tuned based on the steam pipe leakage sample image through a freezing training strategy to obtain the preset leakage detection model.

5. The method according to claim 1, characterized in that Determining the target leakage area corresponding to the steam pipeline to be detected according to the registered overlapping area includes: For any overlapped area after registration, determining a first sub-area to which the overlapped area belongs from the first leakage area, and determining a second sub-area to which the overlapped area belongs from the second leakage area; Calculating the total area of ​​the first sub-region and the second sub-region, and calculating the area intersection-over-combination ratio between the first sub-region and the second sub-region according to the area corresponding to any overlapping region and the total area; When the area intersection-and-union ratio is greater than a preset intersection-and-union ratio threshold, the proportion of any overlapping area in the registered image is determined, and when the proportion is greater than a preset proportion threshold, any overlapping area is used as the target leakage area corresponding to all steam pipes to be detected.

6. The method according to claim 1, characterized in that The steam pipe to be inspected is located in a target area, a detection track and a mobile detection device are arranged in the target area, the mobile detection device is arranged to move along the detection track, a plurality of detection points are arranged on the detection track, the mobile detection device is equipped with an infrared camera and a high-definition camera, and the high-definition infrared dual-mode video corresponding to the steam pipe to be inspected is captured by the mobile detection device at the detection points on the detection track.

7. The method according to claim 6, characterized in that The method further comprises: When the first leakage area and / or the second leakage area is detected, alarm information is generated based on the first leakage area and / or the second leakage area and the corresponding detection points, and the alarm information is sent to a preset terminal.

8. A steam pipeline leakage detection device based on high-definition infrared, characterized in that: include: A video acquisition module, used to acquire high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected; A first leakage area determination module is used to determine multiple frames of target high-definition images from the high-definition infrared dual-mode video, and input the multiple frames of target high-definition images into a preset leakage detection model, and determine a first leakage area according to the detection results; A second leakage area determination module is used to determine multiple frames of target infrared images from the high-definition infrared dual-mode video, and construct a target background image based on the first frame of the multiple frames of target infrared images, and determine a second leakage area according to the target background image and the remaining frames of the multiple frames of target infrared images; The registration module is used to perform registration processing on the first leakage area and the second leakage area of ​​the steam pipe to be detected, and determine the target leakage area corresponding to the steam pipe to be detected according to the overlapped area after registration.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

Images