Steam pipeline leakage detection method, device and storage medium based on high-definition infrared

Abstract

The present application discloses a steam pipe leakage detection method and device based on high-definition infrared, a storage medium, and a computer device. The method includes: obtaining a high-definition infrared dual-mode video corresponding to the steam pipe to be detected; determining multiple frames of target high-definition images from the high-definition infrared dual-mode video, and inputting the multiple frames of target high-definition images into a preset leakage detection model, and determining a first leakage area according to the detection results; determining multiple frames of target infrared images from the high-definition infrared dual-mode video, and constructing a target background image based on the first frame of the multiple frames of target infrared images, and determining a second leakage area according to the target background image and the remaining frames of the multiple frames of target infrared images; performing registration processing on the first leakage area and the second leakage area of ​​the steam pipe to be detected, and determining the target leakage area corresponding to the steam pipe to be detected according to the overlapping area after registration.

Description

Technical Field

[0001] The present application relates to the technical field of pipeline leakage detection, and in particular to a steam pipeline leakage detection method and device based on high-definition infrared, a storage medium, and a computer device. Background Art

[0002] Modern society's growing demand for energy has driven the use of thermal power plants, nuclear power plants, and various thermal energy conversion devices. Steam pipelines, essential equipment for large-scale energy facilities, are prone to steam leaks during daily operation due to factors such as condensation, vibration, and sealing. Leaks in steam pipelines inevitably result in significant energy waste and can even lead to safety concerns such as burns, explosions, and radiation leaks. Therefore, leak detection in steam pipelines is of paramount importance.

[0003] Existing leak detection techniques for steam pipelines rely on either manual on-site inspections or the installation of fixed-point sensor systems, such as pressure flow meters and acoustic emission leak detection equipment, on the steam pipelines. Manual on-site inspections are inefficient and risky, while the installation of fixed-point sensor systems on steam pipelines makes it difficult to accurately locate leaks and carries a high cost per unit area. Summary of the Invention

[0004] In light of this, this application provides a high-definition infrared-based steam pipe leak detection method and device, storage medium, and computer equipment. By jointly determining the leak area of ​​a steam pipe using high-definition and infrared images, the presence of a leak can be determined without manual on-site inspections. By combining these two methods, the accuracy of leak area determination can be further improved. Furthermore, by using equipment capable of capturing high-definition infrared dual-mode video and computing equipment, the leak area can be accurately determined, significantly reducing detection costs.

[0005] According to one aspect of the present application, a steam pipeline leak detection method based on high-definition infrared is provided, comprising:

[0006] Obtain high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected;

[0007] Determining multiple frames of target high-definition images from the high-definition infrared dual-mode video, inputting the multiple frames of target high-definition images into a preset leakage detection model, and determining a first leakage area according to the detection results;

[0008] Determining multiple frames of target infrared images from the high-definition infrared dual-mode video, constructing a target background image based on a first frame of the multiple frames of target infrared images, and determining a second leakage area based on the target background image and the remaining frames of the multiple frames of target infrared images;

[0009] The first leakage area and the second leakage area of ​​the steam pipe to be detected are registered, and a target leakage area corresponding to the steam pipe to be detected is determined according to the overlapped area after registration.

[0010] According to another aspect of the present application, a steam pipeline leakage detection device based on high-definition infrared is provided, comprising:

[0011] Video acquisition module, used to obtain high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected;

[0012] A first leakage area determination module is configured to determine multiple frames of target high-definition images from the high-definition infrared dual-mode video, input the multiple frames of target high-definition images into a preset leakage detection model, and determine a first leakage area based on the detection results;

[0013] a second leakage area determination module, configured to determine a plurality of target infrared images from the high-definition infrared dual-mode video, construct a target background image based on a first frame of the plurality of target infrared images, and determine a second leakage area based on the target background image and the remaining frames of the plurality of target infrared images;

[0014] The registration module is used to perform registration processing on the first leakage area and the second leakage area of ​​the steam pipe to be detected, and determine the target leakage area corresponding to the steam pipe to be detected according to the overlapped area after registration.

[0015] According to another aspect of the present application, a storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the above-mentioned steam pipe leakage detection method based on high-definition infrared is implemented.

[0016] According to another aspect of the present application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor. When the processor executes the program, the above-mentioned high-definition infrared-based steam pipe leakage detection method is implemented.

[0017] By means of the above technical solution, the present application provides a steam pipe leak detection method and device based on high-definition infrared, a storage medium, and a computer device. First, a high-definition infrared dual-mode video corresponding to the steam pipe to be detected is obtained. Multiple frames of high-quality, representative images are selected from the high-definition infrared dual-mode video as target high-definition images. These target high-definition images are then sequentially input into a preset leak detection model to obtain the detection results corresponding to each frame of the target high-definition image. The first leakage area is determined based on these detection results. Next, multiple frames of target infrared images are extracted from the high-definition infrared dual-mode video. The first frame of these target infrared images is used as a reference to construct a target background image. Then, based on the target background image, the infrared radiation changes caused by the leak are identified from the remaining frames except the first frame, thereby determining the second leakage area. The two areas can further be aligned so that they can accurately correspond to the same physical location. Finally, the target leakage area corresponding to the steam pipe to be detected is determined based on the overlapped area after alignment. The present embodiment uses both high-definition and infrared images to determine the leak area of ​​a steam pipeline, eliminating the need for manual on-site inspections to determine if a leak exists. This combined use of two methods can further improve the accuracy of leak area determination. Furthermore, precise determination of the leak area can be achieved using equipment capable of capturing high-definition, dual-mode infrared video and computing equipment, significantly reducing detection costs.

[0018] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A schematic diagram of a flow chart of a steam pipeline leakage detection method based on high-definition infrared provided in an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of a process for detecting steam pipe leaks based on high-definition infrared imaging is shown in an embodiment of the present application;

[0022] Figure 3 A schematic diagram illustrating a dual-mode video acquisition process provided in an embodiment of the present application is shown;

[0023] Figure 4A schematic diagram of a registered image provided by an embodiment of the present application is shown;

[0024] Figure 5 A schematic structural diagram of a steam pipe leakage detection device based on high-definition infrared provided in an embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram of the device structure of a computer device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0026] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that, unless there is a conflict, the embodiments and features in the embodiments of the present application can be combined with each other.

[0027] In this embodiment, a steam pipeline leakage detection method based on high-definition infrared is provided. Figure 1 As shown, the method includes:

[0028] Step 101: Obtain a high-definition infrared dual-mode video corresponding to the steam pipe to be inspected.

[0029] The embodiments of this application provide a high-definition infrared-based steam pipe leak detection method that can detect leak areas on steam pipes using both high-definition and infrared images. Before performing steam pipe leak detection, a high-definition infrared dual-mode video corresponding to the steam pipe to be inspected can be obtained. This video not only contains traditional visible light video information but also infrared radiation information, enabling detection to simultaneously utilize two different physical properties. The high-definition infrared dual-mode video can be captured using a high-definition infrared dual-mode camera.

[0030] Step 102: determine multiple frames of target high-definition images from the high-definition infrared dual-mode video, input the multiple frames of target high-definition images into a preset leakage detection model, and determine a first leakage area based on the detection results.

[0031] In this embodiment, multiple frames of high-quality, representative images are selected from the HD dual-mode infrared video as target HD images. These target HD images are then sequentially input into a pre-set leak detection model. This model can be trained using a machine learning or deep learning algorithm and is capable of analyzing features in the HD images and determining the first leak area based on these features.

[0032] Step 103: determine multiple frames of target infrared images from the high-definition infrared dual-mode video, construct a target background image based on the first frame of the multiple frames of target infrared images, and determine the second leakage area based on the target background image and the remaining frames of the multiple frames of target infrared images.

[0033] In this embodiment, multiple frames of target infrared images are extracted from the high-definition infrared dual-mode video. The first frame of these target infrared images is used as a reference to construct a target background image. This target background image represents the infrared radiation state of the pipeline in the absence of a leak. Then, based on the target background image, changes in infrared radiation caused by the leak are identified in the remaining frames, excluding the first frame, to determine the second leak area.

[0034] Step 104 : performing registration processing on the first leakage area and the second leakage area of ​​the steam pipe to be detected, and determining a target leakage area corresponding to the steam pipe to be detected according to the overlapped area after registration.

[0035] In this embodiment, since the first and second leak areas are determined based on the HD image and infrared image, respectively, there may be positional deviations or inconsistencies between them. Therefore, these two areas can be further registered, adjusting their positions and sizes so that they accurately correspond to the same physical location. Finally, based on the registered overlapping area, the target leak area corresponding to the steam pipe to be inspected is determined. This area represents the leak point confirmed by both methods, providing high reliability and accuracy.

[0036] By applying the technical solution of this embodiment, a high-definition infrared dual-mode video corresponding to the steam pipe to be inspected is first acquired. Multiple frames of high-quality, representative images are selected from the high-definition infrared dual-mode video as target high-definition images. These target high-definition images are then sequentially input into a pre-set leak detection model to obtain detection results corresponding to each target high-definition image. Based on these detection results, a first leak area is determined. Next, multiple frames of target infrared images are extracted from the high-definition infrared dual-mode video. The first frame of these target infrared images is used as a reference to construct a target background image. Based on the target background image, infrared radiation changes caused by the leak are identified in the remaining frames, excluding the first frame, to determine the second leak area. The two areas can then be registered to accurately correspond to the same physical location. Finally, the target leak area corresponding to the steam pipe to be inspected is determined based on the registered overlapping area. This embodiment of the present application uses both high-definition and infrared images to determine the leak area of ​​the steam pipe, eliminating the need for manual on-site inspections to determine whether a leak exists in the steam pipe. By combining these two methods to determine the leak area of ​​the steam pipe, the accuracy of leak area determination can be further improved. In addition, the leakage area can be accurately determined by using equipment that can capture high-definition infrared dual-mode video and computing equipment, which can greatly reduce the detection cost.

[0037] Furthermore, as a refinement and expansion of the specific implementation of the above embodiment, in order to fully illustrate the specific implementation process of this embodiment, another steam pipeline leakage detection method based on high-definition infrared is provided, such as Figure 2 As shown, the method includes:

[0038] Step 201: Obtain a high-definition infrared dual-mode video corresponding to the steam pipe to be inspected.

[0039] In an embodiment of the present application, optionally, the steam pipe to be inspected is located in a target area, a detection track and a mobile detection device are provided in the target area, the mobile detection device is configured to move along the detection track, a plurality of detection points are provided on the detection track, the mobile detection device is equipped with an infrared camera and a high-definition camera, and the high-definition infrared dual-mode video corresponding to the steam pipe to be inspected is captured by the mobile detection device at the detection point on the detection track.

[0040] In this embodiment, the target area can be a specific area where steam pipe leak detection is required, which can include one or more steam pipes. The detection trajectory can be one or more paths defined within the target area, along which the mobile detection device can move to conduct a comprehensive inspection. The detection trajectory can be determined based on factors such as the layout of the steam pipe and detection efficiency. The mobile detection device is a device that can move autonomously along the detection trajectory, offering high flexibility and adaptability, and capable of operating under diverse environmental conditions. The mobile detection device can be equipped with an infrared camera and a high-definition camera, which are used to capture infrared radiation and visible light information from the steam pipe, respectively. Infrared cameras are highly sensitive to detecting temperature anomalies caused by leaks, while high-definition cameras can provide detailed visual information about the pipe. The infrared camera captures infrared images. This can be achieved by using an infrared lamp that emits infrared light in multiple frequency bands. After absorption by the leaking steam and reflection from the surrounding environment, the infrared light passes through specific narrowband filters and reaches a broadband refrigerated infrared core, generating a corresponding infrared spectral grayscale image. Furthermore, the detection trajectory is configured with multiple detection points, where the mobile detection device stops and captures video during its movement. The setting of detection points should be based on the actual situation of the steam pipeline and the detection requirements to ensure that possible leakage points can be fully covered. When the mobile detection device moves to each detection point along the detection trajectory, it can use an infrared camera and a high-definition camera to shoot a few seconds of dual-mode video of the steam pipeline at the detection point based on the set posture. The shooting time can be determined according to demand. In this way, each detection point can correspond to a high-definition infrared dual-mode video containing infrared radiation information and visible light information. These dual-mode videos are then used for subsequent leak detection analysis to identify and locate the leakage points of the steam pipeline by comparing information such as changes in infrared radiation and the status of the pipeline under visible light. The embodiment of the present application achieves comprehensive, efficient and real-time leakage detection of steam pipelines by setting target areas, detection trajectories and detection points, and using a mobile detection device equipped with an infrared camera and a high-definition camera for shooting. This method not only improves the efficiency, accuracy and reliability of detection, but also reduces the labor intensity and cost of manual detection.

[0041] Furthermore, the mobile detection device can also be equipped with a communication device, specifically a multi-frequency WiFi module. This module can utilize multi-band wireless communication technology, using high-frequency radio waves for communication in areas subject to significant fog interference, and low-frequency communication in narrow areas with numerous obstacles. Specifically, different communication frequency bands can be pre-set for different detection points. When the mobile detection device reaches a detection point along its detection trajectory, it first determines the preset communication frequency band for that point and then uses the preset communication frequency band to transmit high-definition infrared dual-mode video to a server. The server then performs the subsequent steps of leak area detection.

[0042] In one embodiment, if Figure 3 As shown, the mobile detection device can be composed of a four-arm crawler mobile robot equipped with a dual-spectrum monitoring platform. The dual-spectrum monitoring platform consists of a high-definition camera and an infrared camera, and the infrared camera is equipped with an infrared lamp, which is coaxial with the infrared camera lens. The infrared camera in the dual-spectrum monitoring platform collects images of the leaking heat source, and the high-definition camera obtains high-definition images of the leak scene. The mobile robot can move along the detection trajectory, such as Figure 3 As shown, the robot can first move to the detection point corresponding to device 1, capture a dual-mode video, then move to the detection point corresponding to device 2 and capture another dual-mode video. After each capture, the mobile robot can use its communication device to transmit the dual-mode video via a wireless base station to a server, which then performs subsequent data analysis.

[0043] Step 202: determine multiple frames of target high-definition images from the high-definition infrared dual-mode video, input the multiple frames of target high-definition images into a preset leakage detection model, and determine a first leakage area based on the detection results.

[0044] In this embodiment, after each target HD image is input into a preset leak detection model, the corresponding leak area of ​​the target HD image is identified. Finally, among the leak areas corresponding to these target HD images, the target HD image with the largest leak area is found, and the leak area corresponding to this target HD image is designated as the first leak area.

[0045] Step 203 : determining multiple frames of target infrared images from the high-definition infrared dual-mode video, and obtaining a leak-free background image corresponding to the steam pipe to be detected.

[0046] In this embodiment, after determining multiple frames of target infrared images, a leak-free background image corresponding to the steam pipe to be inspected can also be obtained. Specifically, an infrared image of the steam pipe without leaks is obtained. This image is typically taken when the pipe is operating normally and no leaks are detected. This leak-free background image can serve as a baseline for subsequent analysis, allowing comparison and identification of changes in infrared radiation caused by leaks.

[0047] Step 204 : constructing a target background image through the first frame target infrared image and the non-leakage background image, and fine-tuning the preset hybrid model based on the target background image to obtain a target hybrid model.

[0048] In this embodiment, further, the first frame of the target infrared image in the multiple frames of target infrared images (usually a frame taken at the beginning of the detection) is combined with the no-leakage background image to construct a target background image. This target background image can be a fusion of the information of the first frame of the target infrared image and the no-leakage background image in some way (such as weighted averaging, image fusion, etc.) to more accurately reflect the infrared radiation status of the pipeline under the current environmental conditions. Then, based on this target background image, the preset hybrid model is fine-tuned. Specifically, the preset hybrid model can be a mathematical model for analyzing the grayscale distribution of pixels in an infrared image, which can distinguish normal infrared radiation from abnormal radiation caused by leakage by learning the features of the target background image and the current frame target infrared image. The target hybrid model obtained by fine-tuning can better adapt to the current detection scene and improve the accuracy of detection.

[0049] In one embodiment, the target mixture model can be expressed as follows:

[0050]

[0051] Where K is the number of Gaussian distributions included in the target mixture model. Its size is determined by the number of background and foreground modes in the image and the computing power used, and is generally 3 to 5; ω i,t is the weight of the Gaussian distribution corresponding to the target infrared image of the tth frame of the i-th model in the target mixture model, X t is the pixel in the target infrared image of frame t, μ i,t is the average value of the Gaussian distribution corresponding to the target infrared image of the t-th frame in the target mixture model, Σ i,t is the covariance matrix of the Gaussian distribution corresponding to the t-th frame target infrared image in the target mixture model, and η is the probability density function of the Gaussian distribution.

[0052] Step 205: For each frame of the remaining target infrared images, based on any pixel point in the image, calculate the first grayscale distribution corresponding to the any pixel point through the target mixture model, and based on the first grayscale distribution corresponding to the any pixel point and the second grayscale distribution of the any pixel point determined based on the image, calculate the distribution difference between the first grayscale distribution and the second grayscale distribution, and determine whether the any pixel point is a target pixel point based on the distribution difference.

[0053] In this embodiment, for each of the remaining target infrared images, any pixel in the image is selected for analysis. Specifically, a fine-tuned target mixture model can be used to calculate a first grayscale distribution corresponding to the pixel. This distribution reflects the target mixture model's prediction of the pixel's grayscale value under normal (or background) conditions. Simultaneously, a second grayscale distribution is determined based on the pixel's actual grayscale value in the current image. This distribution can be statistically derived from the grayscale values ​​of the pixels surrounding the pixel and reflects the pixel's actual grayscale characteristics in the current image. The distribution difference between the first and second grayscale distributions is then calculated. This distribution difference quantifies the inconsistency between the target mixture model's prediction and the actual observation, which may be caused by changes in infrared radiation due to leakage. Based on the magnitude of the distribution difference, it can be determined whether the pixel is a target pixel (i.e., an abnormal pixel that may be due to leakage). For example, if the distribution difference exceeds a preset threshold, the pixel is considered likely to be the target pixel. Specifically, the distribution difference between the first and second grayscale distributions can be calculated using the Wasserstein distance formula.

[0054] Step 206 : obtaining a second leakage area according to the target pixel points identified from each frame of the remaining target infrared images.

[0055] In this embodiment, by performing the above analysis on all pixels in each frame of the target infrared image, the target pixels in each frame can be identified. The identified target pixels in all frames are then aggregated and labeled to ultimately determine the second leakage region. This region may contain one or more areas of abnormal infrared radiation caused by the leak and is a key target for subsequent inspection and repair.

[0056] The embodiment of the present application uses a non-leakage background image as a benchmark and compares and analyzes it with the target infrared image taken in real time, so that the infrared radiation changes caused by leakage can be more accurately identified. This method based on background contrast can reduce the interference of environmental factors (such as temperature fluctuations, light changes, etc.) on the detection results. In addition, the embodiment of the present application introduces a hybrid model and adapts it to the current detection scene through fine-tuning. This adaptive mechanism enables the detection method to be adjusted according to different environmental conditions (such as pipeline layout, ambient temperature, etc.), thereby improving the flexibility and accuracy of detection.

[0057] Step 207: performing registration processing on the first leakage area and the second leakage area of ​​the steam pipe to be detected.

[0058] In this embodiment, the VoxelMorph algorithm can be used to align the first leakage area and the second leakage area, with the target high-definition image corresponding to the first leakage area as the fixed image and the target infrared image corresponding to the second leakage area as the image to be aligned. The fixed image and the image to be aligned are transformed into 3D vectors through dimensional expansion, and then the fixed image and the image to be aligned are sent to the U-net for encoding and decoding to generate a deformation field. According to the deformation field, the image to be aligned is pixel-reshaped and interpolated through the spatial transformation network to obtain the aligned image. Figure 4 As shown, a schematic diagram of the image after registration is given. Figure 4 , the leakage region C and the leakage region A are the first leakage region, and the leakage region B and the leakage region D are the second leakage region.

[0059] Step 208 : For any overlapped area after registration, determine a first sub-area to which the overlapped area belongs from the first leakage area, and determine a second sub-area to which the overlapped area belongs from the second leakage area.

[0060] In this embodiment, there may be some overlapping areas in the registered images, which are covered by both detection methods (i.e., high-definition image acquisition and infrared image acquisition). For any overlapping area, the first sub-area to which the area belongs is first determined from the first leakage area, and similarly, the second sub-area to which the area belongs is also determined from the second leakage area. Figure 4 As shown, the overlapping area is A∩B, then the first sub-area is the leakage area A, and the second sub-area is the leakage area B.

[0061] Step 209 : Calculate the total area of ​​the first sub-region and the second sub-region, and calculate the area intersection-over-union ratio between the first sub-region and the second sub-region based on the area corresponding to any overlapping region and the total area.

[0062] In this embodiment, the total area of ​​the region refers to the sum of the areas of the first sub-region and the second sub-region. Figure 4 In [1], the total area of ​​the region can be the area of ​​A∪B. The intersection-over-union (IoU) is a metric that measures the degree of overlap between two regions. It is calculated as the ratio of the area of ​​the intersection of the two regions to the area of ​​their union. In this step, the IoU of the first subregion and the second subregion can be calculated to assess the degree of overlap between them.

[0063] Step 210: When the IoU of the regions is greater than a preset IoU threshold, the proportion of any overlapping region in the registered image is determined, and when the proportion is greater than a preset proportion threshold, any overlapping region is used as the target leakage region corresponding to all steam pipes to be detected.

[0064] In this embodiment, when the IoU ratio (Intersection-Over-Union) is greater than a preset IoU threshold, it indicates a high degree of overlap between the two sub-regions, potentially indicating a true leak. However, IoU alone is not sufficient to determine the target leak region, as there may be some small, insignificant overlapping regions. Therefore, the proportion of any overlapping region in the registered image can be further determined. If this proportion exceeds a preset threshold, the overlapping region is considered significant and is selected as the target leak region for the steam pipe to be inspected.

[0065] The embodiment of the present application integrates the results of multiple detection methods to more accurately determine the leakage area of ​​the steam pipe. By calculating the intersection-over-union ratio and judging the ratio, some unimportant or false overlapping areas can be excluded, thereby improving the accuracy and reliability of the detection.

[0066] In the embodiment of the present application, optionally, step 206 includes:

[0067] Step 206 - 1 : Based on a target high-definition image frame used to determine a first leakage area in the multiple frames of target high-definition images, determine a first acquisition time corresponding to the target high-definition image frame.

[0068] Step 206 - 2 : determining a second acquisition time corresponding to each frame in the multiple frames of target infrared images, and finding a target second acquisition time having a minimum time difference with the first acquisition time from the second acquisition times.

[0069] Step 206-3: Use the target infrared image frame corresponding to the second acquisition time of the target as the reference image, and based on the target pixel points identified from each frame of the remaining target infrared images, mark the leakage area indicated by the target pixel points on the reference image, perform morphological opening and closing operations on the reference image with the leakage area marked, and obtain the second leakage area.

[0070] In this embodiment, among multiple target HD images, the HD image frame used to identify the first leak area is first found, referred to as the "target HD image frame," and the first acquisition time corresponding to the target HD image frame is determined. It should be noted that each target HD image frame, after being input into the preset leak detection model, produces a corresponding recognition result. In this case, the target HD image with the largest leak area in the recognition result can be used as the target HD image frame. Next, multiple target infrared images are considered. Each target infrared image frame has a corresponding second acquisition time. From the acquisition times of these target infrared images, the target second acquisition time with the smallest time difference from the first acquisition time of the target HD image frame is found, and the target infrared image frame corresponding to the target second acquisition time is used as the reference image. Because the target HD image frame and the target infrared image frame are closest in time, they also observe the most similar steam pipe conditions. Next, the target pixels identified from the other target infrared images are mapped onto the reference image, and the leak areas indicated by these pixels are marked accordingly on the reference image. A morphological opening and closing operation is performed on the reference image with the marked leak areas. Morphological opening operations are usually used to remove small foreground objects (such as noise) and smooth the boundaries of larger objects while keeping their areas unchanged. Morphological closing operations are used to fill small holes or black areas within foreground objects and smooth their boundaries. By combining opening and closing operations (i.e., morphological opening and closing operations), the leakage areas in the image can be further cleaned and enhanced to make them clearer and more accurate. After the baseline image is processed by the morphological opening and closing operations, the leakage area will be clearer and more prominent. This processed area is the second leakage area, which provides a more reliable basis for the detection of steam pipe leakage. The embodiment of the present application ensures that the steam pipe states observed by the two are as consistent as possible by finding the target infrared image frame that is closest in time to the target high-definition image frame as the baseline image, so that when the first leakage area and the second leakage area are subsequently aligned, the alignment results can be made more accurate, thereby improving the robustness of the detection results. In addition, the morphological opening and closing operations further clean and enhance the leakage area in the image, so that it is not affected by interference factors such as noise and small holes, further improving the detection accuracy.

[0071] In an embodiment of the present application, optionally, before "inputting the multi-frame target high-definition image into the preset leakage detection model" in step 202, the method further includes: obtaining a smoke sample image, and performing model training on the initial detection model based on the smoke sample image to obtain a target pre-trained detection model; obtaining a steam pipe leakage sample image, and fine-tuning the target pre-trained detection model based on the steam pipe leakage sample image through a freezing training strategy to obtain the preset leakage detection model.

[0072] In this embodiment, smoke sample images can be obtained first. Smoke sample images are a set of images containing smoke, which can come from various sources, such as public datasets, laboratory simulations, or actual on-site shooting. The selection of smoke sample images is as diverse as possible to cover different smoke types, concentrations, lighting conditions, etc., so that the trained model has better generalization ability. Then, the initial detection model is trained based on the smoke sample images. Specifically, the initial detection model can be a deep learning model, such as a convolutional neural network (CNN), YOLO, SSD, etc., which perform well in image detection tasks. Afterwards, the initial detection model is trained using the collected smoke sample images. During the training process, the initial detection model can learn the characteristics of the smoke, such as shape, texture, color, etc., so that it can distinguish between smoke and non-smoke areas. After multiple iterative training, the initial detection model gradually converges and has good smoke detection capabilities. The model obtained at this time is called a target pre-trained detection model.

[0073] Next, sample steam pipe leak images can be obtained. These images are a set of images containing steam pipe leak areas, with the leak areas labeled. These images cover a variety of leak types, pipe layouts, and lighting conditions. The sample steam pipe leak images are then used to fine-tune the target pre-trained detection model using a frozen training strategy. Fine-tuning is a common model optimization method that uses a model trained on a similar task as a starting point and fine-tunes it on a dataset for the new task. In this process, most layers of the target pre-trained detection model (such as the early convolutional layers) are first frozen. These layers have already learned general image features and are still useful for the new task. Then, only the later layers of the model (such as the fully connected layers or the specific detection layers) are adjusted, as these layers are more focused on the specific detection task. The model is fine-tuned using the sample steam pipe leak images, allowing it to learn the features of the leak areas. Because most layers are frozen, the fine-tuning process is typically much faster than training from scratch and requires fewer sample steam pipe leak images. After fine-tuning, the model will be more suitable for the task of detecting steam pipe leaks, and the model obtained at this time is the preset leak detection model. The embodiment of the present application first trains a general smoke detection model, and then uses steam pipe leakage sample images for fine-tuning, and finally obtains a model specifically for steam pipe leak detection. This method makes full use of the idea of ​​transfer learning, which not only saves training time but also improves the detection performance of the model. In addition, by using smoke sample images to train the initial detection model, the number of leakage images required to train the model is reduced. Only a small number of steam pipe leakage sample images are required to determine the final preset leak detection model, and the determined preset leak detection model has a good ability to identify leakage areas under smoke, making the preset leak detection model more suitable for the application scenario of the present application.

[0074] In an embodiment of the present application, optionally, the method further includes: after detecting the first leakage area and / or the second leakage area, generating alarm information based on the first leakage area and / or the second leakage area, and the corresponding detection points, and sending the alarm information to a preset terminal.

[0075] In this embodiment, if a first leakage area and / or a second leakage area is detected in a dual-mode video collected at a certain detection point, then an alarm message can be generated based on the detected first leakage area, the second leakage area and the corresponding detection point, and the generated alarm message can be sent to the user of the preset terminal. These preset terminal users can be engineers, operators or managers responsible for maintaining steam pipelines, etc. The alarm information can be sent in various ways, including but not limited to: SMS notifications, e-mails, etc. The embodiment of the present application uses automated image processing and machine learning technology to quickly and accurately detect leakage areas on steam pipelines, and generates alarm information in the form of alarms to promptly convey it to relevant personnel so that they can quickly take measures to deal with it, thereby ensuring the safe operation of the steam pipeline.

[0076] In an embodiment of the present application, optionally, after step 203, the method further includes: constructing a target background image through the first frame target infrared image and the leakage-free background image; for each frame image of the remaining frame target infrared images, based on each pixel point in the image, calculating the first grayscale value corresponding to the pixel point, and calculating the second grayscale value of the pixel point in the target background image, and judging whether the pixel point is a target pixel point according to the grayscale difference between the first grayscale value and the second grayscale value; and obtaining a second leakage area according to the target pixel point identified from each frame image of the remaining frame target infrared images.

[0077] In this embodiment, another method for determining the second leakage area is provided. First, the first target infrared image (typically the frame captured at the beginning of the inspection) of multiple target infrared images is combined with a leak-free background image to construct a target background image. This target background image can be a fusion of the first target infrared image and the leak-free background image through some method (such as weighted averaging or image fusion) to more accurately reflect the infrared radiation status of the pipeline under the current environmental conditions. Then, for each subsequent target infrared image (i.e., all frames except the first), the following operation is performed: For each pixel in each target infrared image, its grayscale value in the current frame is calculated. This value is referred to as the "first grayscale value." Similarly, the corresponding pixel in the previously constructed target background image is found and its grayscale value is calculated. This value is referred to as the "second grayscale value." Next, the difference between the first grayscale value and the second grayscale value (i.e., the grayscale difference) is compared to determine whether the pixel belongs to the target (i.e., the potential leakage area). If the grayscale difference exceeds a preset threshold, the pixel is considered to have undergone a significant change, potentially due to a leak, and is therefore marked as a target pixel. This analysis is then repeated for all pixels in each frame of the target infrared image to identify the target pixel in each frame. The identified target pixels across all frames are then aggregated and marked, ultimately resulting in a second leakage region. This region may contain one or more areas of infrared radiation anomalies caused by the leak and is a key target for further inspection and repair.

[0078] Further, as Figure 1 The specific implementation of the method, the embodiment of the present application provides a steam pipe leakage detection device based on high-definition infrared, such as Figure 5 As shown, the device includes:

[0079] Video acquisition module, used to obtain high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected;

[0080] A first leakage area determination module is configured to determine multiple frames of target high-definition images from the high-definition infrared dual-mode video, input the multiple frames of target high-definition images into a preset leakage detection model, and determine a first leakage area based on the detection results;

[0081] a second leakage area determination module, configured to determine a plurality of target infrared images from the high-definition infrared dual-mode video, construct a target background image based on a first frame of the plurality of target infrared images, and determine a second leakage area based on the target background image and the remaining frames of the plurality of target infrared images;

[0082] The registration module is used to perform registration processing on the first leakage area and the second leakage area of ​​the steam pipe to be detected, and determine the target leakage area corresponding to the steam pipe to be detected according to the overlapped area after registration.

[0083] Optionally, the second leakage area determination module is configured to:

[0084] Acquire a leak-free background image corresponding to the steam pipeline to be detected;

[0085] Constructing a target background image through the first frame target infrared image and the non-leakage background image, and fine-tuning a preset hybrid model based on the target background image to obtain a target hybrid model;

[0086] For each frame of the remaining target infrared images, based on any pixel point in the image, using the target mixture model, calculate a first grayscale distribution corresponding to the any pixel point, and based on the first grayscale distribution corresponding to the any pixel point and a second grayscale distribution determined for the any pixel point based on the image, calculate a distribution difference between the first grayscale distribution and the second grayscale distribution, and determine whether the any pixel point is a target pixel point based on the distribution difference;

[0087] A second leakage area is obtained according to target pixel points identified from each frame of the remaining target infrared images.

[0088] Optionally, the second leakage area determination module is further configured to:

[0089] Determining a first acquisition time corresponding to a target high-definition image frame used to determine a first leakage area among the multiple frames of target high-definition images;

[0090] Determining a second acquisition time corresponding to each frame of the multiple frames of target infrared images, and finding a target second acquisition time having a minimum time difference with the first acquisition time from the second acquisition times;

[0091] The target infrared image frame corresponding to the second acquisition time of the target is used as the reference image, and according to the target pixel points identified from each frame of the remaining target infrared images, the leakage area indicated by the target pixel points is marked on the reference image, and a morphological opening and closing operation is performed on the reference image with the leakage area marked to obtain the second leakage area.

[0092] Optionally, the device further comprises:

[0093] A pre-training module is used to obtain smoke sample images before inputting the multiple frames of target high-definition images into a preset leakage detection model, and to perform model training on an initial detection model based on the smoke sample images to obtain a target pre-trained detection model;

[0094] The fine-tuning module is used to obtain a steam pipe leakage sample image, and fine-tune the target pre-trained detection model based on the steam pipe leakage sample image through a frozen training strategy to obtain the preset leakage detection model.

[0095] Optionally, the registration module is used to:

[0096] For any overlapped area after registration, determining a first sub-area to which the overlapped area belongs from the first leakage area, and determining a second sub-area to which the overlapped area belongs from the second leakage area;

[0097] Calculating the total area of ​​the first sub-region and the second sub-region, and calculating the area intersection-over-union ratio between the first sub-region and the second sub-region based on the area corresponding to any overlapping region and the total area;

[0098] When the area intersection-over-union ratio is greater than a preset intersection-over-union ratio threshold, the proportion of any overlapping area in the registered image is determined, and when the proportion is greater than a preset proportion threshold, any overlapping area is used as the target leakage area corresponding to all steam pipes to be detected.

[0099] Optionally, the steam pipe to be inspected is located in a target area, and a detection track and a mobile detection device are provided in the target area. The mobile detection device is configured to move along the detection track, and a plurality of detection points are provided on the detection track. The mobile detection device is equipped with an infrared camera and a high-definition camera, and the high-definition infrared dual-mode video corresponding to the steam pipe to be inspected is captured by the mobile detection device at the detection point on the detection track.

[0100] Optionally, the device further comprises:

[0101] The alarm module is used to generate alarm information based on the first leakage area and / or the second leakage area and the corresponding detection points after detecting the first leakage area and / or the second leakage area, and send the alarm information to a preset terminal.

[0102] It should be noted that for other corresponding descriptions of the functional units involved in the high-definition infrared steam pipe leakage detection device provided in the embodiment of the present application, please refer to Figures 1 to 4 The corresponding description in the method will not be repeated here.

[0103] The present application also provides a computer device, which can be a personal computer, a server, a network device, etc. Figure 6 As shown, the computer device includes a bus, a processor, a memory, and a communication interface, and may also include an input / output interface and a display device. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store location information. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, the steps of each method embodiment are implemented.

[0104] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0105] In one embodiment, a computer-readable storage medium is provided. The computer-readable storage medium may be non-volatile or volatile, and stores a computer program thereon. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0106] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0107] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0108] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0109] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0110] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A steam pipeline leakage detection method based on high-definition infrared, characterized in that: include: Obtain high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected; Determining multiple frames of target high-definition images from the high-definition infrared dual-mode video, inputting the multiple frames of target high-definition images into a preset leakage detection model, and determining a first leakage area based on the detection results; Determining multiple frames of target infrared images from the high-definition infrared dual-mode video, constructing a target background image based on a first frame of the multiple frames of target infrared images, and determining a second leakage area based on the target background image and the remaining frames of the multiple frames of target infrared images; Performing registration processing on the first leakage area and the second leakage area of ​​the steam pipe to be detected, and determining the target leakage area corresponding to the steam pipe to be detected according to the overlapped area after registration; The step of constructing a target background image based on a first frame of the multiple target infrared images, and determining a second leakage area according to the target background image and the remaining frames of the multiple target infrared images includes: Acquire a leak-free background image corresponding to the steam pipeline to be detected; Constructing a target background image through the first frame target infrared image and the non-leakage background image, and fine-tuning a preset hybrid model based on the target background image to obtain a target hybrid model; For each frame of the remaining target infrared images, based on any pixel point in the image, calculate the first grayscale distribution corresponding to the any pixel point through the target mixture model, and based on the first grayscale distribution corresponding to the any pixel point and the second grayscale distribution determined for the any pixel point based on the image, calculate the distribution difference between the first grayscale distribution and the second grayscale distribution, and judge whether the any pixel point is a target pixel point according to the distribution difference, wherein the first grayscale distribution reflects the grayscale value prediction of the any pixel point in a normal or background state by the target mixture model, and the second grayscale distribution is obtained based on the statistics of the grayscale values ​​of the pixels surrounding the any pixel point, reflecting the actual grayscale characteristics of the any pixel point in the current image; A second leakage area is obtained according to target pixel points identified from each frame of the remaining target infrared images.

2. The method according to claim 1, characterized in that The step of obtaining a second leakage area according to target pixel points identified from each frame of the remaining target infrared images includes: Determining a first acquisition time corresponding to a target high-definition image frame used to determine a first leakage area among the multiple frames of target high-definition images; Determining a second acquisition time corresponding to each frame of the multiple frames of target infrared images, and finding a target second acquisition time having a minimum time difference with the first acquisition time from the second acquisition times; The target infrared image frame corresponding to the second acquisition time of the target is used as the reference image, and according to the target pixel points identified from each frame of the remaining target infrared images, the leakage area indicated by the target pixel points is marked on the reference image, and a morphological opening and closing operation is performed on the reference image with the leakage area marked to obtain the second leakage area.

3. The method according to claim 1, characterized in that Before inputting the multiple frames of target high-definition images into a preset leakage detection model, the method further includes: Acquire a smoke sample image, and perform model training on an initial detection model based on the smoke sample image to obtain a target pre-trained detection model; A steam pipe leakage sample image is obtained, and the target pre-trained detection model is fine-tuned based on the steam pipe leakage sample image through a frozen training strategy to obtain the preset leakage detection model.

4. The method according to claim 1, wherein Determining the target leakage area corresponding to the steam pipe to be detected according to the registered overlapping area includes: For any overlapped area after registration, determining a first sub-area to which the overlapped area belongs from the first leakage area, and determining a second sub-area to which the overlapped area belongs from the second leakage area; Calculating the total area of ​​the first sub-region and the second sub-region, and calculating the area intersection-over-union ratio between the first sub-region and the second sub-region based on the area corresponding to any overlapping region and the total area; When the area intersection-over-union ratio is greater than a preset intersection-over-union ratio threshold, the proportion of any overlapping area in the registered image is determined, and when the proportion is greater than a preset proportion threshold, any overlapping area is used as the target leakage area corresponding to all steam pipes to be detected.

5. The method according to claim 1, wherein The steam pipe to be inspected is located in a target area, and a detection track and a mobile detection device are set in the target area. The mobile detection device is configured to move along the detection track, and multiple detection points are set on the detection track. The mobile detection device is installed with an infrared camera and a high-definition camera. The high-definition infrared dual-mode video corresponding to the steam pipe to be inspected is obtained by shooting the mobile detection device at the detection points on the detection track.

6. The method according to claim 5, characterized in that The method further comprises: When the first leakage area and / or the second leakage area is detected, alarm information is generated based on the first leakage area and / or the second leakage area and the corresponding detection points, and the alarm information is sent to a preset terminal.

7. A steam pipe leak detection device based on high-definition infrared, characterized in that: include: Video acquisition module, used to obtain high-definition infrared dual-mode video corresponding to the steam pipeline to be inspected; A first leakage area determination module is configured to determine multiple frames of target high-definition images from the high-definition infrared dual-mode video, input the multiple frames of target high-definition images into a preset leakage detection model, and determine a first leakage area based on the detection results; a second leakage area determination module, configured to determine a plurality of target infrared images from the high-definition infrared dual-mode video, construct a target background image based on a first frame of the plurality of target infrared images, and determine a second leakage area based on the target background image and the remaining frames of the plurality of target infrared images; a registration module, configured to perform registration processing on the first leakage area and the second leakage area of ​​the steam pipe to be detected, and determine a target leakage area corresponding to the steam pipe to be detected based on the overlapped area after registration; The second leakage area determination module is configured to: Acquire a leak-free background image corresponding to the steam pipeline to be detected; Constructing a target background image through the first frame target infrared image and the non-leakage background image, and fine-tuning a preset hybrid model based on the target background image to obtain a target hybrid model; For each frame of the remaining target infrared images, based on any pixel point in the image, calculate the first grayscale distribution corresponding to the any pixel point through the target mixture model, and based on the first grayscale distribution corresponding to the any pixel point and the second grayscale distribution determined for the any pixel point based on the image, calculate the distribution difference between the first grayscale distribution and the second grayscale distribution, and judge whether the any pixel point is a target pixel point according to the distribution difference, wherein the first grayscale distribution reflects the grayscale value prediction of the any pixel point in a normal or background state by the target mixture model, and the second grayscale distribution is obtained based on the statistics of the grayscale values ​​of the pixels surrounding the any pixel point, reflecting the actual grayscale characteristics of the any pixel point in the current image; A second leakage area is obtained according to target pixel points identified from each frame of the remaining target infrared images.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. A computer device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.

Images