Leakage area detection method and device based on unmanned aerial vehicle image

Abstract

The application discloses a leakage area detection method and device based on unmanned aerial vehicle images, and relates to the field of leakage area detection.The method comprises the following steps: acquiring visible light images and thermal infrared images of a to-be-detected area by visible light cameras and thermal infrared cameras carried by an unmanned aerial vehicle; acquiring a preliminary leakage area and a preliminary non-leakage area in the visible light images; performing filtering processing on the thermal infrared images to obtain thermal infrared filtered images, and acquiring a preliminary leakage area and a preliminary non-leakage area in the thermal infrared filtered images; combining the preliminary leakage area and the preliminary non-leakage area in the visible light images and the preliminary leakage area and the preliminary non-leakage area in the thermal infrared filtered images to determine a preliminary screened target leakage area in the thermal infrared filtered images, and generating a mask image of the thermal infrared filtered images; acquiring a binary image of the thermal infrared filtered images by using an Otsu binary algorithm; according to the mask image and the binary image, acquiring pixel points marked as true at the same time as target pixel points corresponding to the target leakage area; and acquiring the target leakage area in the to-be-detected area according to the target pixel points.

Description

Technical Field

[0001] This application relates to the field of leakage area detection, and in particular to a method and apparatus for leakage area detection based on UAV imagery. Background Technology

[0002] In modern engineering, leakage monitoring technology is widely used in the maintenance of critical infrastructure in industries such as manufacturing, construction, water conservancy, and transportation, especially in ensuring the safety of structures such as underground utility tunnels, dams, pipelines, and bridges. Timely detection and accurate location of leakage areas are not only crucial for maintaining structural integrity but also have a profound impact on reducing potential economic losses.

[0003] Thermal infrared detection technology is one of the most commonly used methods for leak detection, identifying leak areas by capturing temperature anomalies on the surface of an object. Currently, leak detection can be mainly divided into five categories: First, based on traditional image processing (such as Chinese patents CN102636313A and CN117474894A), using rule-based algorithms such as filtering, enhancement, and threshold segmentation. This is suitable for scenes with obvious features and simple backgrounds; the algorithms are simple and efficient but have weak anti-interference capabilities and perform poorly in complex environments. Second, based on multimodal fusion (such as Chinese patents CN119600605A and CN119339106A), fusing information from multiple sources such as infrared and visible light images to improve detection accuracy and robustness in complex environments. However, it is sensitive to registration and fusion parameters, the algorithms are more complex, and real-time performance is limited. Third, deep learning-based methods (such as Chinese patents CN118675072A and CN116580328B) rely on end-to-end detection using deep models. These methods are adaptable to complex and small-target scenarios, possess strong automatic feature extraction capabilities, high accuracy, and good scalability. However, they require a large amount of high-quality data, and training and inference demand significant computing power. Fourth, comprehensive early warning and monitoring methods combine thermal infrared temperature threshold detection, early warning, and location (such as Chinese patent CN114544456A). Based on infrared and visible light images, image fusion is performed for Faster R-CNN tunnel leakage detection and identification (Chinese patent CN111899288B). These systems are complex, have high deployment costs, and require custom development. Fifth, there is the automated detection based on drones. Chinese patent CN118485972A acquires infrared and visible light images and uses multi-scale feature extraction, band fusion and semantic association analysis to realize the detection and discrimination of dam seepage. Chinese patent CN118675072A proposes an infrared image detection method for dam seepage based on an improved deep learning model, and uses a Yolov5 model that introduces GhostNet and CBAM attention mechanisms for seepage detection.

[0004] Combining the five types of dam seepage detection methods mentioned above, it can be seen that single image types or single technical means are often limited in complex environments. For example, although traditional image processing methods are simple and efficient, they are difficult to handle complex scenes with large changes in lighting or concealed targets; while high-precision models such as deep learning have powerful feature extraction and adaptive capabilities, their detection accuracy is limited by the completeness of the original image information, and their accuracy depends on the richness and diversity of the input data. UAVs equipped with visible light and thermal infrared dual-modal sensors can acquire structural information and temperature anomalies on the dam surface from different dimensions, offering the following unique advantages: visible light images reflect details such as the dam surface structure, texture, and color, helping to determine surface damage and abnormal changes; thermal infrared images can intuitively display temperature distribution, enabling the detection of internal hazards such as seepage and infiltration that are difficult to detect with the naked eye. By using UAVs for flexible and mobile full-coverage inspections to collect these two complementary types of information, the spatial coverage of seepage detection can be improved, providing a richer and more discriminative data foundation for subsequent multimodal fusion and deep learning algorithms. Furthermore, complex environments vary greatly, and seepage signs are diverse, making it difficult for existing methods to be "universal and versatile." Therefore, it is of great significance to continuously develop detection methods that are more adaptable to complex and ever-changing real-world environments. Summary of the Invention

[0005] The purpose of this application is to provide a method and apparatus for detecting leakage areas based on UAV imagery, which can improve the reliability of target leakage area detection.

[0006] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a method for detecting leakage areas based on UAV imagery, the method comprising: The visible light camera and thermal infrared camera mounted on the drone are used to acquire visible light images and thermal infrared images of the area to be detected, respectively. Acquire preliminary leak areas and preliminary non-leak areas in visible light images; The thermal infrared image is filtered to obtain a thermal infrared filtered image, and the preliminary leakage area and preliminary non-leakage area in the thermal infrared filtered image are obtained. Based on the preliminary leakage area and preliminary non-leakage area in the visible light image, and the preliminary leakage area and preliminary non-leakage area in the thermal infrared filtered image, the target leakage area in the thermal infrared filtered image is determined. A mask map of the thermal infrared filtered image is generated based on the target leakage area in the preliminary screening. The pixel corresponding to the target leakage area in the mask map is marked as true, and other pixels are marked as false. The Otsu binarization algorithm is used to obtain the binary image of the thermal infrared filtered image. In the binary image, the foreground is marked as real and the background is marked as false. Based on the mask image and the binary image, obtain the pixels that are simultaneously marked as real, and use them as the target pixels corresponding to the target leakage area; The target leakage area in the region to be detected is obtained based on the target pixel.

[0007] Secondly, this application provides a leakage area detection device based on UAV imagery, the device comprising: The first acquisition module is used to acquire visible light images and thermal infrared images of the area to be detected by using a visible light camera and a thermal infrared camera mounted on the UAV, respectively. The second acquisition module is used to acquire the initial leakage area and the initial non-leakage area in the visible light image; The third acquisition module is used to filter the thermal infrared image to obtain a thermal infrared filtered image, and to acquire the preliminary leakage area and the preliminary non-leakage area in the thermal infrared filtered image. The determination module is used to determine the target leakage area in the thermal infrared filtered image based on the initial leakage area and initial non-leakage area in the visible light image, and the initial leakage area and initial non-leakage area in the thermal infrared filtered image. The module generates a mask image of the thermal infrared filtered image based on the target leakage area. In the mask image, the pixel corresponding to the target leakage area is marked as true, and other pixels are marked as false. The labeling module is used to obtain a binary image of the thermal infrared filtered image using the Otsu binarization algorithm. In the binary image, the foreground is labeled as true and the background is labeled as false. The fourth acquisition module is used to acquire pixels that are simultaneously marked as real based on the mask image and the binary image, and use them as the target pixels corresponding to the target leakage area; The fifth acquisition module is used to acquire the target leakage area in the area to be detected based on the target pixel points.

[0008] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides a method and apparatus for detecting leakage areas based on UAV imagery. The method includes: first, acquiring visible light and thermal infrared images of the area to be detected using a visible light camera and a thermal infrared camera mounted on a UAV, respectively; then, acquiring preliminary leakage and non-leakage areas from the visible light images; simultaneously, filtering the thermal infrared images to obtain filtered thermal infrared images, and acquiring preliminary leakage and non-leakage areas from the filtered images; finally, based on the preliminary leakage and non-leakage areas in the visible light and thermal infrared filtered images... This paper identifies preliminary target seepage areas in the thermal infrared filtered image. A mask image of the thermal infrared filtered image is then generated based on these preliminary target seepage areas. In the mask image, pixels corresponding to the preliminary target seepage areas are marked as "true," while other pixels are marked as "false." Simultaneously, a binary image of the thermal infrared filtered image is obtained using the Otsu binarization algorithm. In this binary image, the foreground is marked as "true," and the background as "false." Finally, based on the mask image and the binary image, pixels simultaneously marked as "true" are obtained and used as the target pixels corresponding to the target seepage areas. The target seepage areas in the detection area are then identified based on these target pixels. This disclosure employs a strategy of fusing visible light and thermal infrared images. Visible light images provide rich texture information, and a random forest algorithm is used for land cover classification to determine preliminary seepage areas and preliminary non-seepage areas. This is then combined with the results obtained from the thermal infrared filtered image using the K-Means clustering algorithm to filter out the target seepage areas. The two image datasets complement each other, effectively reducing the impact of environmental interference on thermal infrared detection and improving the reliability of target seepage area detection. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating a method for detecting leakage areas based on UAV imagery, provided in an embodiment of this application; Figure 2 A schematic diagram of the functional modules of a leak area detection device based on UAV imagery provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0011] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0013] In modern engineering, leakage monitoring technology is widely used in the maintenance of critical infrastructure in industries such as manufacturing, construction, water conservancy, and transportation, especially in ensuring the safety of structures such as underground utility tunnels, dams, pipelines, and bridges. Timely detection and accurate location of leakage areas are not only crucial for maintaining structural integrity but also have a profound impact on reducing potential economic losses. However, with the aging of infrastructure and the increasing complexity of environmental conditions, traditional leakage detection methods are gradually failing to meet the demands for high-efficiency and high-precision monitoring.

[0014] Currently, thermal infrared detection technology is one of the most commonly used methods for leak detection, identifying leak areas by capturing temperature anomalies on the surface of an object. This technology is favored due to its ease of operation and rapid response. However, the reliability of thermal infrared detection is significantly insufficient in complex environments (such as high background temperature, multiple interfering heat sources, or weak leak signals). These external factors often severely affect the detection results, leading to high false alarm rates and missed detections. Especially when the thermal signal of the leak area is weak or the area is small, a single thermal infrared detection method often struggles to effectively distinguish the actual leak signal from environmental noise.

[0015] To address these issues, several improvements are emerging, including optimizations of thermal infrared segmentation algorithms and the introduction of machine learning methods. However, much research remains focused on optimizing single models, lacking a comprehensive and systematic technical process. Existing algorithms often rely on human experience for data analysis, such as manually setting temperature difference thresholds to screen for leaks. This method is not only complex but also highly subjective, making it unsuitable for monitoring tasks requiring high levels of automation. Furthermore, traditional thermal infrared segmentation algorithms typically lack flexibility when handling complex scenarios and cannot effectively adapt to different monitoring needs. These algorithms often fail under conditions of small temperature differences or significant environmental interference, failing to meet the demands of practical applications. These technical challenges have driven the research and application of more advanced multi-constraint optimization segmentation methods.

[0016] Figure 1 This is a flowchart illustrating a leak area detection method based on UAV imagery according to an exemplary embodiment, such as... Figure 1 As shown, the method includes the following steps S101-S107: In step S101, visible light images and thermal infrared images of the area to be detected are acquired by the visible light camera and thermal infrared camera mounted on the UAV, respectively.

[0017] In one embodiment, step S101 includes the following sub-steps S1011-S1013: S1011. Using a visible light camera and a thermal infrared camera mounted on a drone, acquire multiple visible photon images and multiple thermal infrared images of the area to be detected.

[0018] S1012. Stitch together multiple visible photon images to obtain a visible light image.

[0019] Specifically, image correction can be performed on each of the visible photon images; image enhancement can be performed on each corrected visible photon image; and the enhanced visible photon images can be stitched together to obtain a visible light image.

[0020] S1013. Stitch together multiple thermal infrared sub-images to obtain a thermal infrared image.

[0021] Specifically, image correction can be performed on each of the thermal infrared sub-images; image enhancement can be performed on each corrected thermal infrared sub-image; and the enhanced thermal infrared sub-images can be stitched together to obtain a thermal infrared image.

[0022] The system acquires imagery of all planned paths within the area to be inspected, obtaining multiple visible light and thermal infrared images of the area. The UAV system is equipped with both visible light and thermal infrared cameras, and can also integrate a high-precision GPS positioning module and an inertial measurement unit. Using a pre-set algorithm, it generates planned flight paths based on the spatial extent of the area to be inspected and sets the aerial photography altitude. To ensure the clarity and effectiveness of the acquired images, the aerial photography altitude is set between 20 and 50 meters. The two cameras have synchronous shooting capabilities, simultaneously acquiring visible light and thermal infrared images of the area to be inspected. Each image contains the specific time, geographical location, and aerial photography altitude corresponding to its capture.

[0023] Preprocessing and stitching multiple visible photon and thermal infrared images captured by drones aids in the classification of the stitched images, providing important auxiliary judgment for K-Means clustering results. Image preprocessing includes image geometric correction, color space transformation for image enhancement, and geometric registration image stitching techniques to ensure the geometric accuracy and clear features of the images, generating stitched images with smooth edges and no obvious gaps that cover a larger area.

[0024] In step S102, the preliminary leakage area and the preliminary non-leakage area in the visible light image are obtained.

[0025] In one embodiment, step S102 includes the following sub-steps: S1021. Use the random forest algorithm to classify land features in visible light images and generate a land feature category map.

[0026] S1022. Obtain the preliminary leakage area and preliminary non-leakage area in the visible light image based on the land cover category map.

[0027] The Random Forest algorithm is selected to classify land features in visible light imagery, generating a land feature category map. Based on the classification results, preliminary infiltrated and non-infiltrated areas in the visible light imagery are initially determined. At this stage, the preliminary non-infiltrated areas can be selected based on empirical rules, such as excluding water bodies and hardened surfaces (concrete roads). After determining the preliminary infiltrated and non-infiltrated areas in the visible light imagery, a mask map is generated. In this mask map, pixels corresponding to preliminary infiltrated areas are marked as true (labeled as 1), and pixels corresponding to preliminary non-infiltrated areas are marked as false (labeled as 0).

[0028] Ground features refer to entities or regions on the Earth's surface that have clear boundaries and unique characteristics, and may include: (1) Natural features: Water bodies: rivers, lakes, reservoirs, oceans, etc., with typical spectral characteristics of high reflectance in the blue band (e.g., water appears blue), smooth texture (still water surface) or dynamic ripples (flowing water body).

[0029] Vegetation: forests, grasslands, farmland, etc., often appear green or bright green in images because chlorophyll absorbs red light and reflects near-infrared light (for example, dense forests are darker in color, while sparse grasslands are lighter in color).

[0030] Bare land and rocky areas: deserts, bare soil, mountain rocks, etc., with high spectral reflectance (grayish-white), rough texture (such as uneven rock surfaces), and no vegetation cover.

[0031] Ice and snow: Glaciers, snow cover, etc., reflect light highly across the entire spectrum and appear white in images. They are commonly found in high-altitude or high-latitude regions.

[0032] (2) Artificial features: Buildings and infrastructure: houses, roads, bridges, airports, etc., mostly made of materials such as concrete and asphalt, with medium spectral reflectance (grayish), regular shape (such as rectangular buildings and straight roads), and relatively uniform texture.

[0033] Agriculture and artificial vegetation: farmland (such as rice paddies and wheat fields), artificial forests, etc., which differ from natural vegetation in that they are more regularly distributed (such as farmland arranged in blocks) and change significantly with the seasons (such as the color of crops changing from green to yellow during the growth cycle).

[0034] Artificial water bodies include ponds, canals, and artificial lakes, which are often distinguished from natural water bodies. They are mostly regular geometric shapes (such as rectangular ponds) and may be adjacent to farmland or buildings.

[0035] (3) Special features: Ecological monitoring scenarios: wetlands (combining water and vegetation characteristics), saline-alkali land (bare soil with high salinity, appearing whitish in the spectrum), and mining areas (areas where bare soil is mixed with industrial facilities).

[0036] Urban planning scenarios: commercial areas (dense high-rise buildings, grayish spectrum), residential areas (mixed with low-rise buildings and green spaces), and industrial areas (large factories combined with roads).

[0037] The Random Forest algorithm is used to classify land cover in visible light imagery and generate land cover category maps. The specific process is as follows: First, representative training samples are selected from the visible light imagery, i.e., pixel regions corresponding to different land cover (such as vegetation, water bodies, buildings, etc.) are manually labeled. The spectral features (such as RGB band values ​​and spectral reflectance) and spatial features (such as texture and shape context) of each pixel are extracted as input features. Then, these training samples are input into the Random Forest model. This model constructs multiple decision trees (each tree randomly selects some features and samples for training), and uses ensemble learning to allow each tree to vote on the classification of samples, ultimately determining the optimal classification rule. After training, the features of each pixel in the entire visible light imagery are input into the model. The model predicts each pixel based on the rules obtained from training, determining which land cover category it belongs to. Finally, a corresponding land cover category label is assigned to each pixel according to the prediction results, and an intuitive land cover category map is generated through color or legend encoding. The colors or patterns of different areas in the map represent different land cover types, thereby realizing the visualization of the spatial distribution of various land cover types in the imagery. The Random Forest algorithm is suitable for land cover classification tasks in visible light imagery due to its ability to effectively handle high-dimensional features, strong noise resistance, and lack of complex parameter tuning. It can improve the accuracy and robustness of classification.

[0038] In step S103, the thermal infrared image is filtered to obtain a thermal infrared filtered image, and the preliminary leakage area and preliminary non-leakage area in the thermal infrared filtered image are obtained.

[0039] In one embodiment, the step S103 of acquiring the preliminary leakage area and the preliminary non-leakage area in the thermal infrared filtered image includes S1031-S1032: S1031. The thermal infrared filtered image is classified using the K-Means clustering algorithm to obtain the category result map.

[0040] S1032. Obtain the preliminary leakage area and preliminary non-leakage area in the thermal infrared filtered image based on the category result map.

[0041] The acquired thermal infrared image is filtered to suppress image noise and enhance the robustness and accuracy of subsequent algorithms. Lee filtering is a classic adaptive filtering method, particularly suitable for processing random noise in images, and can preserve structural details. After Lee filtering, the pixel matrix of the thermal infrared image is... Transformed into thermal infrared filtered image .

[0042] For thermal infrared filtered images, the K-Means clustering algorithm is used for classification to generate a category result map. The optimal number of clusters k is determined by introducing the elbow method. opt To ensure the scientific validity and accuracy of the classification results, the K-Means algorithm is a mainstream unsupervised learning algorithm suitable for dividing image data into different clusters. Determining the optimal number of clusters using the elbow method is a standard practice. For thermal infrared filtered images... The generated category result image : The category map, as a visual representation, displays the distribution of different categories. After K-Means clustering, each pixel in the thermal infrared filtered image is assigned to a category, and different categories are represented by different colors. The generated category map helps in analyzing cold / heat sources and object boundaries within the image.

[0043] Specifically, the first step is to analyze the actual temperature range corresponding to the category, and combine the temperature characteristics of the seepage area (such as the difference between the water temperature at normal temperature and the surrounding land features) to determine the low-temperature or high-temperature category of the suspected seepage. Then, by setting a temperature threshold, areas that do not meet the seepage temperature characteristics are filtered out. Morphological operations are used to remove isolated noise and retain continuous areas with a certain area and regular shape. Next, the land feature classification results of visible light images are combined to exclude non-seepage areas such as natural water bodies. The spatial correlation characteristics between the seepage area and the water source are used for further verification and screening. Finally, areas that meet the criteria of temperature anomaly, spatial continuity and seepage logic are determined as preliminary seepage areas, and the remaining areas (such as high-temperature building areas, natural water body areas, and low-temperature areas that do not meet the characteristics) are classified as preliminary non-seepage areas.

[0044] In step S104, based on the preliminary leakage area and preliminary non-leakage area in the visible light image, and the preliminary leakage area and preliminary non-leakage area in the thermal infrared filtered image, the target leakage area in the thermal infrared filtered image is determined. A mask image of the thermal infrared filtered image is generated based on the target leakage area in the preliminary screening. The pixel corresponding to the target leakage area in the preliminary screening in the mask image is marked as true, and other pixels are marked as false.

[0045] In one embodiment, step S104 includes the following sub-steps: S1041. Obtain the area where the preliminary leakage area in the visible light image and the preliminary leakage area in the thermal infrared filtered image overlap, and use it as the target leakage area for preliminary screening in the thermal infrared filtered image.

[0046] S1042. Generate a mask image corresponding to the visible light image based on the initially screened target leakage areas. In the mask image, the pixels corresponding to the initially screened target leakage areas are marked as true, and other pixels are marked as false.

[0047] By combining the K-Means classification results map and the land cover category map, preliminary target seepage areas are obtained. First, the preliminary seepage areas in the K-Means classification results map are examined to determine whether these categories fall within the preliminary seepage areas defined by the land cover category map. K-Means categories that fall within the area are marked as "true" (represented by "1"), while categories that do not fall within the area are marked as "false" (represented by "0"). This method generates a high-quality mask map M(i,j), where: If pixel (i, j) belongs to the preliminary infiltration area defined by the land cover category map, then M(i, j) = 1; If pixel (i, j) does not belong to the initial infiltration area defined by the land cover category map, then M(i, j) = 0; The resulting mask image can effectively aid subsequent analysis by providing clear target information.

[0048] In step S105, a binary image of the thermal infrared filtered image is obtained by using the Otsu binarization algorithm. In the binary image, the foreground is marked as true and the background is marked as false.

[0049] Based on the global grayscale characteristics of thermal infrared filtered images, the Otsu binarization algorithm is applied to achieve optimal segmentation of the image grayscale distribution. This method adaptively determines the optimal threshold t for binary segmentation based on the idea of ​​maximizing inter-class variance. opt By binarizing the thermal infrared filtered image using an optimal threshold, a binary image B(i, j) is generated, where a value of 1 represents the foreground and 0 represents the background. The specific rules are as follows: ; In step S106, based on the mask image and the binary image, the pixels that are simultaneously marked as real are obtained as the target pixels corresponding to the target leakage area.

[0050] By performing a logical intersection operation, M(i, j) and B(i, j) are jointly extracted to generate a double-constrained result map R(i, j). Specifically, M(i, j) and B(i, j) are combined pixel by pixel, with the following logical expression: ; The symbol ^ represents a logical AND operation. Only when M(i, j) = 1 and B(i, j) = 1 will the pixel value of the resulting image R(i, j) be 1, which means that the pixel belongs to the target leakage area.

[0051] In step S107, the target leakage area in the region to be detected is obtained based on the target pixel.

[0052] Specifically, the region contour of the target leakage area can be extracted based on the target pixels; and the target leakage area in the area to be detected can be obtained based on the region contour.

[0053] The boundary contour of the target leakage area is generated by analyzing the dual-constraint result map R(i, j) using a contour extraction algorithm, which is then used for subsequent morphological analysis. The specific steps are as follows: Pixel region identification. All pixel regions with a value of "1" are extracted from the dual-constraint result map. Connectivity checks are performed on these regions. Adjacent pixels (top / bottom, left / right, diagonal) form connected regions, which are then classified as independent target leakage regions. Disconnected regions, although they cannot be grouped together, are still considered independent. This process uses connected component analysis techniques to ensure that all target leakage regions are complete and independent.

[0054] Contour extraction. After completing the connected component analysis, a contour extraction algorithm (Canny edge detection or contour detection method) is applied to obtain the outer boundary of each target leakage region. This will generate a series of contour data.

[0055] Vectorized storage. The extracted target leakage area contour is stored in vector form, forming a complete target boundary contour map. This vector graphic structure facilitates subsequent morphological analysis and other calculations.

[0056] In one possible approach, the region outline can be overlaid with both visible light and thermal infrared images to display the boundary and temperature distribution changes of the target leakage area.

[0057] By combining the optimized region contour with thermal infrared temperature field imagery, the temperature distribution of the target leakage area is visualized. The specific steps are as follows: the extracted optimized contour is overlaid with the thermal infrared image, ensuring the contour accurately covers the corresponding position in the thermal infrared image. Through contour overlay, the boundary range of the target leakage area and the corresponding temperature distribution changes can be intuitively displayed, facilitating the analysis of the physical and thermal characteristics of the target leakage area. The generated combined image provides temperature distribution information overlaid on the boundary of the target leakage area, which can be used for subsequent intuitive analysis and judgment.

[0058] By combining the optimized region contour with visible light imagery, the target leakage area is visualized under visible light. The specific steps are as follows: the extracted optimized contour is overlaid with the visible light image, ensuring the contour accurately covers the corresponding visible light image. Through contour overlay, the boundary range of the target leakage area can be intuitively displayed, thereby better understanding the on-site environment and conditions related to visible light, facilitating the analysis of the appearance characteristics of the target leakage area and its relationship with the surrounding environment. The generated image shows the combination of the target leakage area's boundary and visible light image information, enabling users to identify the target leakage area in the actual environment.

[0059] This disclosure enables efficient detection and accurate analysis of potential target leakage areas. The method fully integrates visible light and thermal infrared image data, utilizing advanced K-Means clustering and Otsu threshold segmentation algorithms. By comprehensively leveraging the characteristics of visible light ground cover classification and thermal infrared imagery, it improves the reliability and accuracy of monitoring, ensuring precise identification and clear visualization of target leakage areas and reducing false alarm rates. The acquired images undergo a combination of image correction, image enhancement, and image stitching to guarantee high image quality and geometric accuracy. By applying algorithms such as random forest and K-Means clustering, the system can effectively distinguish between preliminary leakage areas and preliminary non-leakage areas, and the Otsu threshold segmentation algorithm improves the accuracy of binarization results, thereby accurately extracting contours. Finally, the optimized contours are superimposed onto thermal infrared and visible light images, visually displaying the boundaries and temperature distribution characteristics of the target leakage area. This intuitive visualization method greatly enhances the analysis of the physical characteristics of the target leakage area, enabling users to make quick and accurate judgments and decisions, thereby improving the efficiency of on-site monitoring and real-time response capabilities. It provides important support for the prevention and treatment of potential leakage problems, and is particularly suitable for the maintenance of critical infrastructure such as underground pipe corridors, dams and building facilities, demonstrating significant engineering promotion value.

[0060] The above analysis shows that: (1) Traditional thermal infrared detection is easily interfered with in complex environments. This disclosure adopts a strategy of fusing visible light images and thermal infrared images. Visible light images provide rich texture information of ground features. Ground features are classified using a random forest algorithm to determine preliminary seepage areas and preliminary non-seepage areas. Then, the results obtained by K-Means clustering algorithm on the thermal infrared filtered images are combined to screen out the target seepage areas. The two types of image data complement each other, effectively reducing the impact of environmental interference on thermal infrared detection and improving the reliability of detection.

[0061] (2) This disclosure constructs a complete leakage area detection process, from UAV image acquisition to the processing of visible light and thermal infrared images, and then to the use of multiple algorithms for area screening and determination, forming a systematic technical solution. First, the preliminary leakage area is obtained by using visible light and thermal infrared images respectively. Then, the target leakage area is initially screened based on the overlapping area of ​​the two. Finally, the target leakage area is determined by combining the binary image and mask image obtained by the Otsu binarization algorithm, thus avoiding the limitations of single model optimization.

[0062] (3) Solving the problem of traditional algorithms relying on human experience and being highly subjective: This disclosure uses the random forest algorithm to classify land features in visible light images and the K-Means clustering algorithm to classify thermal infrared filtered images. These algorithms are based on automatic data processing, reducing subjective operations such as manually setting thresholds. By automatically analyzing image data through algorithms, the objectivity and automation of detection are improved, meeting the needs of highly automated monitoring tasks.

[0063] (4) Solving the problem of insufficient flexibility of traditional thermal infrared segmentation algorithms: By combining multiple algorithms and multi-source data fusion, the adaptability of the detection method to different scenarios is enhanced. Under different temperature differences and environmental interference conditions, combined with the information provided by visible light images, it can handle complex scenarios more flexibly, effectively avoiding the detection failure problem caused by insufficient flexibility of traditional thermal infrared segmentation algorithms in complex scenarios.

[0064] (5) Improve detection accuracy and visualization: By extracting the regional contour of the target leakage area and superimposing it with visible light image and thermal infrared filtered image, not only can the leakage area be accurately determined, but the boundary and temperature distribution changes of the target leakage area can also be displayed intuitively, which further improves the accuracy and visualization effect of the detection results, making it easier for staff to analyze and make decisions.

[0065] Based on the same inventive concept, this application also provides a UAV-based imagery-based leakage area detection device for implementing the aforementioned UAV-based imagery-based leakage area detection method. The solution provided by this device is similar to the implementation described in the above method. Therefore, the specific limitations of one or more UAV-based imagery-based leakage area detection device embodiments provided below can be found in the limitations of the UAV-based imagery-based leakage area detection method described above, and will not be repeated here.

[0066] In one exemplary embodiment, such as Figure 2 As shown, a leak detection device based on UAV imagery is provided, comprising: The first acquisition module 11 is used to acquire visible light images and thermal infrared images of the area to be detected by the visible light camera and thermal infrared camera mounted on the UAV, respectively. The second acquisition module 12 is used to acquire the initial leakage area and the initial non-leakage area in the visible light image; The third acquisition module 13 is used to filter the thermal infrared image to obtain a thermal infrared filtered image, and to acquire the preliminary leakage area and the preliminary non-leakage area in the thermal infrared filtered image. The determination module 14 is used to determine the target leakage area in the thermal infrared filtered image based on the initial leakage area and initial non-leakage area in the visible light image, and the initial leakage area and initial non-leakage area in the thermal infrared filtered image, and to generate a mask map of the thermal infrared filtered image based on the initial target leakage area. In the mask map, the pixel corresponding to the initial target leakage area is marked as true, and other pixels are marked as false. The labeling module 15 is used to obtain a binary image of the thermal infrared filtered image through the Otsu binarization algorithm. In the binary image, the foreground is labeled as true and the background is labeled as false. The fourth acquisition module 16 is used to acquire pixels that are simultaneously marked as real based on the mask image and the binary image, and use them as target pixels corresponding to the target leakage area. The fifth acquisition module 17 is used to acquire the target leakage area in the area to be detected based on the target pixel.

[0067] As an optional implementation, the second acquisition module 12 is specifically used for: The random forest algorithm is used to classify ground features in visible light images and generate a ground feature category map. Based on the aforementioned land cover category map, the preliminary infiltrated areas and preliminary non-infiltrated areas in the visible light image are obtained.

[0068] As an optional implementation, the third acquisition module 13 is specifically used for: The K-Means clustering algorithm was used to classify the thermal infrared filtered images to obtain the category result map; Based on the category result map, obtain the preliminary leakage area and preliminary non-leakage area in the thermal infrared filtered image.

[0069] As an optional implementation, the determining module 14 is specifically used for: The area where the initial leakage area in the visible light image and the initial leakage area in the thermal infrared filtered image overlap is used as the target leakage area for initial screening in the thermal infrared filtered image. A mask image corresponding to the visible light image is generated based on the initially screened target leakage areas. In the mask image, the pixels corresponding to the initially screened target leakage areas are marked as true, and other pixels are marked as false.

[0070] As an optional implementation, the fifth acquisition module 17 is specifically used for: Extract the region contour of the target leakage area based on the target pixels; The target leakage area in the area to be detected is obtained based on the area contour.

[0071] As an optional implementation, the device further includes: The display module is used to overlay the region outline with visible light images and thermal infrared images respectively, so as to display the boundary and temperature distribution changes of the target leakage area.

[0072] As an optional implementation, in obtaining preliminary infiltrated areas and preliminary non-infiltrated areas in the visible light image based on the land cover category map, the second acquisition module 12 is specifically used for: The areas corresponding to water bodies and hardened surfaces in the land feature category map are identified as preliminary non-permeable areas, while other areas are identified as preliminary permeable areas.

[0073] As an optional implementation, the first acquisition module 11 is specifically used for: Multiple visible light and thermal infrared images of the area to be detected are acquired using visible light and thermal infrared cameras mounted on the drone. Multiple visible photon images are stitched together to obtain a visible light image; Multiple thermal infrared sub-images are stitched together to obtain a thermal infrared image.

[0074] As an optional implementation method, In terms of stitching together multiple visible photon images to obtain a visible light image, the first acquisition module 11 is specifically used for: Image correction is performed on each of the aforementioned visible photon images; Image enhancement is performed on each corrected visible photon image; The enhanced visible photon images are stitched together to obtain a visible light image; In terms of stitching together multiple thermal infrared sub-images to obtain a thermal infrared image, the first acquisition module 11 is specifically used for: Image correction is performed on each of the aforementioned thermal infrared sub-images; Image enhancement is performed on each corrected thermal infrared sub-image; The enhanced thermal infrared sub-images are stitched together to obtain a thermal infrared image.

[0075] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 3As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media to run. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for detecting leakage areas based on UAV imagery.

[0076] Those skilled in the art will understand that Figure 3 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0077] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0078] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0079] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0080] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0081] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0082] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0083] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for detecting leakage areas based on UAV imagery, characterized in that, The method includes: The visible light camera and thermal infrared camera mounted on the drone are used to acquire visible light images and thermal infrared images of the area to be detected, respectively. Acquire preliminary leak areas and preliminary non-leak areas in visible light images; The thermal infrared image is filtered to obtain a thermal infrared filtered image, and the preliminary leakage area and preliminary non-leakage area in the thermal infrared filtered image are obtained. Based on the preliminary leakage area and preliminary non-leakage area in the visible light image, and the preliminary leakage area and preliminary non-leakage area in the thermal infrared filtered image, the target leakage area in the thermal infrared filtered image is determined. A mask map of the thermal infrared filtered image is generated based on the target leakage area in the preliminary screening. The pixel corresponding to the target leakage area in the mask map is marked as true, and other pixels are marked as false. The Otsu binarization algorithm is used to obtain the binary image of the thermal infrared filtered image. In the binary image, the foreground is marked as real and the background is marked as false. Based on the mask image and the binary image, obtain the pixels that are simultaneously marked as real, and use them as the target pixels corresponding to the target leakage area; The target leakage area in the region to be detected is obtained based on the target pixel.

2. The method according to claim 1, characterized in that, The acquisition of preliminary leaked and preliminary non-leaked areas in visible light images includes: The random forest algorithm is used to classify ground features in visible light images and generate a ground feature category map. Based on the aforementioned land cover category map, the preliminary infiltrated areas and preliminary non-infiltrated areas in the visible light image are obtained.

3. The method according to claim 1, characterized in that, The acquisition of the preliminary leakage area and preliminary non-leakage area in the thermal infrared filtered image includes: The K-Means clustering algorithm was used to classify the thermal infrared filtered images to obtain the category result map; Based on the category result map, obtain the preliminary leakage area and preliminary non-leakage area in the thermal infrared filtered image.

4. The method according to claim 1, characterized in that, The step of determining the preliminary target leakage area in the thermal infrared filtered image based on the preliminary leakage area and preliminary non-leakage area in the visible light image, and the preliminary leakage area and preliminary non-leakage area in the thermal infrared filtered image, and generating a mask map of the thermal infrared filtered image based on the preliminary target leakage area, includes: The area where the initial leakage area in the visible light image and the initial leakage area in the thermal infrared filtered image overlap is used as the target leakage area for initial screening in the thermal infrared filtered image. A mask image corresponding to the visible light image is generated based on the initially screened target leakage areas. In the mask image, the pixels corresponding to the initially screened target leakage areas are marked as true, and other pixels are marked as false.

5. The method according to claim 1, characterized in that, The step of obtaining the target leakage area in the region to be detected based on the target pixels includes: Extract the region contour of the target leakage area based on the target pixels; The target leakage area in the area to be detected is obtained based on the area contour.

6. The method according to claim 5, characterized in that, The method further includes: The region outline is overlaid with visible light images and thermal infrared images respectively to show the boundary and temperature distribution changes of the target leakage area.

7. The method according to claim 2, characterized in that, The step of obtaining preliminary infiltrated areas and preliminary non-infiltrated areas in the visible light image based on the land cover category map includes: The areas corresponding to water bodies and hardened surfaces in the land feature category map are identified as preliminary non-permeable areas, while other areas are identified as preliminary permeable areas.

8. The method according to claim 1, characterized in that, The process of acquiring visible light and thermal infrared images of the area to be detected using a visible light camera and a thermal infrared camera mounted on a drone includes: Multiple visible light and thermal infrared images of the area to be detected are acquired using visible light and thermal infrared cameras mounted on the drone. Multiple visible photon images are stitched together to obtain a visible light image; Multiple thermal infrared sub-images are stitched together to obtain a thermal infrared image.

9. The method according to claim 8, characterized in that, The process of stitching together multiple visible photon images to obtain a visible light image includes: Image correction is performed on each of the aforementioned visible photon images; Image enhancement is performed on each corrected visible photon image; The enhanced visible photon images are stitched together to obtain a visible light image; The process of stitching together multiple thermal infrared sub-images to obtain a thermal infrared image includes: Image correction is performed on each of the aforementioned thermal infrared sub-images; Image enhancement is performed on each corrected thermal infrared sub-image; The enhanced thermal infrared sub-images are stitched together to obtain a thermal infrared image.

10. A leakage area detection device based on UAV imagery, characterized in that, The device includes: The first acquisition module is used to acquire visible light images and thermal infrared images of the area to be detected by using a visible light camera and a thermal infrared camera mounted on the UAV, respectively. The second acquisition module is used to acquire the initial leakage area and the initial non-leakage area in the visible light image; The third acquisition module is used to filter the thermal infrared image to obtain a thermal infrared filtered image, and to acquire the preliminary leakage area and the preliminary non-leakage area in the thermal infrared filtered image. The determination module is used to determine the target leakage area in the thermal infrared filtered image based on the initial leakage area and initial non-leakage area in the visible light image, and the initial leakage area and initial non-leakage area in the thermal infrared filtered image. The module generates a mask image of the thermal infrared filtered image based on the target leakage area. In the mask image, the pixel corresponding to the target leakage area is marked as true, and other pixels are marked as false. The labeling module is used to obtain a binary image of the thermal infrared filtered image using the Otsu binarization algorithm. In the binary image, the foreground is labeled as true and the background is labeled as false. The fourth acquisition module is used to acquire pixels that are simultaneously marked as real based on the mask image and the binary image, and use them as the target pixels corresponding to the target leakage area; The fifth acquisition module is used to acquire the target leakage area in the area to be detected based on the target pixel points.

Images