Dam leakage detection method and system based on unmanned aerial vehicle and deep learning

Abstract

The invention discloses a dam leakage detection method and system based on an unmanned aerial vehicle and deep learning. The method comprises the following steps: acquiring dam surface image data through the unmanned aerial vehicle carrying image acquisition equipment; preprocessing the obtained dam surface image data; performing feature extraction and fusion on the preprocessed image data based on a deep learning model to obtain a multi-modal coupling feature image related to the leakage area; performing target detection on the multi-modal coupling feature image to identify a leakage area; and carrying out spatial positioning on the identified leakage area to determine a positioning coordinate of the leakage area. Dam leakage is detected by combining the unmanned aerial vehicle technology and the deep learning algorithm, the automation and intelligence level of detection is improved, manual intervention and errors are reduced, and the detection precision and efficiency are improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of safety monitoring of hydraulic engineering, and particularly relates to a dam leakage detection method and system based on an unmanned aerial vehicle and deep learning. BACKGROUND

[0002] As one of the main diseases of the dam, leakage can cause water to abnormally seep out through channels such as cracks and holes, and if not handled in time, it will lead to structural instability and even dam collapse accidents.

[0003] The traditional dam leakage detection method mainly relies on manual inspection, and the manual inspection method faces three technical bottlenecks: first, the dam line is long and the terrain is complex, and manual inspection has the problems of heavy workload and poor timeliness; second, the traditional method is difficult to realize high-frequency and full-coverage detection; third, the inspection personnel face safety hazards such as high-altitude operation and bad weather. Although the unmanned aerial vehicle technology provides a new solution for dam detection, it can quickly obtain high-resolution image data, but there are still significant technical defects in actual application.

[0004] Firstly, the contradiction between massive inspection data and intelligent processing demand is prominent. The characteristics of long dam line and wide inspection range, combined with the requirement of image overlap degree that must be ensured by unmanned aerial vehicle aerial photography, result in TB-level data volume generated by single inspection, and existing image processing algorithms are difficult to meet the real-time analysis demand. Secondly, vegetation interference seriously affects the detection accuracy. The grass and shrub revetment set for the protection of the dam will produce similar thermal characteristics to leakage in the infrared image due to its water-containing characteristics, resulting in high false detection rate of the traditional infrared detection method.

[0005] In view of the above problems, the existing technology needs to be improved. SUMMARY

[0006] The purpose of the present application is to solve the problems in the background art, and to provide a dam leakage detection method and system based on an unmanned aerial vehicle and deep learning, which has the advantages of improving detection efficiency and accuracy, effectively eliminating vegetation interference, and realizing accurate positioning.

[0007] The technical scheme adopted by the present application is: a dam leakage detection method based on an unmanned aerial vehicle and deep learning, comprising the following steps: acquiring dam surface image data by an unmanned aerial vehicle carrying an image acquisition device; preprocessing the acquired dam surface image data; extracting and fusing features from the preprocessed image data based on a deep learning model to obtain a multi-modal coupled feature image related to a leakage area; detecting the leakage area by target detection on the multi-modal coupled feature image; spatially positioning the identified leakage area to determine the positioning coordinates of the leakage area.

[0008] Further, the dam surface image data obtained by the unmanned aerial vehicle carrying the image acquisition device comprises: An infrared camera is carried on the unmanned aerial vehicle, the unmanned aerial vehicle is controlled to fly along a set route, and infrared image data and visible light image data of the back of the dam are obtained to form the dam surface image data.

[0009] Further, before the preprocessing of the obtained dam surface image data, the method further comprises: The image range collected is cropped with the current water level of the water area as the cropping line, the image data above the cropping line is removed, and the image data below the water level is obtained.

[0010] Further, the preprocessing of the obtained dam surface image data comprises sequentially performing image denoising, image registration and image enhancement operations on the obtained dam surface image data.

[0011] Further, the feature extraction and fusion of the preprocessed image data based on the deep learning model to obtain the multi-modal coupled feature image related to the leakage area comprises: The CMAF-Net model is used to extract the infrared image temperature features and the visible light image vegetation texture features in the image data, and the CCA module is introduced in the feature extraction process to train from the channel and spatial attention dimensions to enhance the ability to distinguish the leakage and vegetation features. The physical constraint fusion network model is used to fuse the features of the extracted infrared image and visible light image, and the leakage area temperature field physical constraint is introduced in the fusion process to obtain the multi-modal coupled feature image.

[0012] Further, the target detection of the multi-modal coupled feature image to identify the leakage area comprises: The texture branch detection head and the temperature branch detection head are constructed, the texture branch detection head processes the texture feature information of the visible light image to obtain a texture anomaly probability map, and the temperature branch detection head processes the temperature feature information of the infrared image to obtain a temperature anomaly probability map. The texture anomaly probability map and the temperature anomaly probability map are cross-modal feature fused, the fused feature map is pixel-level detected and post-processed to obtain a continuous and complete leakage area contour.

[0013] Further, the cross-modal feature fusion is performed by the following formula: ; Wherein, is the fused feature map; is the temperature anomaly probability map; is the texture anomaly probability map.β is a weight.

[0014] Further, the pixel-level detection and post-processing comprises: An Otsu algorithm is used to calculate an optimal segmentation threshold of the fused feature map, and a binary detection mask is generated based on the optimal segmentation threshold; The binary detection mask is subjected to an erosion operation and a dilation operation to form a continuous and complete seepage area profile.

[0015] Still further, the spatial positioning of the identified seepage area comprises: The identified seepage area is subjected to spatial positioning by a high-precision GPS module built in the unmanned aerial vehicle; Or the identified seepage area is subjected to spatial positioning by using a multi-view oblique photogrammetry technology.

[0016] A dam seepage detection system based on an unmanned aerial vehicle and deep learning comprises An image acquisition module, which is an unmanned aerial vehicle carrying an image acquisition device, is used to acquire dam surface image data; An image preprocessing module is used to preprocess the acquired dam surface image data; A feature extraction and fusion module is used to extract and fuse features of the preprocessed image data based on a deep learning model to obtain a multi-modal coupling feature image related to a seepage area; A target detection and identification module is used to detect and identify a seepage area based on a deep learning network model on the multi-modal coupling feature image; A spatial positioning module is used to spatially position the identified seepage area to determine the positioning coordinates of the seepage area.

[0017] The present application has the following advantages: The present application uses an unmanned aerial vehicle to acquire images, avoids the problems of heavy workload and poor timeliness of manual inspection, and realizes high-frequency and full-coverage detection; the deep learning model is used for feature extraction and fusion, which improves the detection accuracy, can automatically identify the seepage area, and reduces the subjectivity and errors of manual judgment; the visible light and infrared image data are combined, which can more comprehensively capture the information of the dam surface and improve the reliability of the seepage detection. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The present application is a detection flowchart.

[0019] Figure 2 The present application is a general framework diagram of the deep learning network model.

[0020] Figure 3 The present application is a schematic diagram of the CCA module. Detailed Implementation

[0021] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings. It should be noted that these descriptions are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0022] like Figure 1 As shown, this invention provides a method for detecting dam seepage based on unmanned aerial vehicles (UAVs) and deep learning. The specific technical solution is as follows: 1) Drones quickly acquire image data of the dam surface.

[0023] Using a drone equipped with an infrared camera, infrared and visible light image data of the back of the dam were acquired to form an image dataset. Based on the actual characteristics of the dam project, a customized flight route was established, camera parameters were set, and data acquisition quality requirements were defined. Considering the need for later cropping of the acquired area to reduce data processing workload, if the drone and infrared camera equipment have built-in GPS positioning modules and geospatial positioning capabilities, the positioning data from the equipment or image data can be directly utilized. Otherwise, during drone flight for data acquisition, directional and lateral overlap were added according to the data acquisition technology requirements of oblique photogrammetry to facilitate later 3D reconstruction using oblique photogrammetry to extract elevation information.

[0024] Infrared cameras are optical devices capable of capturing the thermal radiation characteristics of objects. Specifically, they can be implemented using uncooled microbolometer infrared sensors, and their function is to detect leakage areas through temperature differences. Preset flight path refers to automatic navigation flight according to a preset sequence of latitude and longitude coordinates, which can be achieved using an RTK positioning system combined with waypoint planning software. Its function is to ensure that image acquisition covers the entire area behind the dam. The back of the dam refers to the opposite slope of the dam structure's water-facing side. Image acquisition can be achieved by controlling the UAV's flight altitude and tilt angle parameters, and its function is to directly monitor areas prone to leakage. Visible light image data refers to color images acquired through RGB sensors, specifically using optical lenses with at least 20 megapixels. Its function is to provide texture feature information of vegetation-covered areas.

[0025] Specifically, the infrared and visible light images are synchronously acquired through a dual-mode data acquisition method, wherein the temperature characteristics of the infrared image can represent the thermodynamic anomaly of the leakage area, and the texture characteristics of the visible light image can reflect the spatial distribution of the vegetation coverage. When the unmanned aerial vehicle flies according to the preset route, the flight height can be controlled within the range of 10 meters to 50 meters, the flight speed can be set to 3 meters to 5 meters per second, and the heading overlap degree is set to 70% to 80%, so as to ensure the spatial continuity and integrity of the acquired data. When the image acquisition is performed on the back of the dam, the gimbal carried by the unmanned aerial vehicle can be adjusted to an angle of 15 degrees to 30 degrees with the normal direction of the slope surface, so as to effectively avoid image distortion. Through the spatio-temporal alignment of the two data sources, a data basis is established for subsequent feature fusion and elimination of vegetation interference.

[0026] The present application provides vegetation texture information through a visible light image, combines temperature field data of an infrared image, constructs a discrimination basis that can distinguish two types of features, effectively reduces the false detection rate caused by vegetation slope protection in an infrared image, and improves the accuracy of leakage detection through a dual-mode data complementary mechanism. At the same time, the automatic flight mode based on the preset route ensures the complete acquisition of image data on the back of the dam, and avoids the monitoring blind area that may exist in manual patrol.

[0027] 2) Data acquisition area range clipping.

[0028] The dam leakage is formed due to the water pressure difference, and from the elevation level, therefore, the dam leakage point can only be below the water surface line. The water level line of the current water area can be used as a clipping line to clip the acquisition range of the unmanned aerial vehicle, and the image data above the clipping line is excluded to obtain the image data below the water level line, so as to reduce the data acquisition range and the amount of data to be processed, and improve the timeliness of data processing.

[0029] The current water level line of the water area refers to the real-time water level elevation of the water area (reservoir or river) where the dam is located, and can be determined through a water level sensor or historical hydrological data, and is used to divide the effective area of leakage detection. The image data above the clipping line refers to the part above the water level line in the image acquired by the unmanned aerial vehicle, which is marked through geographical spatial coordinate comparison, and can be realized by using a geographic information system or an image processing algorithm, and is used to exclude the invalid detection area.

[0030] Specifically, before image preprocessing, the dam surface image collected by the unmanned aerial vehicle is spatially cropped by the water line. The dam body area below the water line is the potential location of leakage, while the area above the water line is usually covered with slope vegetation. The cropping operation directly eliminates the image data above the water line through geographic coordinate mapping, and only retains the effective detection area below the water line. This step is implemented before data preprocessing, so that the subsequent deep learning model only processes the effective data related to leakage, avoiding the formation of temperature abnormality artifacts in infrared images due to the water-containing characteristics of vegetation. At the same time, the reduction of invalid data volume reduces the computational complexity of image registration and feature extraction.

[0031] The present application divides the physical leakage area boundary by the water line, actively eliminates invalid data, eliminates vegetation interference, reduces computational resource consumption, solves the problem of high leakage detection misjudgment rate caused by vegetation interference above the water line in dam surface image data, and at the same time reduces the processing amount of invalid data, improves the detection efficiency and accuracy.

[0032] 3) Data preprocessing.

[0033] The cropped infrared image and visible light image data are preprocessed, mainly including image denoising, image registration, image enhancement, etc.

[0034] (1) Image denoising Image denoising refers to eliminating random noise interference generated in the image acquisition process. The noise in the infrared image mainly includes thermal noise, shot noise, photon noise, atmospheric absorption and scattering noise, background radiation noise, etc. Convolutional Neural Network (CNN) is used for image denoising, and convolutional layer, pooling layer and full connection layer of CNN are established. Through the combination of convolutional layer, pooling layer and full connection layer, CNN can learn the feature pattern of noise in the process of denoising, automatically extract the features of the image, and remove them from the noisy image.

[0035] (2) Image registration Image registration refers to aligning the spatial coordinates of different modal images. Through a three-layer convolutional neural network (CNN), the features of infrared and visible light images are aligned, so that the consistency of corresponding pixel points in spatial position is achieved, so as to facilitate subsequent fusion, analysis, detection and other processing. SIFT algorithm is used for feature point detection and matching, corresponding points in the two images are found, affine transformation matrix is estimated based on the matching points, and affine transformation is performed on the infrared image. TV-L1 optical flow method is used to estimate the sub-pixel level displacement, and thin plate spline (TPS) transformation is further used to adjust the position of the infrared image, realizing sub-pixel level registration.

[0036] (3) Image enhancement Image enhancement refers to improving the visual saliency of target features in the image. By constructing a multi-scale pyramid decomposition (Gaussian / Laplacian pyramid), the image is analyzed and processed at different scales, thereby enhancing the feature expression of different leakage scales of the dam.

[0037] Specifically, in the preprocessing process of the dam surface image data, first, the image denoising operation is used to eliminate the image noise caused by the unmanned aerial vehicle flight vibration and the environmental light change; then, the denoised infrared image and the visible light image are registered, the spatial mapping relationship between the two images is established by extracting the SIFT feature points, and the pixel-level alignment of the cross-modal data is realized; finally, the registered image is subjected to contrast enhancement processing, the temperature gradient difference of the infrared image and the vegetation texture details of the visible light image are enhanced respectively, and the feature difference between the leakage area and the background environment is more obvious. The three processing steps form a progressive data optimization process, which eliminates noise interference, unifies the data space reference, and strengthens the feature contrast, so as to provide high-quality input data for the subsequent deep learning model.

[0038] By constructing the complete preprocessing chain including denoising, registration and enhancement, the present application systematically solves the problem of uneven quality of multi-source image data in complex engineering environment, especially realizes the spatial consistency of infrared and visible light data through cross-modal registration, and lays a foundation for multi-feature fusion analysis. Through the above technical scheme, the present application effectively reduces the interference of image noise on feature extraction, eliminates the feature matching error caused by the spatial misplacement of multi-modal data, and enhances the visual discrimination of leakage-related features, thereby improving the accuracy and reliability of the subsequent deep learning model for leakage area detection.

[0039] 4) Feature extraction and fusion.

[0040] In dam leakage detection, vegetation such as weeds and shrubs show similar characteristics to leakage areas in infrared images due to the presence of water, leading to misjudgment during detection. Therefore, feature extraction and fusion technology is needed to eliminate vegetation interference and improve detection accuracy. To this end, the CMAF-Net model (cross-modal attention feature extraction model) is used to extract temperature features from infrared images and vegetation texture features from visible light images. During the process, the CCA module is introduced for training from the channel and spatial attention dimensions. The channel attention focuses on the infrared low-temperature area and the visible light vegetation texture feature channel, and the spatial attention perceives the feature spatial position and shape to enhance the ability to distinguish between leakage and vegetation features. After extraction, the features are enhanced and fused using the PhysConFusion model. During the process, the heat conduction equation is introduced to construct a temperature-texture coupling model, forcing the temperature gradient and texture gradient direction of the leakage area to be consistent. At the same time, the cascade cross-attention mechanism (including channel and spatial attention modules) is used to enhance the attention training in two dimensions. By strengthening the attention of key channels and spatial positions, the fused image can effectively filter out vegetation interference during leakage detection, ultimately reducing the misjudgment rate and improving the detection accuracy of the leakage area.

[0041] The CMAF-Net model refers to a dual-modal feature extraction network, which can be implemented using a dual-branch convolutional neural network structure. The infrared branch is used to extract temperature features, and the visible light branch is used to extract vegetation texture features. The CCA module refers to a channel and spatial attention module. It can filter temperature-sensitive features through channel attention mechanisms and suppress vegetation texture interference areas through spatial attention mechanisms to enhance the ability to distinguish between leakage features and vegetation features. The PhysConFusion model refers to a neural network structure that introduces temperature field distribution rules during fusion. It can use temperature gradient constraints as loss functions to ensure that the fused features meet the physical characteristics of the temperature field in the leakage area.

[0042] Specifically, the temperature features of the infrared image and the vegetation texture features of the visible light image are input into the two branches of the CMAF-Net model. In the infrared branch, the channel attention mechanism is used to enhance the response of the temperature anomaly area, and the spatial attention mechanism is used to suppress the interference of the vegetation coverage area. In the visible light branch, the spatial attention mechanism is used to filter the vegetation details unrelated to leakage during texture feature extraction. Subsequently, the PhysConFusion network weights and fuses the feature maps output by the two branches, while introducing temperature field gradient constraints to ensure that the fused multi-modal features not only retain the temperature anomaly information of the infrared image but also meet the physical laws of the temperature field in the leakage area. This collaborative mechanism of dual-modal feature decoupling and physical constraint fusion effectively avoids the interference of vegetation texture on temperature features.

[0043] The present application realizes feature decoupling through attention mechanism, guides feature fusion combined with temperature field physical constraints, solves the multi-modal data conflict problem, can effectively distinguish the temperature feature difference between the vegetation coverage area and the real leakage area in the infrared image, and reduces the false detection rate caused by vegetation interference; meanwhile, through the multi-modal feature fusion guided by physical constraints, the boundary recognition accuracy of the leakage area is improved, so that the detection result is more in line with the physical characteristics of the actual leakage phenomenon.

[0044] 5) Target detection of the leakage area image.

[0045] The dual-flow U-Net deep learning network is a neural network architecture improved based on the traditional U-Net network, which can be used for target detection of the fused image to identify the leakage area therein. The network constructs two parallel branches of texture branch detection head and temperature branch detection head. The texture branch focuses on processing the texture feature information of the visible light image, and the temperature branch specially processes the temperature feature information of the infrared image, both of which extract features from different angles. In the feature extraction process, the temperature field heat conduction physical constraint of the leakage area is introduced. Since the temperature change of the leakage area follows the heat conduction physical law, by converting this physical property into a constraint condition for the network, the network can better capture the features related to the leakage. Then, the feature results extracted by the two branches are fused to further optimize the feature representation by using the complementarity of multi-modal information. Finally, pixel-level detection and post-processing are performed on the fused feature map to obtain a continuous and complete leakage area contour.

[0046] Among them, the texture branch detection head refers to a neural network structure for extracting vegetation texture feature information in the visible light image, which can be specifically implemented by using a convolutional neural network combined with an attention mechanism, to generate a probability distribution map representing the texture abnormal area by analyzing the texture regularity difference of the vegetation coverage area. The temperature branch detection head refers to a neural network structure for extracting temperature field distribution features in the infrared image, which can be specifically implemented by using a fully convolutional network combined with a heat conduction physical model, to generate a probability distribution map representing the temperature abnormal area by capturing the temperature gradient difference between the leakage area and the surrounding dam. Cross-modal feature fusion refers to joint analysis of the features extracted from different modal data, which can be specifically implemented by using weighted superposition or channel splicing, to eliminate interference factors in single modal data by combining the physical correlation of temperature field and texture field. Pixel-level detection and post-processing refer to pixel-by-pixel classification and morphological optimization of the fused feature map, which can be specifically implemented by using threshold segmentation combined with erosion and dilation operations, to eliminate isolated noise points by spatial continuity analysis.

[0047] Specifically, the vegetation covered area in the visible light image has regular texture features, while the leakage area will cause abnormal vegetation growth due to water infiltration, forming irregular textures. The texture branch detection head extracts multi-scale texture features through a convolution layer, and combines an attention mechanism to enhance the response value of the abnormal texture area, and outputs a texture anomaly probability map. In the infrared image, the leakage area forms a low-temperature anomaly due to the heat absorption effect of water evaporation. The temperature branch detection head extracts temperature field distribution features through a fully convolutional network, and combines a heat conduction equation to constrain the temperature gradient variation rule, and outputs a temperature anomaly probability map. The two probability maps are fused through a weighted superposition method. By using the characteristics of the vegetation area in the visible light texture anomaly but normal infrared temperature, the interference area misjudged as leakage by a single mode is suppressed. The fused feature map is segmented into a binary mask through an adaptive threshold, and then the discrete noise points are removed through a morphological erosion and dilation operation, and finally a spatially continuous leakage area contour is formed.

[0048] The present application extracts visible light texture features and infrared temperature features through the construction of double branch detection heads, establishes physical correlation constraints by using the feature differences of the vegetation area in the two modal data, and effectively suppresses the misjudgment caused by vegetation interference in the feature fusion stage.

[0049] Through the above technical solutions, the present application effectively solves the problem of high leakage detection false detection rate caused by vegetation interference in the infrared image. Through cross-modal fusion of visible light texture features and infrared temperature features, the vegetation covered area and the real leakage area are accurately distinguished, and the discrete noise points are eliminated through spatial continuity post-processing, which significantly improves the accuracy and reliability of leakage area detection, and is suitable for leakage detection scenes with complex vegetation environment on the dam back slope.

[0050] Specifically, after obtaining the fused feature map, first analyze its gray histogram distribution through the Otsu algorithm, automatically calculate the optimal threshold for segmenting the leakage area and the background, and convert the feature map into a binary detection mask containing only 0 and 255 gray values based on the threshold. Then, the mask is processed using a morphological erosion operation, which eliminates isolated areas with an area smaller than the size of the structure element by scanning each pixel with a structure element, effectively removing discrete pixel points caused by image noise or false detection. After completing noise elimination, further expand the boundary of the remaining leakage area contour through dilation operation, fill the small holes and contour breaks caused by segmentation threshold setting or feature discontinuity through structure element, and finally form a leakage area contour with complete closed boundary.

[0051] The present application combines adaptive threshold segmentation and cascade morphological operation, ensures the objectivity of the segmentation threshold, realizes the synergistic effect of noise elimination and contour repair, effectively solves the problem of isolated false detection area caused by noise interference in the segmentation process of the fusion feature map, repairs the contour break phenomenon caused by feature discontinuity, makes the leakage area detection result form a complete closed boundary in spatial distribution, and improves the accuracy of leakage area positioning and the integrity of contour description.

[0052] 6) Spatial positioning of the leakage area.

[0053] When the detection of the leakage area is completed, the next step is to deliver the detection result to the flood control workers, who need to go to the engineering site for verification and take corresponding engineering measures to implement leakage plugging treatment. In this process, accurate spatial positioning of the leakage area detection result is particularly critical. The unmanned aerial vehicle and other data collection platforms are generally equipped with high-precision GPS modules, so the geographic spatial coordinate information at the time of image collection can be directly obtained by using the module. By associating the leakage area with the geographic coordinates of the image, the spatial positioning of the leakage area can be realized, providing strong support for the flood control workers to quickly and accurately lock the leakage position. However, if the unmanned aerial vehicle or camera and other data collection equipment do not have positioning function, tilt photogrammetry and other technologies can also be used to achieve the spatial positioning goal. Tilt photogrammetry captures the target area from multiple different angles to obtain rich image data. Using these image data, three-dimensional reconstruction can be performed to construct a high-precision three-dimensional model containing terrain, building, and other information. Then, through feature point matching technology, the feature points of the leakage area in the two-dimensional image are matched with the corresponding feature points in the three-dimensional model. Based on the matching result, the position of the leakage area in the actual space can be calculated, thus completing the spatial positioning task and ensuring that the flood control workers can timely carry out on-site work according to the positioning result.

[0054] Among them, the high-precision GPS module refers to the satellite positioning device integrated in the unmanned aerial vehicle flight platform, which can be realized by using centimeter-level differential positioning technology. By receiving multi-band satellite signals and calculating real-time position coordinates, it can directly associate the pixel position of the leakage area in the infrared or visible light image with the geographic coordinates. Multi-view tilt photogrammetry technology refers to capturing multi-view images of the target area by the unmanned aerial vehicle carrying a multi-angle camera. Specifically, it can be realized by using the motion recovery structure algorithm, and based on the multi-view geometry principle, the three-dimensional spatial coordinates of the leakage area are reconstructed to eliminate the terrain occlusion error caused by single-view imaging.

[0055] Specifically, in an open terrain scene, when the flight trajectory of the unmanned aerial vehicle is stable and the satellite signal is well received, the high-precision GPS module can obtain the longitude and latitude coordinates of the leakage area in real time, the positioning data is synchronized with the image acquisition time stamp, and rapid coordinate mapping is realized. In a complex terrain scene, for example, there is a mountain obstruction or a vegetation covered area, a multi-view oblique photogrammetry technology acquires multi-angle images of the same leakage area, generates three-dimensional spatial coordinates of the leakage point by using a three-dimensional reconstruction algorithm, and overcomes the problem of precision decline caused by signal attenuation in single GPS positioning. The two positioning methods form a redundancy mechanism, which automatically switches or processes in parallel according to the actual environmental conditions, and ensures the positioning reliability in different working conditions.

[0056] The application further provides a dam leakage detection system based on an unmanned aerial vehicle and deep learning, which comprises an image acquisition module, an image preprocessing module, a feature extraction and fusion module, a target detection and identification module and a spatial positioning module.

[0057] The image acquisition module is an unmanned aerial vehicle carrying an infrared camera and a visible light camera, which can specifically acquire infrared images and visible light image data on the back of the dam through a preset flight route, so as to realize large-scale and rapid data acquisition. The image preprocessing module is an algorithm unit for denoising, registration and enhancement of original images, which can specifically eliminate infrared image noise by using Gaussian filtering, realize multi-source image registration by SIFT feature matching, and enhance image contrast by histogram equalization, so as to improve the subsequent processing precision. The feature extraction and fusion module is a deep learning architecture comprising a CMAF-Net model and a physical constraint fusion network, which can specifically extract temperature and texture features by using channel and spatial attention mechanisms, realize feature fusion by combining the physical law of the temperature field of the leakage area, and effectively distinguish vegetation interference and real leakage. The target detection and identification module is a double-path detection network with a texture branch and a temperature branch, which can specifically generate a leakage contour by cross-modal probability map fusion, and form a continuous region boundary by combining morphological post-processing. The spatial positioning module is a positioning unit integrated with a high-precision GPS or an oblique photogrammetry technology, which can specifically convert the detection result into an actual engineering coordinate system by geographic coordinate mapping, and realize accurate positioning of the leakage point. The specific ways in which each module realizes the corresponding functions have been described in the above detection method, and will not be repeated here.

[0058] Specifically, the unmanned aerial vehicle synchronously collects infrared and visible light images when flying along a preset route, a preprocessing module performs noise reduction and registration processing on the original data to eliminate image distortion caused by environmental interference. A feature extraction network extracts temperature anomaly features from the infrared images and vegetation texture features from the visible light images, and strengthens the leakage-related features through an attention mechanism. A physically constrained fusion network combines the two types of features with the temperature field distribution to generate a coupled feature map. A target detection network weights and fuses the temperature anomaly probability map and the texture anomaly probability map, and extracts complete leakage areas through threshold segmentation and morphological operations. A positioning module calculates the spatial coordinates of the leakage areas based on the flight trajectory of the unmanned aerial vehicle and the image acquisition position, and in combination with a three-dimensional geographic information model.

[0059] Embodiments The present application will be further described below in combination with specific parameter information.

[0060] 1) Unmanned aerial vehicle data collection An infrared camera is mounted on the unmanned aerial vehicle to automatically collect infrared image data of the back of the dam. Existing infrared thermal imagers can mostly collect infrared image data and visible light data simultaneously. According to the technical requirements of aerial photography for three-dimensional modeling, the heading overlap is required to be between 65%-80%, and the lateral overlap is required to be between 30%-60%. The collected data is preprocessed, mainly for image correction and feature matching, and the infrared images and visible light images are registered.

[0061] 2) Leakage detection range cropping The data collection area range is cropped using the common sense that the leakage point is below the water line. The specific steps are as follows: (1) Obtain the current water level data of the reservoir or river from the hydrological department sharing, and use the water level data as the elevation cropping line.

[0062] (2) If the unmanned aerial vehicle or camera device has a GPS module, use the GPS positioning function to obtain the elevation coordinate information of the collected image data.

[0063] (3) If the unmanned aerial vehicle or camera device does not have a GPS module and does not have a GPS positioning function, the data collection route is formulated according to the technical requirements of oblique photography for three-dimensional modeling when the unmanned aerial vehicle is flying, and a three-dimensional model is reconstructed using the collected visible light images to extract elevation information.

[0064] (4) According to the geographic spatial coordinate information (elevation information) of the collected images, the collected area is cropped using the elevation cropping line (the current water level of the reservoir or river), and the image data above the cropping line is excluded. Only the image data below the cropping line is analyzed and processed, thereby reducing the amount of image data required for processing and reducing the time required for image processing.

[0065] 3) Data preprocessing The pre-processing of the cropped infrared and visible light image data mainly includes image denoising, image registration, image enhancement, etc.

[0066] (1) Image denoising There are thermal noise, shot noise, photon noise, atmospheric absorption and scattering noise, background radiation noise, and other complex noises in the infrared image. In view of these noises, a convolutional neural network (CNN) is used for denoising. The CNN is composed of convolutional layers, pooling layers and fully connected layers. The convolutional layers perform convolution operations by sliding the convolution kernel over the image to extract various features including noise feature patterns, generating multiple feature maps. The pooling layer generally follows it to reduce the resolution of the feature map in the form of maximum pooling or average pooling, reduce the amount of calculation, suppress noise and highlight key information. The fully connected layer then comprehensively processes the flattened feature map to further distinguish image and noise features, and achieve noise removal. When training the model, the mean square error loss (MSE) function is used. A large number of noisy infrared images are input into the model, and the MSE value between the output after each iteration of the model and the original noise-free image is calculated. With the help of the back propagation algorithm, the parameters are adjusted according to the gradient to minimize the MSE value. The trained model is used for new image denoising, and the denoising effect is evaluated by the peak signal-to-noise ratio (PSNR). The larger the PSNR value, the closer the denoised image is to the original noise-free image, and the better the denoising effect.

[0067] (2) Image registration The features of the infrared and visible light images are aligned by a three-layer convolutional neural network (CNN), so that the consistency of the corresponding pixel points in the spatial position is achieved, so as to facilitate subsequent fusion, analysis, detection and other processing.

[0068] Affine transformation and thin plate spline (TPS) are used to achieve sub-pixel level registration of infrared and visible light. The SIFT algorithm (Scale-Invariant Feature Transform) is used for feature point detection and matching to find corresponding points in the two images. Based on the matching points, the affine transformation matrix is estimated, and the infrared image is subjected to affine transformation. The TV-L1 (Total Variation-L1) optical flow method is used to estimate the sub-pixel level displacement, and the thin plate spline (TPS) transformation is used to further adjust the position of the infrared image, achieving sub-pixel level registration.

[0069] The adaptive Retinex algorithm is used to enhance the visible light image, separate the reflection component and the illumination component in the image, eliminate the influence of illumination, and enhance the details and contrast of the image. The adaptive Retinex algorithm dynamically adjusts the size or weight of the Gaussian kernel according to the local contrast change of the visible light image. For the area with high contrast, a smaller Gaussian kernel is used to retain more details; for the area with large brightness change, the weight of the corresponding scale is increased. The adaptive Retinex algorithm can adaptively adjust the parameters of the algorithm according to the local features of the image, so as to better adapt to the illumination change in different areas.

[0070] (3) Image enhancement A multi-scale feature extraction and enhancement system is constructed to deeply analyze and process the image at different scales, so as to strengthen the feature presentation of different leakage scales of the dam.

[0071] For the infrared image and the visible light image, an 8-layer Gaussian pyramid is constructed, and the specific steps are as follows: ①, input the original image As the 0th layer of the pyramid.

[0072] ②, for the image of the i th layer , use the Gaussian kernel with a standard deviation of to perform convolution operation on the th layer to obtain the smoothed image.

[0073] ③, downsample the smoothed image, i.e. remove even rows and even columns, to obtain .

[0074] ④, repeat steps ② and ③ until the number of layers of the pyramid reaches 8.

[0075] A Laplacian pyramid is constructed to enhance the expression of different leakage scales, which is used to represent the detail information of the image at different scales. The Laplacian pyramid is constructed by the Gaussian pyramid, and each layer of the Laplacian image is obtained by the difference between the adjacent two layers of the Gaussian pyramid. In this way, the changes of the image at different scales can be highlighted, which is very useful for enhancing the expression of different scales. The construction steps of the Laplacian pyramid are as follows: ①, construct the Gaussian pyramid , , , …, .

[0076] ②, the i th layer of the Laplacian image , where It is an upsampling operation that enlarges the image of the next layer to the size of the current layer.

[0077] ③ For the last layer of Laplacian image .

[0078] A Gaussian / Laplacian pyramid multi-scale feature enhancement method is used to strengthen the representation of features at different leakage scales. Features of different leakage scales are reflected to varying degrees in different layers of the Laplacian pyramid. Smaller leakage features are generally more prominent in higher resolution layers (the upper layers of the Laplacian pyramid), while larger leakage features are more prominent in lower resolution layers (the lower layers of the Laplacian pyramid). Enhancement coefficients are set, and different layers of the Laplacian pyramid are multiplied by these coefficients for enhancement processing. The processed Laplacian pyramid is then reconstructed into an enhanced image. The reconstruction process involves adding the processed Laplacian images layer by layer, and the optimal enhancement effect is obtained by continuously adjusting the values ​​of the enhancement coefficients.

[0079] 4) Feature extraction and fusion (1) Feature extraction The MAAF-Net model (Multi-Modal Adaptive Attention Fusion Network) was used to extract temperature features from infrared images and texture features from visible light images, such as... Figure 2 As shown.

[0080] ① Construction of a visible light branch encoder Using visible light images as input, the visible light branch encoder aims to extract multi-scale texture features from these images. The encoder consists of multiple stacked convolutional and pooling layers. Initial convolutional layers use small kernels (e.g., 3×3) to capture local texture details such as edges and lines. As network depth increases, the number of kernels gradually increases, while pooling layers (e.g., max pooling) are introduced to reduce the size of the feature maps, thereby extracting texture features at different scales. For example, shallow convolutional layers can extract microscopic texture features such as brick textures and fine veins in vegetation; while deeper convolutional layers can obtain macroscopic texture features such as building outlines and large-area vegetation distribution.

[0081] ② Construction of Infrared Branch Encoder The infrared branch encoder focuses on extracting temperature features of the infrared image, taking the infrared image as input. The encoder also adopts a convolutional neural network structure, but the parameters of the convolution kernel are specially designed to adapt to the extraction of temperature features. Since the temperature information of the infrared image has continuity in space, the encoder focuses on capturing the temperature change trend during processing. Through convolution operations, it can identify high-temperature areas, low-temperature areas, and areas with obvious temperature gradient changes in the infrared image, thereby extracting temperature-related features.

[0082] During feature extraction, a CCA module is introduced, as shown in Figure 3 The module includes a channel attention mechanism submodule and a spatial attention mechanism submodule.

[0083] Channel attention mechanism: its role is to focus on key feature information of the image. For infrared images, the channel attention mechanism analyzes the temperature feature distribution of each channel and focuses on low-temperature areas. This is because in the dam leakage detection scenario, the leakage area usually exhibits low-temperature anomalies. By weighting and enhancing the relevant channels of the low-temperature area, it can more effectively extract temperature features related to leakage. For visible light images, the channel attention mechanism focuses on channels related to vegetation texture features, and by calculating the correlation between channels, it enhances the weight of channels where vegetation texture features are located, thereby highlighting vegetation texture information.

[0084] Spatial attention mechanism: mainly used to perceive the spatial position and shape of features. When processing infrared images, the spatial attention mechanism accurately perceives the position and shape of low-temperature areas. It determines the boundaries and contours of low-temperature areas by analyzing the temperature features of each position in the feature map, providing a basis for subsequent leakage area positioning. For visible light images, the spatial attention mechanism can capture the spatial distribution position and shape of vegetation texture, such as identifying the growth area, distribution range, and texture direction of vegetation, etc.

[0085] (2) Feature fusion The PhysConFusion model is used to realize the feature fusion of infrared images and visible light images, build a temperature-texture correlation model, and force the temperature gradient of the leakage area to be consistent with the texture gradient direction. In the feature fusion processing link, the physical constraints of the leakage area temperature field are integrated to further improve the accuracy of leakage area detection.

[0086] Physical properties of temperature field in leakage area: Leakage area will cause temperature anomaly. Due to the difference in specific heat capacity between water and dam material, the temperature of the leakage area will be lower than that of the surrounding dam in the daytime, and the temperature of the leakage area will be higher than that of the surrounding dam at night. The dynamic change of temperature approximately obeys the heat conduction equation. When the dam leaks, the leakage water flow will carry heat and affect the heat conduction process. In the Cartesian coordinate system, the heat conduction equation can be expressed as: wherein, T is the temperature field , unit K or Celsius, is the space-time coordinate, t is the time, α is the thermal diffusivity, k is the thermal conductivity, p is the density, c is the specific heat capacity, q is the internal heat source intensity, is the velocity vector of the leakage water flow, represents the convection term, which describes the heat transfer caused by water flow movement.

[0087] Add PDE residual term to loss function: In order to integrate physical constraints into model training, add PDE residual term to loss function: wherein, is the model predicted temperature field. Since it is difficult to directly solve the derivative in actual calculation, the derivative can be approximated by finite difference method. By adding this term to the loss function, the model can learn the temperature field change that conforms to the physical law, thereby improving the accuracy of leakage area detection.

[0088] 5) Leakage area detection A lightweight model detection architecture is used to process the multi-modal coupled features after feature extraction and fusion, realizing pixel-level accurate detection of dam leakage area. The fused features include multi-scale texture features of visible light images (such as dam surface cracks, abnormal vegetation distribution, etc.) and temperature anomaly features of infrared images (such as low temperature area boundary, temperature gradient change, etc.). Through cross-modal attention mechanism, feature enhancement and spatial alignment are realized, providing high discriminative input for the detection model.

[0089] (1) Detection model architecture design A double-branch detection head is constructed to process the visible light texture branch and infrared temperature branch in the fused features respectively, and finally realize joint detection through feature splicing.

[0090] ① Texture feature detection branch Input: multi-scale texture feature maps (dimension ) extracted by visible branch, containing shallow edge features (3x3 convolution extracted masonry texture details) and deep semantic features (pooling extracted vegetation patch distribution, embankment structure contour).

[0091] Processing flow: further extract texture anomaly features using two layers of convolution blocks (3x3 convolution + ReLU + batch normalization), the first layer focuses on microscopic textures such as leaf curling and soil moisture, and the second layer captures macroscopic anomalies such as large-area vegetation sparsification and embankment surface peeling. Output texture anomaly probability map , representing the texture anomaly degree of each pixel point (texture damage confidence related to leakage).

[0092] ② Temperature feature detection branch Input: multi-scale temperature feature maps (dimension ) extracted by infrared branch, containing low-temperature region contour (temperature anomaly boundary enhanced after convolution kernel optimization) and temperature field gradient features (reflecting temperature change trend of water flow direction).

[0093] Processing flow: filter background temperature fluctuations through two layers of band-pass convolution (center frequency corresponding to the temperature range of the leakage area features), and retain 2-5℃ low-temperature anomaly signals; introduce a spatial attention mechanism (based on temperature variance weighting of feature map spatial position), highlighting the connectivity and geometric morphology of low-temperature regions (such as band-shaped and patch-shaped low-temperature areas), output temperature anomaly probability map .

[0094] ③ Cross-modal feature fusion Weighted fusion of and at pixel level: , where the weight β is dynamically generated by the CCAA module, and the contribution of texture and temperature features is adaptively adjusted according to the scene lighting conditions (such as day / night) (for example, visible light texture information is more reliable during the day, β set to 0.6; infrared temperature signal is more prominent at night, β set to 0.4).

[0095] (2) Pixel-level detection and post-processing Threshold segmentation: use Otsu algorithm to calculate the optimal segmentation threshold τ, generate binary detection mask M: This threshold dynamically balances the recall rate and precision rate of the leakage area, ensuring that both small leakage points (such as pixel-level low-temperature anomalies) and large-area leakage areas (such as patchy texture damage) are effectively detected.

[0096] Morphological optimization: Erosion operation (3×3 rectangular structuring element) removes isolated noise points with an area of ​​less than 5 pixels; dilation operation (5×5 cross-shaped structuring element) connects adjacent leakage areas, repairs boundary breaks caused by multimodal feature alignment errors, and finally forms a continuous and complete leakage area outline.

[0097] (3) Training strategy and loss function Data annotation: Pixel-level annotation is performed on the area below the cropped water surface line to generate a ground truth mask Mgt for the leakage area, and the severity of leakage (strong leakage area, weak leakage area) is labeled as auxiliary training labels.

[0098] Data augmentation: Strategies such as random rotation (±15°), scaling (0.8-1.2 times), and Gaussian noise addition (mean 0, variance 0.01) are employed to improve the model's robustness to changes in drone shooting angle and image resolution differences.

[0099] Loss function: A multi-task loss function is used to jointly optimize detection accuracy. Where: L is the multi-task loss value; L seg L represents the segmentation loss value. consistency γ is the consistency loss value, and γ is the loss weight.

[0100] 6) Spatial location of the leakage area After the leakage area detection is completed, the pixel-level detection results need to be mapped to the real geographic space to provide flood control personnel with accurate on-site coordinates for handling. The spatial positioning scheme is divided into two categories based on the positioning capabilities of the data acquisition equipment, ensuring that accurate geographic coordinate calculation of the leakage area can be achieved under different conditions.

[0101] (1) Direct positioning method based on device GPS module If the drone or camera has a built-in high-precision GPS module (positioning accuracy ≤ 0.5m), spatial positioning can be achieved through the following steps: ① Coordinate information acquisition and calibration When the equipment acquires images, it simultaneously records sensor location information (latitude and longitude (λ, ϕ), elevation h) and attitude parameters (heading angle, pitch angle, roll angle), storing them in the image Exif metadata. By calibrating the camera's intrinsic parameters (focal length, principal point coordinates, distortion coefficients) in the calibration field, and combining this with the equipment's real-time attitude data, a conversion relationship between image pixel coordinates (u, v) and geographic coordinates (X, Y, Z) is established. in f is the principal pixel coordinate, and f is the focal length. Geographical coordinates of the UAV take-off point.

[0102] ② Leakage area coordinate mapping For the detected leakage area mask M, extract the contour pixel coordinates , substitute into the above conversion model, calculate the geographical coordinates corresponding to each pixel . Remove discrete noise points by Gaussian filtering, and take the geometric center coordinates of the continuous leakage area as the final positioning result.

[0103] (2) Indirect positioning method based on oblique photogrammetry If the device does not have a GPS module, use multi-view oblique photogrammetry technology to construct a three-dimensional scene, the specific process is as follows: ① Multi-view image data acquisition The UAV flies according to the "five-directional loop" flight path (forward view, rear view, left view, right view, and normal view), with a single flight line overlap rate of ≥80% and a lateral overlap rate of ≥60%. At least 20 visible light images of different angles are obtained to cover the target dam section.

[0104] ② Three-dimensional reconstruction and coordinate calculation Feature extraction and matching: Use the SIFT algorithm to detect scale-invariant feature points (such as dam corners and vegetation vertices) in the image. Establish a cross-image feature point pair through the FLANN fast matching algorithm. The number of feature points in a single image is ≥500, and the matching pair accuracy is ≥90%.

[0105] Structure from motion (SfM): Based on the matched point pairs, the camera relative pose matrix is solved by the eight-point method, a sparse three-dimensional point cloud is constructed, and the exterior orientation elements (position and attitude) of each image are determined.

[0106] Dense point cloud generation: Use multi-view stereo (MVS) technology to densely expand the sparse point cloud, generate a three-dimensional point cloud model with a resolution of 10cm / pixel, and simultaneously obtain the texture information and geographical coordinates (WGS84 coordinate system) of each point.

[0107] ③ Leakage area spatial mapping Project the detection mask M to the three-dimensional point cloud: For each leakage pixel, find the corresponding three-dimensional point cloud cluster through the camera projection matrix, and calculate the average geographical coordinates of the points in the cluster as the positioning coordinates of the region; Use the RANSAC algorithm to remove outliers, and for leakage areas with an area >1m², further fit a plane through the least squares method, and take the center point of the plane as the final positioning result.

[0108] (3) Positioning result verification and output Regardless of the method, the final positioning result needs to be converted into the commonly used UTM projection coordinate system (coordinate system unification), and verified in the following way: Compare the positioning deviation of the same area at adjacent time, if ΔX / ΔY≤50cm and ΔZ≤30cm, the result is considered reliable; Output KML file containing the geographic coordinates (latitude, longitude, elevation) of the leakage area, area, and contour boundary, which can be directly imported into the GIS system or unmanned ground station to guide the on-site disposal.

[0109] The present application realizes cross-scale mapping from pixel-level detection to meter-level / sub-meter-level spatial positioning by combining built-in positioning and photogrammetry technology, ensuring that the leakage area coordinates can be directly used for accurate investigation and emergency operation in the engineering site.

[0110] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. The contents not described in detail in the specification belong to the prior art known to those skilled in the art.

Claims

1. A method for detecting dam seepage based on unmanned aerial vehicles (UAVs) and deep learning, characterized in that: Image data of the dam surface was acquired using a drone equipped with image acquisition equipment; The acquired dam surface image data is preprocessed; Based on a deep learning model, feature extraction and fusion are performed on the preprocessed image data to obtain a multimodal coupled feature image related to the leakage area; Target detection was performed on the multimodal coupled feature image to identify the leakage area; The identified leakage areas are spatially located to determine their coordinates.

2. The dam seepage detection method based on UAV and deep learning according to claim 1, characterized in that, The acquisition of dam surface image data via a drone equipped with image acquisition equipment includes: An infrared camera is mounted on a drone, which is then controlled to fly along a set route to acquire infrared and visible light image data of the back of the dam, forming image data of the dam surface.

3. The dam seepage detection method based on UAVs and deep learning according to claim 1, characterized in that, Before preprocessing the acquired dam surface image data, the process also includes: Using the current water level as the clipping line, the acquired image range is clipped, and image data with elevations above the clipping line are removed, resulting in image data below the water level.

4. The dam seepage detection method based on UAV and deep learning according to claim 1, characterized in that, The preprocessing of the acquired dam surface image data includes: sequentially performing image denoising, image registration, and image enhancement operations on the acquired dam surface image data.

5. The dam seepage detection method based on UAV and deep learning according to claim 1, characterized in that, The process of extracting and fusing features from preprocessed image data using a deep learning model to obtain a multimodal coupled feature image related to the leakage area includes: The CMAF-Net model is used to extract infrared temperature features and visible light vegetation texture features from image data. During the feature extraction process, the CCA module is introduced for training from the channel and spatial attention dimensions to enhance the ability to distinguish between seepage and vegetation features. A physical constraint fusion network model is used to fuse the features of the extracted infrared and visible light images. During the fusion process, physical constraints on the temperature field of the leakage area are introduced to obtain a multimodal coupled feature image.

6. The dam seepage detection method based on UAV and deep learning according to claim 1, characterized in that, The step of identifying the leakage area by performing target detection on the multimodal coupled feature image includes: A texture branch detection head and a temperature branch detection head are constructed. The texture branch detection head processes the texture feature information of the visible light image to obtain a texture anomaly probability map, and the temperature branch detection head processes the temperature feature information of the infrared image to obtain a temperature anomaly probability map. The texture anomaly probability map and the temperature anomaly probability map are fused across modal features. The fused feature map is then subjected to pixel-level detection and post-processing to obtain a continuous and complete leakage area contour.

7. The dam seepage detection method based on UAV and deep learning according to claim 6, characterized in that, Cross-modal feature fusion is performed using the following formula: ； in, The fused feature map; This is a probability diagram of temperature anomalies. This is a texture anomaly probability map; β For weights.

8. The dam seepage detection method based on UAV and deep learning according to claim 6, characterized in that, The pixel-level detection and post-processing include: The Otsu algorithm is used to calculate the optimal segmentation threshold of the fused feature map, and a binary detection mask is generated based on the optimal segmentation threshold. Erosion and expansion operations are performed on the binary detection mask to form a continuous and complete outline of the leakage area.

9. The dam seepage detection method based on UAV and deep learning according to claim 1, characterized in that, The spatial location of the identified leakage area includes: The drone uses a built-in high-precision GPS module to spatially locate the identified leakage area. Alternatively, multi-view oblique photogrammetry can be used to spatially locate the identified leakage areas.

10. A dam seepage detection system based on unmanned aerial vehicles and deep learning, characterized in that, include: The image acquisition module is a drone equipped with image acquisition equipment, used to acquire image data of the dam surface; The image preprocessing module is used to preprocess the acquired dam surface image data; The feature extraction and fusion module is used to extract and fuse features from preprocessed image data based on a deep learning model to obtain a multimodal coupled feature image related to the leakage area. The target detection and recognition module is used to detect and identify leakage areas from multimodal coupled feature images based on a deep learning network model; The spatial positioning module is used to spatially locate the identified leakage area and determine its coordinates.

Images