Dam crack identification method and system based on unmanned aerial vehicle inspection

Abstract

The invention discloses a dam crack identification method and system based on unmanned aerial vehicle inspection. The method comprises the steps that a multispectral image and point cloud data of the surface of a to-be-detected dam are collected through an unmanned aerial vehicle; carrying out preprocessing and spatial registration on the multispectral image and the point cloud data; fusing the multispectral enhanced image and the point cloud calibration data to obtain a six-channel fusion tensor, and constructing a fused image by the six-channel fusion tensor; extracting fusion features through the six-channel fusion tensor, performing multi-scale context aggregation on the fusion features to obtain crack aggregation features, and performing crack segmentation on the fusion image according to the crack aggregation features to obtain a dam crack recognition model; obtaining a dam crack identification result and a crack size based on the dam crack identification model; according to the invention, data are collected through the unmanned aerial vehicle, multispectral and point cloud information are fused, the model is constructed to accurately identify the dam crack and the measurement size, the detection efficiency and accuracy are improved, and the dam safety is guaranteed.

Description

Technical Field

[0001] The present invention belongs to the technical field of dam monitoring, and in particular relates to a dam crack identification method and system based on drone inspection. Background Art

[0002] Dams are crucial components of water conservancy projects, and cracks are the primary manifestation of structural damage. Timely and accurate identification of cracks is crucial. In recent years, drone inspection technology has become a crucial tool for dam safety monitoring, attracting widespread attention for its efficiency and flexibility.

[0003] However, existing inspection methods still have obvious deficiencies in the accuracy and efficiency of crack identification, especially in complex environments. Only single image data is collected, and the processing and judgment of image data are often interfered with, resulting in frequent misjudgments or missed judgments, which makes it difficult to meet actual engineering needs. Summary of the Invention

[0004] The present invention provides a dam crack identification method and system based on drone inspection. The drone is equipped with a multispectral camera and a lidar, which can quickly reach the dam surface for data collection, greatly improving the inspection efficiency and accuracy.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A first aspect of the present invention provides a dam crack identification method based on drone inspection, comprising:

[0007] Collect multispectral images and point cloud data of the surface of the dam to be measured by using a drone; identifiable reference points are preset on the surface of the dam;

[0008] The microcrack edge enhancement map, water stain mask map, and subsurface anomaly thermal map are obtained by decomposing the multispectral image; the microcrack edge enhancement map, water stain mask map, and subsurface anomaly thermal map are stored in the R channel, G channel, and B channel respectively, and the multispectral enhanced image is output;

[0009] After performing dynamic threshold denoising on the point cloud data, the point cloud data is subjected to surface distortion correction to obtain point cloud calibration data; and the multispectral enhanced image and the point cloud calibration data are spatially registered;

[0010] The multispectral enhanced image and the point cloud calibration data are fused to obtain a six-channel fusion tensor, and a fused image is constructed from the six-channel fusion tensor. Dual-branch features are extracted from the six-channel fusion tensor, and the dual-branch features are fused with cross-modal attention to obtain fusion features. Multi-scale context aggregation is performed on the fusion features to obtain crack aggregation features. The fused image is segmented according to the crack aggregation features to obtain a dam crack recognition model. Based on the dam crack recognition model, dam crack recognition results and crack sizes are obtained.

[0011] Furthermore, the multispectral image and point cloud data of the dam surface to be tested are collected by drone. The specific process includes:

[0012] The coordinates of the reference points are read and a triangulated network coverage algorithm is used to automatically generate an initial flight path for a drone equipped with a multispectral camera and a laser radar. The flight path of the drone is limited to be parallel to the axis or radial direction of the dam. The flight altitude of the drone is dynamically calculated based on the inclination of the dam surface, so that the projection directions of the multispectral camera and the laser radar are perpendicular to the dam surface.

[0013] In response to receiving radar detection data collected by the laser radar, the obstacle is identified based on the radar detection data and a point cloud three-dimensional bounding box of the obstacle is constructed, a UAV detour path is generated based on the point cloud three-dimensional bounding box, and the UAV detour path is used to adjust the UAV's initial flight path;

[0014] In response to receiving the multispectral image captured by the multispectral camera, the abnormal temperature area on the dam surface is identified according to the multispectral image, the distance between the UAV and the dam surface is reduced to a set lower height limit, a UAV identification route is generated according to the abnormal temperature area on the dam surface, and the initial route of the UAV is adjusted using the UAV identification route.

[0015] Furthermore, the microcrack edge enhancement map, water stain mask map and sub-surface anomaly thermal map are obtained by decomposing the multispectral image, including:

[0016] The multispectral image is decomposed into band A spectral sub-image, band B spectral sub-image and band C spectral sub-image; the band A spectral sub-image is subjected to adaptive histogram equalization to obtain a microcrack edge enhancement map; the ratio R of the band A spectral sub-image and the band B spectral sub-image is calculated to obtain a ratio image, and the ratio image is segmented using an adaptive threshold segmentation algorithm to obtain a water stain mask map; the band C spectral sub-image is identified using a temperature inversion algorithm to obtain the characteristics of desiccation cracks and seepage cracks; and a sub-surface anomaly thermal map is constructed based on the characteristics of desiccation cracks and seepage cracks.

[0017] Furthermore, the point cloud data is subjected to dynamic threshold denoising, specifically including:

[0018] Performing statistical analysis on the reflection intensity of the point cloud data to calculate its mean and standard deviation; setting an initial range of the dynamic threshold based on the image mean and image standard deviation of the point cloud data; the point cloud data is composed of a number of sampling pixels;

[0019] The point cloud data is divided into cubic voxels using a voxel grid method, and the local mean and local standard deviation of the sampled pixels within the cubic voxel are recalculated. The initial range of the dynamic threshold is adjusted based on the local mean and local standard deviation to obtain a dynamic threshold correction range. The point cloud within the cubic voxel is denoised based on the dynamic threshold correction range.

[0020] The set of central candidate points within the cube voxel is determined based on the local extreme value characteristics of the reflection intensity, and the position weight of the sampling pixel point is calculated based on the distance from the central candidate point to the sampling pixel point within the adjacent range; the expression formula is:

[0021]

[0022] In the formula, The first The position of the sampling pixel points; is the position of the center candidate point; The first The weight of each sampling pixel; is the position weight standard deviation of the sampling pixel points;

[0023] The weighted average reflection intensity of the sampled pixels in the neighborhood is calculated according to the position weight as the reflection intensity feature of the center candidate point; the expression formula is:

[0024]

[0025] In the formula, is the reflection intensity feature of the center candidate point, The first The reflection intensity characteristics of the sampling pixels; M is the number of sampling pixels in the adjacent range;

[0026] The center point within the cube voxel is determined by screening the center candidate point set according to the reflection intensity characteristics.

[0027] Furthermore, performing surface distortion correction on the point cloud data to obtain point cloud calibration data specifically includes:

[0028] The confidence level of the reflection intensity of the center point is calculated based on the reflection intensity characteristics of the sampling pixels within the center point and the adjacent range. The expression formula is:

[0029]

[0030] In the formula, is the confidence level of the reflection intensity at the center point; is the similarity threshold; The first Reflection intensity characteristics of each sampling pixel; is the indicator function; is the reflection intensity characteristic of the center point; The number of sampled pixels within the adjacent range of the center point;

[0031] The PCA method is used to calculate the local curvature of the center point and set the curvature constraint term. The expression formula is:

[0032]

[0033] In the formula, is the curvature constraint term; is the local curvature of the center point; is the center value of the normal curvature range; is the span of the normal curvature range;

[0034] The curvature distortion loss is constructed by combining the reflection intensity confidence of the center point and the curvature constraint term. The expression formula is:

[0035]

[0036] In the formula, is the curvature distortion loss; N is the number of center points;

[0037] With the goal of minimizing the curvature distortion loss, the coordinates of the sampling pixels in the point cloud data are adjusted, and the surface distortion correction process is performed on the point cloud data repeatedly until the curvature distortion loss converges and the point cloud calibration data is output.

[0038] Furthermore, the multispectral enhanced image and point cloud calibration data are spatially registered. The specific process is as follows:

[0039] The first physical coordinates of the reference point are extracted from the point cloud calibration data, and the second physical coordinates of the reference point are extracted from the multispectral enhanced image. The initial transformation matrix between the first physical coordinates and the measured coordinates of the reference point is calculated using the RANSAC algorithm, and a dam coordinate system is constructed. The second physical coordinates of the reference point are mapped to the dam coordinate system to complete the spatial alignment of the multispectral enhanced image and the point cloud calibration data.

[0040] Furthermore, the multispectral enhanced image and the point cloud calibration data are fused to obtain a six-channel fusion tensor, and a fusion image is constructed from the six-channel fusion tensor, specifically including:

[0041] The crack texture features are obtained from the multispectral enhanced image, and the crack texture features are assigned to the point cloud calibration data points according to the mapping relationship between the multispectral enhanced image and the point cloud calibration data;

[0042] Construct a cube that encloses the point cloud calibration data as an octree node. When the octree node contains crack texture features, continue to divide the octree node into levels to obtain the nodes of the next level of octree.

[0043] Repeat the iterative octree partitioning process until the octree node contains no point cloud calibration data points that give crack texture features or only contains a single point cloud calibration data point that gives crack texture features, and output the final octree structure;

[0044] A six-channel fusion tensor is established for the nodes of the final octree structure. The six-channel fusion tensor includes the image of the texture features of the point cloud calibration data points in the R color channel, the image of the texture features of the point cloud calibration data points in the G color channel, the image of the texture features of the point cloud calibration data points in the B color channel, the pixel values ​​of the point cloud calibration data points mapped to the pixel points in the microcrack edge enhancement image, the pixel values ​​of the point cloud calibration data points mapped to the pixel points in the water stain mask image, and the point cloud attributes of the point cloud calibration data points; a fused image is constructed from the six-channel fusion tensor.

[0045] A second aspect of the present invention provides a hydropower station dam safety online monitoring system based on multi-source data, comprising:

[0046] A data acquisition module is used to collect multispectral images and point cloud data of the surface of the dam to be measured by using a drone; the dam surface is pre-set with identifiable reference points;

[0047] The image enhancement module decomposes the multispectral image to obtain a microcrack edge enhancement map, a water stain mask map, and a subsurface anomaly thermal map; the microcrack edge enhancement map, the water stain mask map, and the subsurface anomaly thermal map are stored in the R channel, G channel, and B channel, respectively, and the multispectral enhanced image is output;

[0048] An image correction module is used to perform dynamic threshold denoising on the point cloud data and perform surface distortion correction on the point cloud data to obtain point cloud calibration data; and to perform spatial registration on the multispectral enhanced image and the point cloud calibration data;

[0049] The crack recognition module is used to fuse the multispectral enhanced image and point cloud calibration data to construct a six-channel fusion tensor, extract dual-branch features from the six-channel fusion tensor, perform cross-modal attention fusion on the dual-branch features to obtain fusion features, and perform multi-scale context aggregation on the fusion features to obtain crack aggregation features.

[0050] The output module is used to perform crack segmentation on the fused image according to the crack aggregation characteristics to obtain a dam crack recognition model; and obtain dam crack recognition results and crack sizes based on the dam crack recognition model.

[0051] A third aspect of the present invention provides an electronic device comprising a storage medium and a processor; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the dam crack identification method described in the first aspect.

[0052] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed, the dam crack identification method described in the first aspect is implemented.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention decomposes multispectral images to obtain a microcrack edge enhancement map, a water spot mask map, and a sub-surface anomaly thermal map; the microcrack edge enhancement map highlights the edges of tiny cracks, the water spot mask map distinguishes between dry and water-soaked areas, and the sub-surface anomaly thermal map reveals differences in thermal radiation characteristics; the microcrack edge enhancement map, the water spot mask map, and the sub-surface anomaly thermal map are stored according to the R channel, G channel, and B channel respectively, and a multispectral enhanced image is output; multi-dimensional information is integrated to improve the distinction between cracks and other surface features.

[0055] The dynamic threshold denoising method of the present invention can remove noise points and enhance the reflection intensity of the reference points according to the reflection intensity characteristics of the point cloud data; the surface distortion correction ensures that the geometric shape and spatial position information of the point cloud data are more accurate by optimizing the distortion of the dam surface, and spatially aligns and fuses the multispectral image and the point cloud data so that the two data can complement each other. The multispectral image provides rich spectral information, which helps to distinguish cracks from other surface features; the point cloud data provides precise geometric shape and spatial position information, which is conducive to accurately identifying cracks on the dam surface.

[0056] This method fuses multispectral enhanced images with point cloud calibration data to form a six-channel fused tensor and construct a fused image, integrating spectral and geometric information to improve crack identification accuracy. It also utilizes dual-branch feature extraction and cross-modal attention fusion to focus on crack features and enhance model recognition capabilities. Multi-scale context aggregation generates aggregated crack features, enabling crack segmentation and sizing, providing quantitative data for dam safety assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 is a flow chart of the dam crack identification method provided in this embodiment 1;

[0058] Figure 2is a flow chart of multispectral image decomposition provided in this embodiment 1;

[0059] Figure 3 This is a flowchart of the spatial registration of multispectral images and point cloud data provided in this embodiment 1. DETAILED DESCRIPTION

[0060] The present invention will be further described below in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0061] Example 1

[0062] like Figure 1 As shown, this implementation provides a dam crack identification method based on drone inspection, including:

[0063] The multispectral image and point cloud data of the surface of the dam to be measured are collected by drone; the dam surface has identifiable reference points preset. The specific process includes:

[0064] The coordinates of the reference points are read and a triangulated network coverage algorithm is used to automatically generate the initial flight path of the UAV. Specifically, a triangular flight zone is divided into units of three adjacent reference points. The UAV is equipped with a multispectral camera and a laser radar. The UAV flight direction is limited to be parallel to the dam axis or radial direction. The UAV flight altitude is dynamically calculated based on the inclination angle of the dam surface, so that the projection direction of the multispectral camera and the laser radar is perpendicular to the dam surface.

[0065] In response to the radar detection data collected by the lidar, the lidar scans the 50m area in front in real time, identifies obstacles based on the radar detection data and constructs a point cloud 3D bounding box of the obstacle. According to the point cloud 3D bounding box, the drone's detour path is generated, and the drone's initial route is adjusted using the detour path. During the process, the minimum safety distance of 2m is maintained while ensuring that the image overlap rate is >75%.

[0066] In response to the multispectral images collected by the multispectral camera, the abnormal temperature area on the dam surface is identified based on the multispectral images, and the probability of crack distribution in the abnormal temperature area on the dam surface is estimated through a lightweight neural network; the distance between the UAV and the dam surface is reduced to the set lower limit of the height, and the UAV identification route is generated according to the abnormal temperature area on the dam surface, and the initial route of the UAV is adjusted using the UAV identification route; among them, the route density is automatically increased in the high probability area (>60%) (the flight strip overlap rate is increased to 90%); the low probability area (<10%) adopts a sparse scanning mode (saving 30% of the flight time).

[0067] When the drone's remaining battery power falls below the lower energy storage threshold, or if the percentage of high-risk areas exceeds 40%, it will land nearby to replace the battery and resume flight. When the percentage of low-risk areas exceeds 80%, it will activate rapid scanning mode (raising its altitude to 15 meters). When the spectrum sensing module detects strong electromagnetic sources such as dam floodgates, it will switch to a backup communication frequency band 200 meters in advance. Key data will be transmitted back to the ground station in real time. If signal loss occurs, the drone will return along a pre-set safe path.

[0068] like Figure 2 As shown in the figure, the microcrack edge enhancement map, water stain mask map and sub-surface anomaly thermal map are obtained by decomposing the multispectral image, including:

[0069] The multispectral image is decomposed into bands of 450nm, 550nm, 650nm, 800nm ​​and 950nm; the bands corresponding to the band A spectral sub-image, the band B spectral sub-image and the band C spectral sub-image are 650nm, 800nm ​​and 950nm respectively.

[0070] Adaptive histogram equalization is performed on the Band A spectral sub-image to obtain a microcrack edge enhancement image. This embodiment first divides the Band A spectral sub-image into 32×32 pixel sub-blocks, and then calculates the histogram characteristics of each block, including the mean and standard deviation. By analyzing this statistical information, the contrast stretching range is limited, thereby eliminating overly bright and dark areas and preventing these areas from interfering with subsequent processing. Furthermore, a directionally adjustable filter is used to enhance the response along the common crack direction (i.e., from 45° to 135°), making the crack edge features more prominent. This process not only improves the image contrast but also emphasizes the crack edge information, providing clearer and more recognizable image features for subsequent crack identification and analysis.

[0071] The ratio R of the spectral sub-image of band A and the spectral sub-image of band B is calculated to obtain the ratio image, which is expressed as follows:

[0072]

[0073] In the formula, is the spectral sub-image of band B, is the spectral sub-image of band A; is the set minimum value.

[0074] The contrast image is segmented using an adaptive threshold segmentation algorithm to obtain a watermark mask.

[0075] An adaptive filtering algorithm is used for preprocessing the Band C spectral sub-image. This embodiment automatically adjusts the filtering intensity based on the noise characteristics of different image regions. This effectively removes random noise caused by factors such as sensor noise and atmospheric scattering, while preserving the detailed radiation characteristics of desiccation cracks and water-seepage cracks to the greatest extent possible. By analyzing the local statistical characteristics of the spectral image, the size and shape of the filtering window are determined, resulting in a clearer visual image after filtering.

[0076] Dynamic correction factors for surface emissivity and atmospheric transmittance were introduced, and the temperature inversion algorithm was used to identify the spectral sub-images of Band C to obtain the characteristics of desiccation cracks and water seepage cracks. The specific process includes:

[0077] In this embodiment, a temperature inversion algorithm is used to extract local depressions in the Band C spectral subimage. Then, gradient and curvature information are combined to determine the potential locations of desiccation cracks. Near each candidate local temperature minimum, the magnitude and direction of the temperature gradient, as well as the curvature, are calculated. By setting gradient and curvature thresholds, regions that meet the radiation characteristics of desiccation cracks are selected. Candidate desiccation crack regions that are close to each other and have similar temperature characteristics are merged to reduce false detections caused by noise or small-scale interference, ultimately accurately identifying the distribution range and morphology of desiccation cracks.

[0078] A clustering similarity measurement function is constructed based on the high-temperature anomaly characteristics of water seepage cracks. The DBSCAN clustering algorithm is used to automatically determine the parameters of the clustering similarity measurement function, including the neighborhood radius and the minimum number of samples. Pixel blocks with similar temperature characteristics and spatially continuous are clustered into the same water seepage crack region. The texture features of each water seepage crack region are calculated, and clustered regions that do not conform to the texture features of water seepage cracks are eliminated, thereby achieving accurate identification and positioning of water seepage cracks.

[0079] A subsurface anomaly thermal map is constructed based on the characteristics of dry cracks and water seepage cracks. The R channel, G channel, and B channel are used to store the microcrack edge enhancement map, water stain mask map, and subsurface anomaly thermal map, respectively, to output a multispectral enhanced image.

[0080] The point cloud data is subjected to dynamic threshold denoising, specifically comprising:

[0081] Performing statistical analysis on the reflection intensity of the point cloud data to calculate its mean and standard deviation; setting an initial range of the dynamic threshold based on the image mean and image standard deviation of the point cloud data; the point cloud data is composed of a number of sampling pixels;

[0082] The point cloud data is divided into cubic voxels using a voxel grid method, and the local mean and local standard deviation of the sampled pixels within the cubic voxel are recalculated. The initial range of the dynamic threshold is adjusted based on the local mean and local standard deviation to obtain a dynamic threshold correction range. The point cloud within the cubic voxel is denoised based on the dynamic threshold correction range.

[0083] The set of central candidate points within the cube voxel is determined based on the local extreme value characteristics of the reflection intensity, and the position weight of the sampling pixel point is calculated based on the distance from the central candidate point to the sampling pixel point within the adjacent range; the expression formula is:

[0084]

[0085] In the formula, The first The position of the sampling pixel points; is the position of the center candidate point; The first The weight of each sampling pixel; is the position weight standard deviation of the sampling pixel points;

[0086] The weighted average reflection intensity of the sampled pixels in the neighborhood is calculated according to the position weight as the reflection intensity feature of the center candidate point; the expression formula is:

[0087]

[0088] In the formula, is the reflection intensity feature of the center candidate point, The first The reflection intensity characteristics of the sampling pixels; M is the number of sampling pixels in the adjacent range;

[0089] Based on the reflection intensity characteristics, the center point within the cube voxel is determined by screening the set of center candidate points. In this embodiment, these noise points are often caused by sensor errors, environmental interference, or non-dam target objects. Their reflection intensity usually deviates significantly from the reflection intensity distribution of the normal dam surface point cloud. Therefore, the dynamic threshold can adapt to the changing characteristics of reflection intensity in different areas, thereby better removing noise points.

[0090] Performing surface distortion correction on the point cloud data to obtain point cloud calibration data specifically includes:

[0091] The confidence level of the reflection intensity of the center point is calculated based on the reflection intensity characteristics of the sampling pixels within the center point and the adjacent range. The expression formula is:

[0092]

[0093] In the formula, is the confidence level of the reflection intensity at the center point; is the similarity threshold; The first Reflection intensity characteristics of each sampling pixel; is the indicator function; is the reflection intensity characteristic of the center point; The number of sampled pixels within the adjacent range of the center point;

[0094] The PCA method is used to calculate the local curvature of the center point and set the curvature constraint term. The expression formula is:

[0095]

[0096] In the formula, is the curvature constraint term; is the local curvature of the center point; is the center value of the normal curvature range; is the span of the normal curvature range;

[0097] The curvature distortion loss is constructed by combining the reflection intensity confidence of the center point and the curvature constraint term. The expression formula is:

[0098]

[0099] In the formula, is the curvature distortion loss; N is the number of center points;

[0100] Aiming to minimize curvature distortion loss, the coordinates of the sampled pixels in the point cloud data are adjusted, and the surface distortion correction process is repeated until the curvature distortion loss converges and the calibrated point cloud data is output. The curvature distortion loss function minimizes the distortion of points with high reflection intensity confidence weights under curvature constraints, thereby guiding the optimization of the entire point cloud data and correcting the distortion of the dam surface.

[0101] like Figure 3 As shown in the figure, the multispectral enhanced image and point cloud calibration data are spatially registered. The specific process is as follows:

[0102] The first physical coordinates of the reference point are extracted from the point cloud calibration data. In this embodiment, a point cloud segmentation algorithm based on reflectivity features is used to automatically extract the position information of the reference point from the complex point cloud data, which is the first physical coordinate of the reference point. The point cloud segmentation algorithm not only considers the threshold screening of the reflectivity, but also integrates the spatial distribution characteristics of the neighborhood points to ensure that the extracted reference point coordinates are highly accurate and reliable.

[0103] The second physical coordinates of the reference point are extracted from the multispectral enhanced image. In this embodiment, the multispectral enhanced image includes a QR code. QR codes have advantages such as large information storage capacity and strong anti-fouling capabilities. Even in harsh outdoor environments, they can maintain the integrity of their encoded information for a long time, making them easy to identify and decode in the multispectral image, thereby achieving precise positioning of the reference point. The YOLOv11 algorithm can quickly and accurately identify the QR code area in the image. By decoding the QR code, the unique identification information stored therein is obtained, and then matched with the records in the physical coordinate database to determine the corresponding physical coordinates of the reference point, which are the second physical coordinates of the reference point.

[0104] The RANSAC algorithm is used to calculate the initial transformation matrix between the first physical coordinates of the benchmark point and the measured coordinates and to construct the dam coordinate system. The second physical coordinates of the benchmark point are mapped to the dam coordinate system to complete the spatial registration of the multispectral enhanced image and the point cloud calibration data.

[0105] The multispectral enhanced image and point cloud calibration data are fused to obtain a six-channel fusion tensor, and a fused image is constructed from the six-channel fusion tensor, specifically including:

[0106] The crack texture features are obtained from the multispectral enhanced image, and the crack texture features are assigned to the point cloud calibration data points according to the mapping relationship between the multispectral enhanced image and the point cloud calibration data;

[0107] Construct a cube that encloses the point cloud calibration data as an octree node. When the octree node contains crack texture features, continue to divide the octree node into levels to obtain the nodes of the next level of octree.

[0108] Repeat the iterative octree partitioning process until the octree node contains no point cloud calibration data points that give crack texture features or only contains a single point cloud calibration data point that gives crack texture features, and output the final octree structure;

[0109] A six-channel fusion tensor is established for the nodes of the final octree structure. The six-channel fusion tensor includes the image of the texture features of the point cloud calibration data points in the R color channel, the image of the texture features of the point cloud calibration data points in the G color channel, the image of the texture features of the point cloud calibration data points in the B color channel, the pixel values ​​of the point cloud calibration data points mapped to the pixel points in the microcrack edge enhancement image, the pixel values ​​of the point cloud calibration data points mapped to the pixel points in the water stain mask image, and the point cloud attributes of the point cloud calibration data points; a fused image is constructed from the six-channel fusion tensor.

[0110] The dual-branch features are extracted from the six-channel fusion tensor. The pre-trained ResNet-34 network is used as the backbone. A channel attention module is added after the residual block in the ResNet-34 network to expand the receptive field while maintaining the resolution, thus solving the problem of missed detection of tiny cracks.

[0111] Using PointNet++ hierarchical feature extraction, we prioritized points in high-curvature areas as key points, increasing crack coverage by 40%. We also combined point cloud reflection intensity and elevation mutation values ​​into a dual-channel input to enhance geometric feature expression.

[0112] The dual-branch features are fused through cross-modal attention to obtain fused features. A bidirectional gated fusion mechanism is employed, including spectral-to-geometric and geometric-to-spectral gates. The spectral-to-geometric gate uses spectral features to generate a geometric feature weight map, suppressing geometric noise in flat areas of concrete. The geometric-to-spectral gate uses curvature features to generate a spectral feature mask, enhancing the spectral response of crack edges. A 3D coordinate projection layer is introduced to strictly align the point cloud feature map with pixel coordinates, developing an offset deformable convolution to adaptively compensate for alignment residuals to obtain fused features.

[0113] Multi-scale contextual aggregation is performed on the fused features to obtain crack aggregation features. Based on these crack aggregation features, the fused image is segmented to obtain a dam crack recognition model. Specifically, for multi-scale contextual aggregation, a feature pyramid network with cross-scale short-circuit connections is employed. Direct channels at 1 / 4 and 1 / 32 scales are added, and depthwise separable convolution is used to reduce computational complexity, addressing the issue of missing deep-level features for small cracks. Adaptive pooling modules are added to each pyramid layer to dynamically adjust the pooling kernel direction (stretching along the crack direction and compressing vertically), preserving crack continuity while suppressing concrete texture noise.

[0114] Based on the crack segmentation results, crack size parameter regression is performed to complete the construction of the dam crack identification model; among them, crack size parameters include crack width, length and inclination.

[0115] Based on the dam crack identification model, the dam crack identification results and crack size are obtained, including:

[0116] Set the starting and ending points on the crack skeleton, calculate the shortest path along the dam surface, introduce a slope correction factor (length compensation of 12% when the inclination angle is >45°), and complete the crack length regression;

[0117] The crack strike vector was fitted to the local plane of the point cloud and obtained through principal component analysis of the skeleton line. Local inclination mutation segments less than 5 cm in length were eliminated, and a sliding window weighted average was used to output the final dam crack identification results and crack size.

[0118] Example 2

[0119] This embodiment discloses a dam crack identification system based on drone inspection. The dam crack identification system is used to execute the dam crack identification method described in Example 1. The dam crack identification system includes:

[0120] A data acquisition module is used to collect multispectral images and point cloud data of the surface of the dam to be measured by using a drone; the dam surface is pre-set with identifiable reference points;

[0121] The image enhancement module decomposes the multispectral image to obtain a microcrack edge enhancement map, a water stain mask map, and a subsurface anomaly thermal map; the microcrack edge enhancement map, the water stain mask map, and the subsurface anomaly thermal map are stored in the R channel, G channel, and B channel, respectively, and the multispectral enhanced image is output;

[0122] An image correction module is used to perform dynamic threshold denoising on the point cloud data and perform surface distortion correction on the point cloud data to obtain point cloud calibration data; and to perform spatial registration on the multispectral enhanced image and the point cloud calibration data;

[0123] The crack recognition module is used to fuse the multispectral enhanced image and point cloud calibration data to construct a six-channel fusion tensor, extract dual-branch features from the six-channel fusion tensor, perform cross-modal attention fusion on the dual-branch features to obtain fusion features, and perform multi-scale context aggregation on the fusion features to obtain crack aggregation features.

[0124] The output module is used to perform crack segmentation on the fused image according to the crack aggregation characteristics to obtain a dam crack recognition model; and obtain dam crack recognition results and crack sizes based on the dam crack recognition model.

[0125] The image enhancement module decomposes the multispectral image to obtain a microcrack edge enhancement map, a water stain mask map, and a subsurface anomaly thermal map, specifically including:

[0126] The multispectral image is decomposed into band A spectral sub-image, band B spectral sub-image and band C spectral sub-image; the band A spectral sub-image is subjected to adaptive histogram equalization to obtain a microcrack edge enhancement map; the ratio R of the band A spectral sub-image and the band B spectral sub-image is calculated to obtain a ratio image, and the ratio image is segmented using an adaptive threshold segmentation algorithm to obtain a water stain mask map; the band C spectral sub-image is identified using a temperature inversion algorithm to obtain the characteristics of desiccation cracks and seepage cracks; and a sub-surface anomaly thermal map is constructed based on the characteristics of desiccation cracks and seepage cracks.

[0127] The image correction module performs spatial registration on the multispectral enhanced image and the point cloud calibration data. The specific process is as follows:

[0128] The first physical coordinates of the reference point are extracted from the point cloud calibration data, and the second physical coordinates of the reference point are extracted from the multispectral enhanced image. The initial transformation matrix between the first physical coordinates and the measured coordinates of the reference point is calculated using the RANSAC algorithm, and a dam coordinate system is constructed. The second physical coordinates of the reference point are mapped to the dam coordinate system to complete the spatial alignment of the multispectral enhanced image and the point cloud calibration data.

[0129] Example 3

[0130] This embodiment provides an electronic device, including a storage medium and a processor; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the dam crack identification method described in Example 1.

[0131] Example 4

[0132] This embodiment provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, and when the computer program is executed, the dam crack identification method described in Example 1 is implemented.

[0133] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0135] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0136] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0137] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A dam crack identification method based on drone inspection, characterized in that: include: Use drones to collect multispectral images and point cloud data of the dam surface to be tested; Identifiable reference points are preset on the surface of the dam; The microcrack edge enhancement map, water stain mask map, and subsurface anomaly thermal map are obtained by decomposing the multispectral image; the microcrack edge enhancement map, water stain mask map, and subsurface anomaly thermal map are stored in the R channel, G channel, and B channel respectively, and the multispectral enhanced image is output; After performing dynamic threshold denoising on the point cloud data, the point cloud data is subjected to surface distortion correction to obtain point cloud calibration data; and the multispectral enhanced image and the point cloud calibration data are spatially registered; The multispectral enhanced image and the point cloud calibration data are fused to obtain a six-channel fusion tensor, and a fusion image is constructed from the six-channel fusion tensor; Dual-branch features are extracted from the six-channel fusion tensor, and the dual-branch features are fused with cross-modal attention to obtain fusion features. Multi-scale context aggregation is performed on the fusion features to obtain crack aggregation features. Crack segmentation is performed on the fused image according to the crack aggregation features to obtain a dam crack recognition model. Dam crack recognition results and crack sizes are obtained based on the dam crack recognition model.

2. The dam crack identification method according to claim 1, characterized in that: The multispectral image and point cloud data of the dam surface to be measured are collected by drone. The specific process includes: The coordinates of the reference points are read and a triangulated network coverage algorithm is used to automatically generate an initial flight path for a drone equipped with a multispectral camera and a laser radar. The flight path of the drone is limited to be parallel to the axis or radial direction of the dam. The flight altitude of the drone is dynamically calculated based on the inclination of the dam surface, so that the projection directions of the multispectral camera and the laser radar are perpendicular to the dam surface. In response to receiving radar detection data collected by the laser radar, the obstacle is identified based on the radar detection data and a point cloud three-dimensional bounding box of the obstacle is constructed, a UAV detour path is generated based on the point cloud three-dimensional bounding box, and the UAV detour path is used to adjust the UAV's initial flight path; In response to receiving the multispectral image captured by the multispectral camera, the abnormal temperature area on the dam surface is identified according to the multispectral image, the distance between the UAV and the dam surface is reduced to a set lower height limit, a UAV identification route is generated according to the abnormal temperature area on the dam surface, and the initial route of the UAV is adjusted using the UAV identification route.

3. The dam crack identification method according to claim 1, characterized in that: The microcrack edge enhancement map, water stain mask map and sub-surface anomaly thermal map are obtained by decomposing the multispectral image, including: The multispectral image is decomposed into band A spectral sub-image, band B spectral sub-image and band C spectral sub-image; the band A spectral sub-image is subjected to adaptive histogram equalization to obtain a microcrack edge enhancement map; the ratio R of the band A spectral sub-image and the band B spectral sub-image is calculated to obtain a ratio image, and the ratio image is segmented using an adaptive threshold segmentation algorithm to obtain a water stain mask map; the band C spectral sub-image is identified using a temperature inversion algorithm to obtain the characteristics of desiccation cracks and seepage cracks; and a sub-surface anomaly thermal map is constructed based on the characteristics of desiccation cracks and seepage cracks.

4. The dam crack identification method according to claim 1, characterized in that: The point cloud data is subjected to dynamic threshold denoising, specifically comprising: Performing statistical analysis on the reflection intensity of the point cloud data to calculate its mean and standard deviation; setting an initial range of the dynamic threshold based on the image mean and image standard deviation of the point cloud data; the point cloud data is composed of a number of sampling pixels; The point cloud data is divided into cubic voxels using a voxel grid method, and the local mean and local standard deviation of the sampled pixels within the cubic voxel are recalculated. The initial range of the dynamic threshold is adjusted based on the local mean and local standard deviation to obtain a dynamic threshold correction range. The point cloud within the cubic voxel is denoised based on the dynamic threshold correction range. The set of central candidate points within the cube voxel is determined based on the local extreme value characteristics of the reflection intensity, and the position weight of the sampling pixel point is calculated based on the distance from the central candidate point to the sampling pixel point within the adjacent range; the expression formula is: ； In the formula, The first The position of the sampling pixel points; is the position of the center candidate point; The first The weight of each sampling pixel; is the position weight standard deviation of the sampling pixel points; The weighted average reflection intensity of the sampled pixels in the neighborhood is calculated according to the position weight as the reflection intensity feature of the center candidate point; the expression formula is: ； In the formula, is the reflection intensity feature of the center candidate point, The first The reflection intensity characteristics of the sampling pixels; M is the number of sampling pixels in the adjacent range; The center point within the cube voxel is determined by screening the center candidate point set according to the reflection intensity characteristics.

5. The dam crack identification method according to claim 1, characterized in that: Performing surface distortion correction on the point cloud data to obtain point cloud calibration data specifically includes: The confidence level of the reflection intensity of the center point is calculated based on the reflection intensity characteristics of the sampling pixels within the center point and the adjacent range. The expression formula is: ； In the formula, is the confidence level of the reflection intensity at the center point; is the similarity threshold; The first Reflection intensity characteristics of each sampling pixel; is the indicator function; is the reflection intensity characteristic of the center point; The number of sampled pixels within the adjacent range of the center point; The PCA method is used to calculate the local curvature of the center point and set the curvature constraint term. The expression formula is: ； In the formula, is the curvature constraint term; is the local curvature of the center point; is the center value of the normal curvature range; is the span of the normal curvature range; The curvature distortion loss is constructed by combining the reflection intensity confidence of the center point and the curvature constraint term. The expression formula is: ； In the formula, is the curvature distortion loss; N is the number of center points; With the goal of minimizing the curvature distortion loss, the coordinates of the sampling pixels in the point cloud data are adjusted, and the surface distortion correction process is performed on the point cloud data repeatedly until the curvature distortion loss converges and the point cloud calibration data is output.

6. The dam crack identification method according to claim 1, characterized in that: Perform spatial registration on the multispectral enhanced image and point cloud calibration data. The specific process is as follows: The first physical coordinates of the reference point are extracted from the point cloud calibration data, and the second physical coordinates of the reference point are extracted from the multispectral enhanced image. The initial transformation matrix between the first physical coordinates and the measured coordinates of the reference point is calculated using the RANSAC algorithm, and a dam coordinate system is constructed. The second physical coordinates of the reference point are mapped to the dam coordinate system to complete the spatial alignment of the multispectral enhanced image and the point cloud calibration data.

7. The dam crack identification method according to claim 1, characterized in that: The multispectral enhanced image and point cloud calibration data are fused to obtain a six-channel fusion tensor, and a fused image is constructed from the six-channel fusion tensor, specifically including: The crack texture features are obtained from the multispectral enhanced image, and the crack texture features are assigned to the point cloud calibration data points according to the mapping relationship between the multispectral enhanced image and the point cloud calibration data; Construct a cube that encloses the point cloud calibration data as an octree node. When the octree node contains crack texture features, continue to divide the octree node into levels to obtain the nodes of the next level of octree. Repeat the iterative octree partitioning process until the octree node contains no point cloud calibration data points that give crack texture features or only contains a single point cloud calibration data point that gives crack texture features, and output the final octree structure; A six-channel fusion tensor is established for the nodes of the final octree structure. The six-channel fusion tensor includes the image of the texture features of the point cloud calibration data points in the R color channel, the image of the texture features of the point cloud calibration data points in the G color channel, the image of the texture features of the point cloud calibration data points in the B color channel, the pixel values ​​of the point cloud calibration data points mapped to the pixel points in the microcrack edge enhancement image, the pixel values ​​of the point cloud calibration data points mapped to the pixel points in the water stain mask image, and the point cloud attributes of the point cloud calibration data points; a fused image is constructed from the six-channel fusion tensor.

8. A dam crack identification system based on drone inspection, characterized by: include: The data acquisition module is used to collect multispectral images and point cloud data of the surface of the dam to be tested through drones; Identifiable reference points are preset on the surface of the dam; The image enhancement module decomposes the multispectral image to obtain a microcrack edge enhancement map, a water stain mask map, and a subsurface anomaly thermal map; the microcrack edge enhancement map, the water stain mask map, and the subsurface anomaly thermal map are stored in the R channel, G channel, and B channel, respectively, and the multispectral enhanced image is output; An image correction module is used to perform dynamic threshold denoising on the point cloud data and perform surface distortion correction on the point cloud data to obtain point cloud calibration data; and to perform spatial registration on the multispectral enhanced image and the point cloud calibration data; The crack recognition module is used to fuse the multispectral enhanced image and point cloud calibration data to construct a six-channel fusion tensor, extract dual-branch features from the six-channel fusion tensor, perform cross-modal attention fusion on the dual-branch features to obtain fusion features, and perform multi-scale context aggregation on the fusion features to obtain crack aggregation features. The output module is used to perform crack segmentation on the fused image according to the crack aggregation characteristics to obtain a dam crack recognition model; and obtain dam crack recognition results and crack sizes based on the dam crack recognition model.

9. An electronic device comprising a storage medium and a processor; the storage medium is used to store instructions; characterized in that: The processor is configured to operate according to the instructions to execute the dam crack identification method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, wherein when the computer program is executed, Implement the dam crack identification method as described in any one of claims 1 to 7.

Images