A crack depth three-dimensional reconstruction method under complex background for unmanned aerial vehicle bridge inspection

Abstract

The application discloses a crack depth three-dimensional reconstruction method under complex background for unmanned aerial vehicle bridge inspection, and belongs to the technical field of bridge engineering health monitoring. The application uses an unmanned aerial vehicle to carry a multi-sensing acquisition module to effectively acquire characteristic data of a bridge area to be inspected, accurately segments a crack area from the effective characteristic data, extracts two-dimensional contour features of the crack, maps the two-dimensional contour features of the crack into a three-dimensional point cloud model to complete preliminary three-dimensional positioning of the crack, reconstructs the depth of the bridge crack in three dimensions, and outputs a crack depth three-dimensional reconstruction model. The application uses a fusion model to realize pixel-level accurate segmentation of the crack area, guarantees high accuracy of crack identification and registration accuracy of two-dimensional and three-dimensional data, solves the problems of fuzzy crack extraction and positioning deviation under complex background, comprehensively integrates depth information of different angles through multi-view point cloud fusion, takes into account comprehensiveness and accuracy of depth calculation, and restores three-dimensional spatial form and depth distribution of the crack.

Description

Technical Field

[0001] This invention relates to the field of bridge engineering health monitoring technology, and in particular to a method for three-dimensional reconstruction of crack depth in complex backgrounds for bridge inspection by unmanned aerial vehicles (UAVs). Background Technology

[0002] Currently, bridge crack inspections mostly rely on manual inspection or simple drone image acquisition. Manual inspection has problems such as high risk of high-altitude operation, low inspection efficiency, and strong subjectivity. Traditional drones rely only on monocular image acquisition, which makes it difficult to accurately identify cracks in complex backgrounds such as bridge water stains, rust, and coating layers, and cannot directly obtain crack depth information.

[0003] Existing 3D reconstruction technology for bridge crack detection often suffers from problems such as poor correlation between multi-source data, low registration accuracy between 2D and 3D data, and the susceptibility of depth calculation to the influence of viewing angle, resulting in large errors in crack depth reconstruction results. This makes it difficult to meet the actual engineering needs for refined and quantitative crack detection. Summary of the Invention

[0004] The purpose of this invention is to provide a method for three-dimensional reconstruction of crack depth in complex backgrounds for bridge inspection using unmanned aerial vehicles (UAVs), so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for three-dimensional reconstruction of crack depth under complex backgrounds for bridge inspection using unmanned aerial vehicles (UAVs), comprising the following steps:

[0006] Using drones equipped with multi-sensor acquisition modules, multi-source data is collected from the bridge area to be inspected, obtaining raw data including bridge surface texture, depth and spectral information.

[0007] The raw collected data is preprocessed to obtain effective feature data. Based on the preset crack feature recognition model, the crack region is accurately segmented and the two-dimensional contour features of the crack are extracted.

[0008] A three-dimensional point cloud model of the bridge inspection area is constructed. The two-dimensional contour features of the crack are mapped to the three-dimensional point cloud model to complete the preliminary three-dimensional location of the crack. Multi-view fusion and depth value iterative optimization algorithm are used to perform depth calculation and accuracy correction on the preliminary three-dimensional location results of the crack. The bridge crack depth is reconstructed in three dimensions, and the crack depth three-dimensional reconstruction model is output.

[0009] Furthermore, the preprocessing of the raw collected data includes:

[0010] The raw acquired data is classified according to the sensing type. The raw acquired data includes image data, depth data and multispectral data. Gaussian filtering and histogram equalization are performed on the image data acquired by the industrial camera, outlier removal is performed on the depth data acquired by the lidar, and spectral reflectance correction is performed on the multispectral data.

[0011] A complex background mask library is established based on the background features of the bridge. The background mask library includes background feature templates for the bridge concrete base color, water stains, rust, and coating layer. Invalid information that matches the background features is removed from the original collected data after classification by matching the background feature templates, and valid feature data is obtained.

[0012] Furthermore, the preset crack feature recognition model is a fusion network model based on YOLOv8 and U-Net, and the model training process includes:

[0013] A bridge crack sample dataset is constructed, which includes crack images of different bridge types and under different complex backgrounds, along with corresponding pixel-level annotation labels.

[0014] The bridge crack sample dataset was divided into training, validation and test sets in a ratio of 8:1:1. The fusion network model was trained and iteratively optimized. The crossover ratio and recognition accuracy of the crack regions were used as evaluation indicators. When the recognition accuracy of the fusion network model was not less than 95%, it was determined to be a crack feature recognition model that had been trained.

[0015] The effective feature data is input into the trained model, and the output is a binarized segmented image of the crack region. The two-dimensional contour features of the crack in the binarized segmented image are extracted. The two-dimensional contour features of the crack include the length, width, direction and edge pixel coordinates of the crack.

[0016] Furthermore, a 3D point cloud model of the bridge inspection area is constructed, specifically including:

[0017] The depth data collected by the lidar is transformed into point cloud data in the local coordinate system of the UAV and converted into point cloud data in the geodetic coordinate system.

[0018] The point cloud data after coordinate transformation is registered and stitched together using the iterative nearest point algorithm to obtain the initial three-dimensional point cloud model of the bridge inspection area. The initial three-dimensional point cloud model is then downsampled and smoothed to obtain a refined three-dimensional point cloud model of the bridge inspection area.

[0019] Furthermore, the two-dimensional contour features of the crack are mapped onto a three-dimensional point cloud model to complete the initial three-dimensional localization of the crack, specifically including:

[0020] Corner points, edges, and textures from image data captured by industrial cameras are extracted as image feature points, and curvature extrema and normal vector abrupt change points from lidar point cloud data are extracted as point cloud feature points.

[0021] Image feature points and point cloud feature points are used as common feature points of the image and point cloud to generate a common feature point set, which contains the correspondence data between image pixel coordinates and original point cloud coordinates.

[0022] Perform coarse and fine matching on the common feature point set;

[0023] By eliminating mismatched points in the coarse matching using the Euclidean distance method, a set of effective common feature points is obtained.

[0024] The effective common feature point set is finely matched and optimized based on the random sampling consensus algorithm to generate optimized common feature point pair data.

[0025] Substitute the optimized common feature point pair data into the homography matrix solution formula to calculate the homography transformation matrix between the image pixel coordinate system and the point cloud 3D coordinate system;

[0026] Substitute the coordinates of all edge pixels of the two-dimensional contour feature of the crack into the homography transformation matrix to perform coordinate transformation calculation and generate three-dimensional spatial coordinate data corresponding to each edge pixel.

[0027] The generated three-dimensional spatial coordinate data is matched with the point cloud coordinates in the refined three-dimensional point cloud model of the bridge inspection area. Point cloud clusters that match the three-dimensional spatial coordinates of the crack are marked, and point cloud marker data of the crack area is generated as the preliminary three-dimensional positioning result of the crack.

[0028] Furthermore, the multi-view fusion and depth value iterative optimization algorithm includes a multi-view data fusion module and a depth value iterative correction module, wherein the working process of the multi-view data fusion module is as follows:

[0029] Collect effective feature data from multiple perspectives of the same crack area taken by drone from different angles, extract the two-dimensional contour features of the crack from each perspective and map them to a three-dimensional point cloud model to obtain a multi-view crack point cloud cluster.

[0030] Multi-view crack point cloud clusters are fused, and the average depth of point clouds at the same spatial location is calculated by weighted averaging to obtain a crack fused point cloud model. The weight values ​​are allocated according to the verticality and sharpness of the drone shooting angle.

[0031] Furthermore, the working process of the depth value iterative correction module is as follows:

[0032] The depth value of the crack fusion point cloud model is used as the initial depth value, and the depth value iteration threshold and the maximum number of iterations are set.

[0033] Depth constraints are established based on the physical characteristics of cracks. These constraints include the ratio of crack depth to width and the limit value of crack depth corresponding to the material properties of the bridge concrete structure.

[0034] The initial depth value is iteratively optimized. After each iteration, the residual of the depth value is calculated. When the residual is less than the iteration threshold or the number of iterations reaches the maximum number of iterations, the iteration is stopped, and the optimized crack depth value is obtained.

[0035] Furthermore, the output includes a three-dimensional reconstruction model of the crack depth, specifically:

[0036] The optimized crack depth value is assigned to the corresponding point cloud cluster of the crack fusion point cloud model to complete the three-dimensional reconstruction of crack depth and generate a three-dimensional reconstruction model of crack depth.

[0037] The three-dimensional feature parameters of the crack are extracted from the three-dimensional reconstruction model of the crack depth. The three-dimensional feature parameters include the three-dimensional spatial coordinates of the crack, the maximum depth, the average depth, the depth variation gradient, the crack volume, and the relative positional relationship between the crack and the bridge structure. The three-dimensional reconstruction model and feature parameters are then used to generate a standardized inspection report.

[0038] Furthermore, the multi-sensor acquisition module carried by the UAV includes a high-resolution industrial camera, a lidar ranging sensor, a multispectral imaging sensor, and an attitude positioning module. The attitude positioning module is a GPS / IMU combined positioning unit, used to collect the UAV's flight attitude, position, and shooting angle data in real time, and to synchronize and bind the data with the multi-source acquisition data using timestamps.

[0039] Furthermore, the background mask library is a dynamically updated library, which annotates and adds newly added complex background features of the bridge during each inspection process, while iteratively optimizing the original feature templates in the library.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] 1. This invention utilizes a drone equipped with multiple types of sensors to simultaneously collect multi-source data from bridges and timestamps the data. Combined with a dynamically updated background mask library to remove invalid information, it achieves integrated acquisition of multi-dimensional data such as texture and depth. A fusion model is used to achieve pixel-level precise segmentation of crack areas, extracting quantified two-dimensional contour features. Through coordinate transformation and precise registration of point clouds to solve the homography transformation matrix, the two-dimensional contour is accurately mapped to a three-dimensional point cloud model, ensuring high accuracy in crack identification and high precision in the registration of two-dimensional and three-dimensional data. This solves the problems of fuzzy crack extraction and positioning deviation in complex backgrounds, and achieves precise three-dimensional positioning of cracks.

[0042] 2. The multi-view fusion and depth value iterative optimization reconstruction of this invention integrates depth information from different angles through multi-view point cloud fusion, and establishes constraints based on crack physical characteristics to iteratively optimize the depth value. Finally, it generates a three-dimensional reconstruction model with accurate depth and extracts quantitative parameters, taking into account both the comprehensiveness and accuracy of depth calculation, restoring the three-dimensional spatial morphology and depth distribution of cracks, and outputting a standardized inspection report to provide quantitative basis for bridge inspection decisions, thereby improving the accuracy of crack reconstruction and the application value of inspection results. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of the three-dimensional reconstruction method for crack depth according to the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0045] Please see Figure 1 The present invention provides the following technical solutions:

[0046] A method for three-dimensional reconstruction of crack depth in complex backgrounds for bridge inspection using unmanned aerial vehicles (UAVs) includes the following steps:

[0047] The drone is equipped with a multi-sensor acquisition module to collect multi-source data on the bridge inspection area, and obtain raw data including bridge surface texture, depth and spectral information. The multi-sensor acquisition module on the drone includes a high-resolution industrial camera, a lidar ranging sensor, a multispectral imaging sensor and an attitude positioning module. The attitude positioning module is a GPS / IMU combined positioning unit, which is used to collect the drone's flight attitude, position and shooting angle data in real time, and to timestamp and bind it with the multi-source acquisition data.

[0048] The original collected data is preprocessed by complex background noise reduction and invalid information removal to obtain effective feature data. Based on the preset crack feature recognition model, the crack region is accurately segmented and the two-dimensional contour features of the crack are extracted.

[0049] A three-dimensional point cloud model of the bridge inspection area is constructed. The two-dimensional contour features of the crack are mapped to the three-dimensional point cloud model to complete the preliminary three-dimensional location of the crack. Multi-view fusion and depth value iterative optimization algorithm are used to perform depth calculation and accuracy correction on the preliminary three-dimensional location results of the crack. The bridge crack depth is reconstructed in three dimensions, and the crack depth three-dimensional reconstruction model and related parameters are output.

[0050] In the above embodiments, multi-source data acquisition for bridge inspection is carried out by using a drone equipped with multiple types of sensor acquisition modules. The acquisition capabilities of high-resolution industrial cameras, LiDAR, and multispectral imaging sensors are integrated, and the attitude and position acquisition function of the GPS / IMU combined positioning unit is combined with the integrated acquisition of texture, depth, spectrum, and drone flight status data. The timestamp synchronization binding method allows different sensor data to form a complete data system with spatiotemporal correlation, providing a multi-dimensional and highly correlated data foundation for crack identification and 3D reconstruction in complex backgrounds. By complementing multi-source data, the problems of missing texture information and insufficient depth perception in complex backgrounds are effectively avoided, improving the effectiveness and accuracy of subsequent data processing.

[0051] The process of preprocessing raw collected data includes:

[0052] The raw data is classified according to the sensor type. The raw data includes image data, depth data and multispectral data. Gaussian filtering and histogram equalization are performed on the image data acquired by the industrial camera, outlier removal is performed on the depth data acquired by the lidar, and spectral reflectance correction is performed on the multispectral data.

[0053] A complex background mask library is established based on bridge background features. The background mask library includes background feature templates for the bridge's concrete base color, water stains, rust, and coating layers. Invalid information that does not match the background features is removed from the original collected data after classification by matching the background feature templates, and valid feature data is obtained. The background mask library is a dynamically updated library. New complex background features of the bridge added during each inspection are labeled and added to the library. At the same time, the original feature templates in the library are iteratively optimized to improve the accuracy of background matching and invalid information removal.

[0054] In the above embodiments, the categorized preprocessing method for the original acquired data enables targeted noise reduction and correction of different sensor data, preserving the effective features of image, depth, and multispectral data. The establishment of a complex background mask library can accurately match and remove invalid background information such as the concrete background color and water stains of the bridge, filtering interference factors from the data source and improving the purity of effective feature data. The dynamic update mechanism of the mask library can continuously incorporate newly added complex background features and optimize the original templates, so that the ability of background matching and invalid information removal can be continuously improved with the number of inspections, adapting to the inspection needs of different bridges and different environments, and effectively solving the problem of poor adaptability of fixed background templates in complex and variable bridge scenarios.

[0055] The preset crack feature recognition model is a fusion network model based on YOLOv8 and U-Net. The model training process includes:

[0056] A bridge crack sample dataset was constructed, which includes crack images of different bridge types and under different complex backgrounds, along with corresponding pixel-level annotation labels.

[0057] The bridge crack sample dataset was divided into training, validation and test sets in a ratio of 8:1:1. The fusion network model was trained and iteratively optimized. The crossover ratio and recognition accuracy of the crack regions were used as evaluation indicators. When the recognition accuracy of the fusion network model was not less than 95%, it was determined to be a crack feature recognition model that had been trained.

[0058] The effective feature data is input into the trained model, and the output is a binarized segmented image of the crack region. The two-dimensional contour features of the crack in the binarized segmented image are extracted. The two-dimensional contour features of the crack include the length, width, direction and edge pixel coordinates of the crack.

[0059] In the above embodiments, a network model integrating YOLOv8 and U-Net is used for crack feature recognition. Combining the fast detection advantage of YOLOv8 with the pixel-level segmentation accuracy of U-Net, accurate and efficient segmentation of crack regions in complex backgrounds is achieved. By constructing a sample dataset containing multiple bridge types and complex backgrounds, the model is fully trained in diverse scenarios. After inputting effective feature data into the trained model, the output binary segmented image can clearly define the crack region. The extracted two-dimensional contour features such as length and width provide two-dimensional basic data for mapping to a three-dimensional point cloud model, effectively solving the problems of fuzzy crack feature extraction and inaccurate localization in complex backgrounds.

[0060] A 3D point cloud model of the bridge inspection area is constructed, and the 2D contour features of the cracks are mapped to the 3D point cloud model to complete the preliminary 3D localization of the cracks. Specifically, this includes:

[0061] The depth data collected by the lidar is transformed into point cloud data in the local coordinate system of the UAV and converted into point cloud data in the geodetic coordinate system.

[0062] The point cloud data after coordinate transformation is registered and stitched based on the iterative nearest point algorithm to obtain the initial three-dimensional point cloud model of the bridge inspection area. The initial three-dimensional point cloud model is then downsampled and smoothed to remove redundant and noise points in the point cloud data. The density and uniformity of the point cloud model are optimized to obtain a refined three-dimensional point cloud model of the bridge inspection area.

[0063] Corner points, edges, and textures from image data captured by industrial cameras are extracted as image feature points, and curvature extrema and normal vector abrupt change points from lidar point cloud data are extracted as point cloud feature points.

[0064] Image feature points and point cloud feature points are used as common feature points of the image and point cloud to generate a common feature point set. The common feature point set contains the correspondence data between image pixel coordinates and original point cloud coordinates.

[0065] Perform coarse and fine matching on the common feature point set;

[0066] By eliminating mismatched points in the coarse matching using the Euclidean distance method, a set of effective common feature points is obtained.

[0067] The effective common feature point set is finely matched and optimized based on the random sampling consensus algorithm to generate optimized common feature point pair data.

[0068] Substitute the optimized common feature point pair data into the homography matrix solution formula to calculate the homography transformation matrix between the image pixel coordinate system and the point cloud 3D coordinate system;

[0069] Substitute the coordinates of all edge pixels of the two-dimensional contour feature of the crack into the homography transformation matrix to perform coordinate transformation calculation and generate three-dimensional spatial coordinate data corresponding to each edge pixel.

[0070] The generated three-dimensional spatial coordinate data is matched with the point cloud coordinates in the refined three-dimensional point cloud model of the bridge inspection area. Point cloud clusters that match the three-dimensional spatial coordinates of the crack are marked, and point cloud marker data of the crack area is generated as the preliminary three-dimensional positioning result of the crack.

[0071] In the above embodiments, point cloud data in the local coordinate system of the UAV is converted into geodetic coordinate system data through coordinate transformation, realizing the standardization of point cloud data and providing a unified coordinate benchmark for point cloud registration and stitching. The application of the iterative nearest point algorithm significantly improves the accuracy of point cloud registration and stitching. Downsampling and smoothing processes optimize the density and uniformity of the point cloud model. The constructed refined three-dimensional point cloud model can realistically restore the spatial structure of the bridge inspection area. By extracting common feature points of the image and point cloud, and after coarse matching and fine matching to remove mismatched points, the homography transformation matrix is ​​solved to achieve accurate registration between the image pixel coordinate system and the point cloud three-dimensional coordinate system. This allows the two-dimensional contour features of the crack to be accurately mapped to the three-dimensional point cloud model. The generated crack area point cloud marker data provides a three-dimensional positioning basis for crack depth calculation and solves the positioning error problem caused by the registration deviation between two-dimensional and three-dimensional data.

[0072] The multi-view fusion and depth value iterative optimization algorithm includes a multi-view data fusion module and a depth value iterative correction module. The working process of the multi-view data fusion module is as follows:

[0073] Collect effective feature data from multiple perspectives of the same crack area taken by drone from different angles, extract the two-dimensional contour features of the crack from each perspective and map them to a three-dimensional point cloud model to obtain a multi-view crack point cloud cluster.

[0074] Multi-view crack point cloud clusters are fused, and the average depth of point clouds at the same spatial location is calculated by weighted average to obtain crack fused point cloud model. The weight values ​​are allocated according to the verticality and sharpness of the drone shooting angle. The higher the verticality of the shooting angle to the crack surface and the better the image sharpness, the larger the weight value.

[0075] The working process of the depth value iterative correction module is as follows:

[0076] The depth value of the crack fusion point cloud model is used as the initial depth value, and the depth value iteration threshold and the maximum number of iterations are set.

[0077] Depth constraints are established based on the physical characteristics of cracks. The constraints include the ratio of crack depth to width and the limit value of crack depth corresponding to the material properties of bridge concrete structures.

[0078] The initial depth value is iteratively optimized. After each iteration, the residual of the depth value is calculated. When the residual is less than the iteration threshold or the number of iterations reaches the maximum number of iterations, the iteration is stopped, and the optimized crack depth value is obtained.

[0079] In the above embodiments, the multi-view data fusion module collects crack feature data from different angles and generates multi-view point cloud clusters. It uses a weighted average method to fuse the point cloud depth values, and assigns weights based on shooting angle and sharpness. This allows the fused crack point cloud model to integrate depth information from multiple perspectives, effectively avoiding the depth perception blind spots and deviations caused by single-view shooting. The depth value iterative correction module uses the fused point cloud depth as the initial value, establishes constraints based on crack physical characteristics, and achieves accurate correction of crack depth values ​​through iterative optimization and setting iteration thresholds and maximum number of iterations. This ensures that the depth calculation results conform to actual physical laws, and the calculation accuracy is guaranteed by residual judgment. By combining the comprehensiveness of multi-view fusion with the accuracy of iterative optimization, it solves the problems of inaccurate depth calculation from a single perspective in complex backgrounds and the discrepancy between crack depth and actual physical characteristics, thus improving the accuracy of crack depth calculation.

[0080] Output a 3D reconstruction model of the crack depth, specifically including:

[0081] The optimized crack depth value is assigned to the corresponding point cloud cluster of the crack fusion point cloud model to complete the three-dimensional reconstruction of crack depth and generate a three-dimensional reconstruction model of crack depth.

[0082] The three-dimensional feature parameters of the crack are extracted from the three-dimensional reconstruction model of the crack depth. The three-dimensional feature parameters include the three-dimensional spatial coordinates of the crack, the maximum depth, the average depth, the depth variation gradient, the crack volume, and the relative positional relationship between the crack and the bridge structure. The three-dimensional reconstruction model and feature parameters are then used to generate a standardized inspection report.

[0083] In the above embodiments, the optimized crack depth value is assigned to the corresponding point cloud cluster of the fused point cloud model, realizing the precise combination of crack depth and three-dimensional point cloud. The generated three-dimensional reconstruction model of crack depth can intuitively and realistically restore the three-dimensional spatial morphology and depth distribution characteristics of cracks in the bridge structure. The feature parameters such as three-dimensional spatial coordinates, maximum depth, and volume extracted from the model realize the quantitative analysis of cracks, providing accurate quantitative basis for the grade assessment and maintenance decision of bridge cracks. The three-dimensional reconstruction model and feature parameters generate a standardized inspection report, making the inspection results form a standardized and systematic output form, which is convenient for relevant personnel to view, analyze and archive, and can more intuitively reflect the actual situation of cracks, effectively improving the application value and decision reference of bridge inspection results.

[0084] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for three-dimensional reconstruction of crack depth under complex backgrounds for bridge inspection using unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: Using drones equipped with multi-sensor acquisition modules, multi-source data is collected from the bridge area to be inspected, obtaining raw data including bridge surface texture, depth and spectral information. The raw collected data is preprocessed to obtain effective feature data. Based on the preset crack feature recognition model, the crack region is accurately segmented and the two-dimensional contour features of the crack are extracted. A three-dimensional point cloud model of the bridge inspection area is constructed. The two-dimensional contour features of the crack are mapped to the three-dimensional point cloud model to complete the preliminary three-dimensional location of the crack. The multi-view fusion and depth value iterative optimization algorithm is used to perform depth calculation and accuracy correction on the preliminary three-dimensional location results of the crack. The bridge crack depth is reconstructed in three dimensions, and the crack depth three-dimensional reconstruction model is output. The multi-view fusion and depth value iterative optimization algorithm includes a multi-view data fusion module and a depth value iterative correction module. The working process of the depth value iterative correction module is as follows: The depth value of the crack fusion point cloud model is used as the initial depth value, and the depth value iteration threshold and the maximum number of iterations are set. Depth constraints are established based on the physical characteristics of cracks. These constraints include the ratio of crack depth to width and the limit value of crack depth corresponding to the material properties of the bridge concrete structure. The initial depth value is iteratively optimized. After each iteration, the residual of the depth value is calculated. When the residual is less than the iteration threshold or the number of iterations reaches the maximum number of iterations, the iteration is stopped, and the optimized crack depth value is obtained. Output a 3D reconstruction model of the crack depth, specifically including: The optimized crack depth value is assigned to the corresponding point cloud cluster of the crack fusion point cloud model to complete the three-dimensional reconstruction of crack depth and generate a three-dimensional reconstruction model of crack depth. The three-dimensional feature parameters of the crack are extracted from the three-dimensional reconstruction model of the crack depth. The three-dimensional feature parameters include the three-dimensional spatial coordinates of the crack, the maximum depth, the average depth, the depth variation gradient, the crack volume, and the relative positional relationship between the crack and the bridge structure. The three-dimensional reconstruction model and feature parameters are then used to generate a standardized inspection report.

2. The method for three-dimensional reconstruction of crack depth under complex backgrounds for UAV bridge inspection as described in claim 1, characterized in that, The process of preprocessing raw collected data includes: The raw acquired data is classified according to the sensing type. The raw acquired data includes image data, depth data and multispectral data. Gaussian filtering and histogram equalization are performed on the image data acquired by the industrial camera, outlier removal is performed on the depth data acquired by the lidar, and spectral reflectance correction is performed on the multispectral data. A complex background mask library is established based on the background features of the bridge. The background mask library includes background feature templates for the bridge concrete base color, water stains, rust, and coating layer. Invalid information that matches the background features is removed from the original collected data after classification by matching the background feature templates, and valid feature data is obtained.

3. The method for three-dimensional reconstruction of crack depth in complex backgrounds for UAV bridge inspection as described in claim 1, characterized in that, The preset crack feature recognition model is a fusion network model based on YOLOv8 and U-Net, and the model training process includes: A bridge crack sample dataset is constructed, which includes crack images of different bridge types and under different complex backgrounds, along with corresponding pixel-level annotation labels. The bridge crack sample dataset was divided into training, validation and test sets in a ratio of 8:1:

1. The fusion network model was trained and iteratively optimized. The crossover ratio and recognition accuracy of the crack regions were used as evaluation indicators. When the recognition accuracy of the fusion network model was not less than 95%, it was determined to be a crack feature recognition model that had been trained. The effective feature data is input into the trained model, and the output is a binarized segmented image of the crack region. The two-dimensional contour features of the crack in the binarized segmented image are extracted. The two-dimensional contour features of the crack include the length, width, direction and edge pixel coordinates of the crack.

4. The method for three-dimensional reconstruction of crack depth in complex backgrounds for UAV bridge inspection as described in claim 1, characterized in that, Constructing a 3D point cloud model of the bridge inspection area, specifically including: The depth data collected by the lidar is transformed into point cloud data in the local coordinate system of the UAV and converted into point cloud data in the geodetic coordinate system. The point cloud data after coordinate transformation is registered and stitched together using the iterative nearest point algorithm to obtain the initial three-dimensional point cloud model of the bridge inspection area. The initial three-dimensional point cloud model is then downsampled and smoothed to obtain a refined three-dimensional point cloud model of the bridge inspection area.

5. The method for three-dimensional reconstruction of crack depth under complex backgrounds for UAV bridge inspection as described in claim 4, characterized in that, The initial 3D localization of the crack is achieved by mapping the 2D contour features of the crack onto a 3D point cloud model. Specifically, this includes: Corner points, edges, and textures from image data captured by industrial cameras are extracted as image feature points, and curvature extrema and normal vector abrupt change points from lidar point cloud data are extracted as point cloud feature points. Image feature points and point cloud feature points are used as common feature points of the image and point cloud to generate a common feature point set, which contains the correspondence data between image pixel coordinates and original point cloud coordinates. Perform coarse and fine matching on the common feature point set; By eliminating mismatched points in the coarse matching using the Euclidean distance method, a set of effective common feature points is obtained. The effective common feature point set is finely matched and optimized based on the random sampling consensus algorithm to generate optimized common feature point pair data. Substitute the optimized common feature point pair data into the homography matrix solution formula to calculate the homography transformation matrix between the image pixel coordinate system and the point cloud 3D coordinate system; Substitute the coordinates of all edge pixels of the two-dimensional contour feature of the crack into the homography transformation matrix to perform coordinate transformation calculation and generate three-dimensional spatial coordinate data corresponding to each edge pixel. The generated three-dimensional spatial coordinate data is matched with the point cloud coordinates in the refined three-dimensional point cloud model of the bridge inspection area. Point cloud clusters that match the three-dimensional spatial coordinates of the crack are marked, and point cloud marker data of the crack area is generated as the preliminary three-dimensional positioning result of the crack.

6. The method for three-dimensional reconstruction of crack depth under complex backgrounds for UAV bridge inspection as described in claim 1, characterized in that, The working process of the multi-view data fusion module is as follows: Collect effective feature data from multiple perspectives of the same crack area taken by drone from different angles, extract the two-dimensional contour features of the crack from each perspective and map them to a three-dimensional point cloud model to obtain a multi-view crack point cloud cluster. Multi-view crack point cloud clusters are fused, and the average depth of point clouds at the same spatial location is calculated by weighted averaging to obtain a crack fused point cloud model. The weight values ​​are allocated according to the verticality and sharpness of the drone shooting angle.

7. The method for three-dimensional reconstruction of crack depth under complex backgrounds for UAV bridge inspection as described in claim 1, characterized in that, The multi-sensor acquisition module carried by the UAV includes a high-resolution industrial camera, a lidar ranging sensor, a multispectral imaging sensor, and an attitude positioning module. The attitude positioning module is a GPS / IMU combined positioning unit, which is used to collect the UAV's flight attitude, position, and shooting angle data in real time, and to timestamp and bind it with the multi-source acquired data.

8. The method for three-dimensional reconstruction of crack depth under complex backgrounds for UAV bridge inspection as described in claim 2, characterized in that, The background mask library is a dynamically updated library. It annotates and adds newly added complex background features of the bridge during each inspection process, while iteratively optimizing the original feature templates in the library.

Images