A tunnel crack three-dimensional quantification method fusing visible light and laser point clouds

Abstract

The application discloses a tunnel crack three-dimensional quantification method fusing visible light and laser point clouds, relates to the technical field of tunnel engineering detection and multi-modal data fusion processing, and comprises the following steps: synchronously collecting visible light images and laser point clouds, constructing offset proxy parameters, projecting an image crack mask to a tunnel coordinate system to obtain a candidate point cloud subset; extracting a crack two-dimensional geometric constraint and a point cloud local geometric feature, iteratively optimizing the offset proxy parameters to make the two reach spatial consistency, and generating an optimized projection point cloud subset through double-stage closed loop verification; fusing three-dimensional coordinates, image features and width estimation to construct a multi-modal feature vector, and calculating a three-dimensional length, width distribution and depth of the crack. The application realizes accurate alignment and heterogeneous feature depth fusion of visible light and point clouds under dynamic acquisition conditions, the output crack three-dimensional parameters are reliable, and the application is suitable for automatic accurate detection and quantitative evaluation of tunnel lining cracks.

Description

Technical Field

[0001] This invention relates to the field of tunnel engineering detection and multimodal data fusion processing technology, specifically a method for three-dimensional quantification of tunnel cracks by fusing visible light and laser point clouds. Background Technology

[0002] Tunnel lining cracks are a major concern affecting structural safety and operational lifespan. Their width, depth, length, and spatial distribution are core quantitative indicators for assessing structural deterioration and developing maintenance plans. Current crack detection primarily relies on single technological approaches such as visual inspection, 2D image recognition, or LiDAR 3D scanning. Visible light imaging methods can capture surface crack texture and edge information, and combined with deep learning, crack identification can be achieved at the pixel level. However, 2D images are essentially projections of 3D space. A single image cannot reconstruct the true depth and spatial orientation of the crack due to perspective distortion, tunnel surface curvature, and lighting conditions. The measured width and length are only approximations of the image plane, making it difficult to establish a strict correspondence with the actual 3D coordinates of the tunnel, and also making it impossible to calculate 3D indicators such as crack volume or cross-sectional attenuation rate. LiDAR-based 3D point cloud methods can directly obtain the spatial coordinates of the lining surface and detect large-scale deformations and misalignments. However, cracks, as fine discontinuous structures, often have geometric abrupt changes smaller than the inherent noise level of the point cloud, and thus possess almost no identifiable geometric features in the original point cloud. Existing extraction algorithms based on normal vectors or curvature abrupt changes are prone to misidentifying dust, water stains, or scanning noise as cracks, while also missing a large number of early-developing cracks, making it difficult to meet engineering requirements in terms of the reliability of the detection results.

[0003] Therefore, fusing visible light images and laser point clouds to achieve 3D crack quantization has become a recognized important direction in this field. However, existing fusion schemes generally adopt "post-fusion" or "shallow fusion" strategies: first, cracks are detected in the image domain and geometric models are established in the point cloud domain, and then the 2D crack contours are mapped to 3D space through an extrinsic parameter matrix. These methods face deep technical problems in dynamic inspection scenarios: spatiotemporal asynchrony and mismatch of heterogeneous features. Specifically, during the movement of the inspection vehicle, the camera exposure integration time and the LiDAR scanning cycle are not synchronized. Even with a time synchronization mechanism, the edge position of the crack captured by the two in the image and the corresponding spatial point in the point cloud will still have a small but non-negligible coordinate offset due to changes in vehicle attitude, vibration, and calibration residuals. This offset may be on the same order of magnitude as the crack width itself, causing the image crack to be projected onto the point cloud and unable to be accurately aligned with a subset of the real crack point cloud. Crack width and depth quantization can only rely on interpolation estimation of sparse neighboring points, and the quantization error significantly exceeds the allowable range in engineering. Furthermore, cracks appear as high-contrast linear edges in images, but as weak depressions without obvious jumps in point clouds. The two modal features are completely different in saliency. Existing fusion networks based on feature splicing or simple attention mechanisms have not yet been able to solve this heterogeneous gap from the underlying feature level.

[0004] In summary, the technical problem to be solved by this invention is how to achieve high-precision spatiotemporal alignment between visible light images and laser point clouds under dynamic acquisition conditions, and how to construct a three-dimensional quantization model of cracks with deep fusion of heterogeneous features. Summary of the Invention

[0005] The purpose of this invention is to provide a method for three-dimensional quantization of tunnel cracks that integrates visible light and laser point clouds, so as to solve the problems mentioned in the background art.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a method for three-dimensional quantization of tunnel cracks by fusing visible light and laser point clouds, comprising the following steps: S1: During the dynamic inspection of the tunnel lining, visible light images and laser point clouds are acquired simultaneously to establish the absolute coordinate system of the tunnel and identify the crack mask in the image domain. In this step, the camera and LiDAR are synchronously acquired through the same hardware trigger signal. The absolute coordinate system of the tunnel is established in real time by fusing the laser point cloud and inertial measurement unit data using real-time localization and map building algorithms. The crack mask is obtained by segmentation using a deep convolutional network with an encoder-decoder structure. S2: Construct offset proxy parameters, project the crack mask onto the absolute coordinate system of the tunnel, and obtain a subset of candidate 3D point clouds; the initial value of the offset proxy parameters is zero, and the range of values ​​in the iterative optimization process is limited by the extrinsic parameter calibration residuals of the camera and lidar and the attitude change range of the inspection vehicle. This subset includes real crack point clouds and noise points.

[0007] S3: Extract the two-dimensional geometric constraint features of the crack mask, extract the local geometric features of the candidate three-dimensional point cloud subset, and iteratively optimize the offset surrogate parameters to make the two-dimensional geometric constraint features and the local geometric features spatially consistent, thereby obtaining the optimized projected point cloud subset; during the iteration process, the candidate point cloud subset is back-projected back into the image domain to generate a virtual crack line map and compare it with the crack mask, and the difference map is used as a constraint term to form a two-way closed-loop verification. The offset is solved by the Levenberg-Marquardt algorithm in a nonlinear least squares form.

[0008] S4: The three-dimensional coordinates of the optimized projection point cloud subset are fused with the corresponding crack mask image features to construct a multimodal fusion feature vector. This vector is composed of the three-dimensional coordinates of each point, the gray value of the corresponding crack mask pixel, and the estimated local crack width obtained by spatial interpolation, while carrying spatial geometric information and texture information.

[0009] S5: Based on the multimodal fusion feature vector, calculate the three-dimensional spatial parameters of the crack, including the length, width distribution, and depth of the three-dimensional centerline, and output structured data. Obtain the three-dimensional centerline by skeletonizing and fitting spline curves to the subset of the projected point cloud. Calculate the width by identifying the edge boundaries based on the point cloud density gradient within the centerline normal plane. Calculate the average depth and maximum depth using the perpendicular distance between the bottom point of the crack and the background surface.

[0010] Preferably, the process of iteratively optimizing the offset proxy parameters in step S3 includes: back-projecting the candidate 3D point cloud subset to the image domain to generate a virtual crack line map; comparing the virtual crack line map with the crack mask; using the comparison result as a constraint term of the optimization objective function; and iteratively updating the offset proxy parameters. This process forms a bidirectional consistency constraint of unidirectional projection and back-verification, ensuring that the optimization objective function simultaneously includes error terms in both the image-to-point-cloud and point-cloud-to-image directions.

[0011] Preferably, the iterative optimization process employs a two-stage closed-loop verification: the first iteration uses the perpendicular relationship between the local crack orientation and the point cloud normal vector as a constraint to solve for the first offset, obtaining a first updated point cloud subset; the second iteration uses the difference map between the virtual crack line generated by the back projection of the first updated point cloud subset and the original crack mask as a new constraint term to solve for the second offset, obtaining a second updated point cloud subset. The first stage solves for global alignment of directions, while the second stage solves for fine-tuning of shape and position. This hierarchical processing, prioritizing geometry over shape, avoids getting trapped in local optima.

[0012] Preferably, step S3 further includes setting an intersection-union ratio (IU) threshold, comparing the IU value of the virtual crack line map and the crack mask with the IU threshold, and triggering the next iteration if the IU value is lower than the threshold, until the IU reaches or exceeds the threshold, at which point the final optimized subset of the projected point cloud is output. This threshold verification mechanism provides a quantifiable criterion for iterative convergence, ensuring that the output results have bidirectional mapping consistency.

[0013] Preferably, the two-dimensional geometric constraint features include the local crack orientation vector, the parallelism constraint of the crack edges, and the image domain estimate of the crack width; the local geometric features include the local normal vector distribution of the point cloud, the local surface curvature, and the point cloud density gradient. The perpendicular relationship between the crack orientation vector and the normal vector distribution is used to constrain the projection direction, the edge parallelism and density gradient are combined to verify the accuracy of the projection position, and the width estimation and surface curvature are combined to determine the spatial morphology of the crack region.

[0014] Preferably, the spatial consistency in step S3 is achieved by solving an optimization objective function. This objective function aims to make the distribution of the point cloud normal vectors within the projection region corresponding to the local orientation vector of the image crack perpendicular to the local orientation vector, and to make the curvature of the point cloud local surface within the projection region exhibit abrupt concave-abrupt characteristics. The optimization objective function uses the perpendicularity of the crack orientation to the normal vector as a first constraint, and the valley-like distribution of curvature (high in the center and low on both sides) as a second constraint. It achieves both directional alignment and morphological conformity by minimizing the weighted sum of these two constraints.

[0015] Preferably, the multimodal fusion feature vector in step S4 is composed of the three-dimensional coordinates of each point in the projection point cloud subset, the corresponding crack mask pixel grayscale value, and the estimated local crack width. This stitching method preserves information on spatial location, texture intensity, and width priors, providing multimodal data support for subsequent width and depth calculations.

[0016] Preferably, the calculation of the three-dimensional spatial parameters of the crack in step S5 specifically includes: skeletonizing and fitting spline curves to the final optimized subset of projected point clouds to obtain a three-dimensional centerline curve, outputting the three-dimensional length and orientation; at equidistant sampling points on the centerline, combining the crack width estimate in the multimodal fusion feature vector with the point cloud density boundary in the normal plane, calculating the true width in three-dimensional space, and outputting the width distribution curve; using the elevation difference between the bottom point of the crack in the subset of projected point clouds and the lining background point cloud, combined with local surface fitting, outputting the average depth and maximum depth of the crack. Crack edges are identified by the location of abrupt changes in point cloud density gradient, avoiding reliance on single-modal information; depth calculation uses statistical values ​​of the vertical distance after background surface fitting.

[0017] Preferably, in step S1, identifying the crack mask in the image domain uses a pixel-level segmentation network based on deep learning; the synchronous acquisition uses a unified timestamp trigger signal; and the tunnel absolute coordinate system is established in real time using an instantaneous localization and mapping algorithm that fuses laser point clouds and inertial measurement units. Hardware triggering ensures time synchronization, and extended Kalman filtering or graph optimization fusion pose estimation enables the point cloud frames to be registered to the world coordinate system and aligned with the design drawings.

[0018] Preferably, the initial value of the offset proxy parameter is zero, and its value range during the iterative optimization process is limited by the extrinsic parameter calibration residuals of the camera and lidar and the attitude change range of the inspection vehicle. This value range is used as a hard boundary embedding optimization solution. During each iteration update, it is determined whether the boundary is exceeded and clamped to ensure convergence to the physically possible parameters.

[0019] This invention provides a method for three-dimensional quantization of tunnel cracks by fusing visible light and laser point clouds. It has the following beneficial effects: This method addresses the spatiotemporal asynchrony between visible light images and laser point clouds under dynamic inspection conditions by constructing offset surrogate parameters and employing a two-order closed-loop iterative projection mechanism. In traditional methods, coordinate offsets caused by carrier attitude changes, vibrations, and calibration residuals directly lead to the inability of the projected image cracks to align with the actual crack point cloud. This method, however, does not aim to eliminate the offset itself but uses it as a surrogate parameter to be optimized. Through bidirectional projection consistency constraints between image features and point cloud geometric features, it actively approximates the true offset value during the iteration process. Simultaneously, by comparing the virtual crack line map generated by the back-projection of the point cloud with the original crack mask and feeding the comparison result back to the optimization objective function, a self-correcting closed loop is formed. This ensures that even with initial mapping deviations, the projected point cloud subset achieves spatial morphological consistency with the image cracks. This mechanism makes the method tolerant to sensor synchronization errors, motion disturbances, and calibration residuals, guaranteeing the reliability of crack projection positioning in real tunnel inspection environments.

[0020] This method successfully addresses the heterogeneous feature mismatch problem between visible light and lidar by extracting the two-dimensional geometric constraints of cracks and the local geometric features of point clouds, and constructing an optimization objective function with both perpendicular orientation and concave shape constraints. Cracks appear as high-contrast linear edges in images, but as slight concavities in point clouds; the feature saliencies of the two modalities are completely different. This method abandons the simple feature stitching or shallow attention fusion strategies of traditional techniques, instead employing a low-level feature alignment method based on spatial projection consistency, enabling semantic matching between the linear crack features in the image and the concave geometric features in the point cloud in the optimization space. Furthermore, it fuses the three-dimensional coordinates, image grayscale, and width estimation into a multimodal feature vector, further supporting the accurate calculation of the crack's three-dimensional length, width distribution, and depth. Therefore, the three-dimensional crack parameters output by this method possess spatial consistency and geometric interpretability, avoiding measurement biases caused by missing information or mismatches during single-modal quantization, and providing an accurate data foundation for tunnel structure safety assessment. Attached Figure Description

[0021] Figure 1 This is a flowchart illustrating the three-dimensional quantization method for tunnel cracks that integrates visible light and laser point clouds, as described in this invention. Figure 2 This is a flowchart of the two-order closed-loop iterative optimization process of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example: Please refer to Figure 1 and Figure 2 This invention provides a method for three-dimensional quantization of tunnel cracks by fusing visible light and laser point clouds: First, step S1 is executed: During the dynamic inspection of the tunnel lining, a visible light camera and a lidar mounted on a mobile platform are synchronously controlled by the edge computing unit using the same trigger signal to acquire visible light images and lidar point cloud frames of the same tunnel section. Simultaneously, the inertial measurement unit records the instantaneous attitude information of the platform. The edge computing unit utilizes the lidar point cloud and inertial measurement unit data to establish or register the data to the tunnel design coordinate system in real time using instant positioning and mapping algorithms, forming an absolute tunnel coordinate system with a unified spatial reference. In the image domain, a pixel-level segmentation network based on deep learning is used to process the visible light images, outputting a binarized crack mask that identifies the location of each pixel belonging to a crack in the image.

[0024] Then, step S2 is executed: Given the unknown coordinate offset between the camera and LiDAR under dynamic acquisition conditions caused by carrier attitude changes, vibration, and calibration residuals, this method does not assume a precise mapping relationship. Instead, it actively constructs a set of two-dimensional offset proxy parameters. These parameters characterize the possible coordinate offset when image pixels are projected onto the point cloud space, and their initial value is set to zero. Using the pre-calibrated extrinsic parameter matrix between the camera and LiDAR, combined with the current offset proxy parameters, each pixel in the crack mask is projected from the image coordinate system to the absolute coordinate system of the tunnel, thereby obtaining a candidate 3D point cloud subset. This subset is not the final result, but rather a preliminary set of spatial points containing the real crack point cloud and a large number of noise points.

[0025] Step S3: For the candidate 3D point cloud subset generated in Step S2, firstly, extract the 2D geometric constraint features from the crack mask, including the local crack orientation vector, the parallelism constraint of the crack edge, and the image domain estimate of the crack width. Simultaneously, spatial clustering is performed on the candidate 3D point cloud subset, and the local geometric features of the point set within each cluster are calculated, including the local normal vector distribution, local surface curvature, and point cloud density gradient. Then, an optimization objective function is constructed. The physical meaning of this function is to achieve spatial consistency between the 2D geometric constraints of the image crack and the local geometric features of the candidate point cloud subset in terms of projection relationship. Specifically, the offset surrogate parameter is iteratively adjusted so that for each local segment on the crack mask, the normal vector distribution within the point cloud subset covered by its projection is perpendicular to the crack orientation vector of that local segment, and the local surface curvature of that point cloud subset exhibits abrupt concavity characteristics. By solving this optimization problem, the first optimized offset is obtained, and this is used to correct the projection parameters to obtain the updated projected point cloud subset.

[0026] Furthermore, to converge the optimization results to a more accurate state, the updated projected point cloud subset is back-projected onto the image domain to generate a virtual crack line map. This virtual crack line map is then compared pixel-by-pixel with the original crack mask to calculate their overlap. The difference in overlap is used as a new constraint term, re-inputted into the optimization objective function, triggering a second iteration to solve for the second optimization offset. This further corrects the projection parameters, yielding the final optimized projected point cloud subset. Through this bidirectional closed-loop verification, the high-contrast linear features of the image and the subtle geometric concavity features of the point cloud gradually achieve semantic alignment during the iteration process, thereby eliminating the impact of initial mapping errors on subsequent quantization.

[0027] In step S4, the 3D coordinates of each point in the final optimized projection point cloud subset are concatenated with the pixel grayscale values ​​and local crack width estimates in the corresponding original crack mask to form a multimodal fusion feature vector. This vector carries both the spatial location information and surface texture information of the crack.

[0028] Finally, step S5 is executed: Based on the aforementioned multimodal fusion feature vector, the three-dimensional spatial parameters of the crack are calculated: The optimized projection point cloud subset is subjected to skeleton extraction and spline curve fitting to obtain the three-dimensional centerline curve of the crack, and the length and direction of this curve in the tunnel's absolute coordinate system are output; sampling points are selected at equal intervals along the centerline, and at each sampling point, using the crack width image estimate from the fusion feature vector, combined with the point cloud density boundary in the point normal plane, the true width of the crack in three-dimensional space is calculated, and the width distribution along the crack direction is output; using the elevation difference between the bottom point of the crack and the surrounding lining background point cloud in the projection point cloud subset, combined with local surface fitting, the average depth and maximum depth of the crack are output. All calculated three-dimensional parameters, along with their spatial coordinate positions, are organized into structured data and output for subsequent maintenance decisions or three-dimensional model overlay display.

[0029] In the specific implementation of step S3, to further eliminate the coordinate deviation caused by the initial projection, after completing the first iteration optimization of the offset proxy parameters and obtaining the updated subset of the projected point cloud, this method does not directly use it as the final result, but performs a reverse verification operation. Specifically, for each 3D spatial point in the updated subset of the projected point cloud, using the inverse mapping matrix between the camera and the LiDAR, and in conjunction with the offset proxy parameters solved in the current iteration, it is projected back from the tunnel absolute coordinate system to the original image pixel coordinate system. This process remaps the 3D point cloud into a set of pixels located on the image plane. Connecting these pixels according to their spatial adjacency relationships generates a virtual crack line map. The specific connection method is as follows: First, for all pixels obtained by back projection, calculate the Euclidean distance between each point and its neighboring points within a radius of 3 pixels. If the distance is less than 2 pixels, it is considered that there is a connection between the two points, and an edge is established in the undirected graph. Then, starting from any unvisited pixel, traverse all points connected by edges according to depth-first search to form a connected component. For each connected component, extract its skeleton line: gradually remove the outer pixels using a morphological thinning algorithm to obtain lines with a single pixel width. If the connected component exhibits a branching structure, retain the longest path as the trunk and remove short branches with a length of less than 5 pixels. Finally, the skeleton lines of all connected components together constitute a virtual crack line graph. This virtual crack line graph represents the position and shape of the crack in the image as "perceived" by the point cloud under the current projection parameters.

[0030] Subsequently, the generated virtual crack line map is compared pixel-by-pixel with the original crack mask obtained through deep learning segmentation in step S1. The comparison involves calculating the area of ​​the overlapping region between the virtual crack line map and the crack mask on the image plane, as well as the area of ​​the virtual crack line map located outside the crack mask. These two comparison results together form a difference map, where each pixel value reflects the degree of deviation between the virtual crack line map and the real crack mask. This difference map is added as a new constraint to the original optimization objective function. Thus, the optimization objective function no longer relies solely on the unidirectional projection consistency from the image to the point cloud, but also incorporates the bidirectional consistency constraint after the point cloud is back-projected to the image.

[0031] In subsequent iterations, the optimization algorithm continues to adjust the offset surrogate parameters with the goal of minimizing the difference map. Each time a parameter adjustment is completed, the projection and inverse projection processes are re-executed to generate a new virtual crack line map and calculate a new difference map. When the area of ​​the non-overlapping region in the difference map gradually shrinks to an acceptable range, it indicates that the current offset surrogate parameters have achieved consistency between the image cracks and the point cloud geometric features under bidirectional mapping. This closed-loop mechanism—projecting the point cloud back into the image domain and comparing it with the original crack mask, then feeding the comparison result back to the optimization function—enables the offset surrogate parameter solution process to possess self-correcting capabilities, effectively overcoming the systemic bias that may accumulate during single-direction projection.

[0032] In the two-stage closed-loop verification process of step S3, the first iteration is performed first. At this time, the two-dimensional geometric constraint features extracted from the image crack mask are preferentially used, among which the local crack orientation vector is the key constraint. For each local cluster in the candidate 3D point cloud subset, the principal direction of its point cloud normal vector is calculated. The optimization objective function is set as follows: the orientation vector of a local segment on the crack mask satisfies the perpendicular relationship with the normal vector distribution within the point cloud cluster covered by the projection of that local segment. The algorithm solves for the first offset by adjusting the offset surrogate parameter and searching segment by segment for the parameter value that minimizes the error of this perpendicular relationship. After correcting the projection parameters using the first offset, the first updated point cloud subset is obtained. This stage mainly solves the global consistency alignment between the image crack direction and the point cloud surface normal, but the shape matching accuracy after the point cloud back projection has not yet been considered.

[0033] After completing the first iteration, the algorithm proceeds to the second iteration. The first updated point cloud subset is back-projected onto the image domain to generate a virtual crack line map, which is then compared pixel-by-pixel with the original crack mask to obtain a difference map. This difference map quantifies the deviation between the virtual crack line and the real crack mask at each pixel position. This difference map is added as a new constraint to the original optimization objective function. At this point, the optimization objective function contains two constraints: the first is the perpendicular relationship between the crack orientation and the point cloud normal vector, and the second is the consistency of the bidirectional mapping represented by the back-projected difference map. In the second iteration, the algorithm readjusts the offset surrogate parameter to simultaneously minimize the weighted sum of these two constraints, solving for the second offset. After correcting the projection parameters with the second offset, the second updated point cloud subset is obtained. This second updated point cloud subset, while maintaining the perpendicular relationship between the crack orientation and the local normal, further reduces the morphological deviation between the virtual crack line generated by the back-projection and the original crack mask to an acceptable range. This two-stage closed-loop design, which first aligns the geometric direction and then fine-tunes the shape and position, avoids residual projection deviation caused by getting stuck in local optima due to single optimization.

[0034] In the two-stage closed-loop iteration process of step S3, this method presets an intersection-union ratio (IU / R) threshold as a criterion for determining iteration convergence. After each iteration, the overlap between the virtual crack line map generated in the current round and the original crack mask is calculated to obtain their IU / R value. This IU / R value reflects the degree of overlap between the virtual crack line map and the real crack mask on the image plane. This IU / R value is compared with the preset IU / R threshold: if the IU / R value is lower than the threshold, it indicates that the bidirectional consistency under the current projection parameters has not yet met the requirements. At this time, the system automatically triggers the next iteration, continues to adjust the offset proxy parameters, and re-executes the projection, inverse projection, and comparison process; if the IU / R value reaches or exceeds the threshold, it is determined that the current optimization result has met the convergence condition, the iteration process is immediately terminated, and the subset of projected point clouds obtained in the current round is output as the final optimization result. Through this threshold verification mechanism, the waste of computational resources caused by infinite iteration is avoided, while ensuring that the output subset of projected point clouds has a quantifiable level of bidirectional mapping consistency, providing a reliable data foundation for the accurate calculation of subsequent crack 3D parameters.

[0035] In step S3, when extracting the two-dimensional geometric constraint features of the crack mask, the crack mask is first skeletonized to obtain the centerline of the crack. The tangent direction is calculated point-by-point along the centerline to obtain the local crack direction vector, which reflects the extension direction of the crack within that local region. Simultaneously, using the binarized boundary of the crack mask, the pixel sets of the crack's two edges are extracted, and the normal distance between the edge points is calculated to form a parallelism constraint on the crack edges. This constraint characterizes the uniformity of the crack width within a local area and the relative positional relationship of the edges. Furthermore, in the image domain, the initial value of the crack width is estimated by combining the pixel distance between edge point pairs with the camera's imaging scale factor. This initial value serves as the image domain estimate of the crack width and is used as a reference for subsequent three-dimensional quantization.

[0036] When extracting local geometric features from candidate 3D point cloud subsets, spatial clustering is first performed on the point cloud subsets, dividing spatially neighboring points into different clusters. For each cluster, principal component analysis is used to calculate its local normal vector distribution, i.e., by fitting a local plane to obtain the direction and dispersion of the normal vector. Simultaneously, the local surface curvature of the point cloud is calculated. This curvature is derived from the eigenvalue relationship of the covariance matrix of neighboring points and is used to characterize the flatness or curvature of the surface in this region. The point cloud density gradient is obtained by statistically analyzing the number of points per unit volume and its spatial rate of change, used to distinguish dense and sparse regions, thereby identifying the sparse features of the point cloud at crack depressions. The extracted 2D geometric constraint features and local geometric features are input into the optimization objective function. The perpendicular relationship between the crack orientation vector and the point cloud normal vector distribution is used to constrain the projection direction; the crack edge parallelism constraint combined with the point cloud density gradient is used to verify the accuracy of the projection position; and the crack width image estimate combined with the local surface curvature is used to determine the spatial morphology of the crack region.

[0037] In step S3, to achieve spatial consistency between the image crack and the geometric features of the point cloud, this method constructs an optimization objective function. The core constraints of this function consist of two parts. The first part concerns the spatial relationship between the crack orientation and the point cloud normal vector: for each local segment on the crack mask, after projecting it into the point cloud space, the distribution of the point cloud normal vector within the projected area should maintain a perpendicular relationship with the crack orientation vector of that local segment. Specifically, the cosine of the angle between the local crack orientation vector and the average normal vector of the projected point cloud subset is first calculated, and the deviation of this angle from 90 degrees is used as an error term during optimization. The second part concerns the local surface morphology of the crack: the crack manifests as a slight concave structure in the point cloud space, and the curvature of the local surface of the point cloud corresponding to this concavity should exhibit a valley-like abrupt change characteristic, i.e., the curvature value reaches a local maximum at the crack centerline and gradually decreases towards both sides. The optimization objective function uses the difference between the point cloud curvature distribution within the projected area and the ideal valley-shaped curve as the second error term. The ideal valley-shaped curve has a local maximum curvature at the crack centerline and decreases towards both sides. The objective function is optimized using a nonlinear least squares form, and the offset surrogate parameters are iteratively solved using the Levenberg-Marquardt algorithm. The goal is to minimize the weighted sum of these two error terms by adjusting the offset surrogate parameters. Specifically, the first error, the vertical orientation error, is calculated as follows: for each local segment on the crack mask, the unit direction of its orientation vector is taken. The normal vector of the point cloud in the corresponding region of the projected point cloud subset is obtained through principal component analysis to obtain the average normal vector. The absolute value of the dot product of these two vectors is calculated; the closer this absolute value is to 0, the better the vertical relationship. The optimization objective is to minimize the sum of the absolute values ​​of the dot products of all local segments. The second error, the morphological concavity error, is calculated as follows: in each local region of the projected point cloud subset, a narrow strip profile is taken along a direction perpendicular to the crack orientation. The curvature value of each point on the profile is calculated. A unimodal curve, such as a Gaussian curve, with the profile center as the peak value is fitted. The cumulative sum of the absolute differences between the actual curvature distribution and the ideal curve is calculated. The optimization objective is to minimize this cumulative sum. The weights of the two errors in the overall objective function are dynamically adjusted based on the actual data noise level, with a default initial weight ratio of 1:1. The offset parameters are iteratively updated using the Levenberg-Marquardt algorithm. After each iteration, the parameter change is checked until the change is less than a preset threshold of 0.01 pixels or the maximum number of iterations is reached (50), at which point iteration stops. This ensures that the point cloud within the final projection area simultaneously satisfies the dual constraints of perpendicular orientation and concave shape, thereby guaranteeing that the projected subset of point clouds not only aligns with the crack direction globally but also matches the spatial concavity features of the real crack in its local geometry.

[0038] In step S4, when constructing the multimodal fusion feature vector, the final optimized projection point cloud subset is used as the data basis. For each three-dimensional spatial point in this point cloud subset, its three-dimensional coordinates in the tunnel absolute coordinate system are first obtained, and these coordinates constitute the spatial position component of the feature vector. Then, using the projection mapping relationship corresponding to the point, its pixel position in the original visible light image is traced back, and the gray value at that pixel is read from the crack mask. This gray value reflects the texture intensity of the crack at that image position and constitutes the image feature component of the feature vector. At the same time, based on the crack width image domain estimate calculated in step S3, the discrete width estimate is assigned to each point in the projection point cloud subset using a spatial interpolation method, forming the local crack width estimate corresponding to each point, which constitutes the width prior component of the feature vector. The above three components are concatenated in the order of spatial coordinates, gray value, and width estimate to generate a multi-dimensional fusion feature vector for each point. The fused feature vector simultaneously carries the spatial geometric information of the point cloud, the texture information of the image, and the preliminary estimate of the crack width. This allows the calculation of the crack's three-dimensional parameters in subsequent steps to utilize data from different modalities, avoiding quantization bias caused by insufficient information from a single modality. The fused feature vectors of all points together constitute the multimodal feature dataset of the entire crack region, serving as the input for step S5.

[0039] In step S5, when calculating the three-dimensional parameters of the crack on the final optimized projection point cloud subset, the three-dimensional centerline is first extracted. A skeletonization algorithm based on Euclidean distance is used to gradually remove points from the edges of the point cloud subset, retaining a single-layer point set located at the center of the crack's extension direction, forming a discrete skeleton point sequence. After sorting these skeleton points according to their spatial adjacency, a cubic spline curve is used for fitting, resulting in a continuous and smooth spatial curve, which is the three-dimensional centerline of the crack. The spatial straight-line distances of each segment along this centerline from the starting point to the ending point are accumulated to obtain the three-dimensional length of the crack in the tunnel's absolute coordinate system; simultaneously, the tangent directions at each point on the centerline are recorded to form a spatial vector description of the crack's direction.

[0040] Multiple sampling points are selected along the centerline at equal arc length intervals. For each sampling point, the crack width image estimate carried in the multimodal fusion feature vector is first used as an initial width reference. Then, a normal plane perpendicular to the tangent of the centerline is constructed at the sampling point. Within the normal plane, all points in the projected point cloud subset falling within a spatial band of a predetermined thickness near the plane are searched, forming a local point cloud slice. Based on the spatial distribution density of these points, the positions of the crack edges are determined: since the point cloud at the bottom of the crack is sparse, while the point cloud of the lining background on both sides is dense, the locations of abrupt changes in density gradient along the normal direction are identified as crack edge boundaries by analyzing the gradient change of the point cloud density. The spatial distance between the two edge boundaries is calculated to obtain the true 3D spatial width at the sampling point. The width values ​​of all sampling points along the centerline are arranged in order to output the crack width distribution curve.

[0041] To calculate the crack depth, the point cloud located at the bottom of the crack is first separated from the projected point cloud subset. These points typically have relatively low elevation values. The specific separation method is as follows: The elevation values ​​of all points in the projected point cloud subset are calculated, i.e., the coordinate components perpendicular to the lining surface in the tunnel's absolute coordinate system, generating an elevation histogram. Points at the crack bottom correspond to local minimum peaks in the low-value region of the histogram, while the background points of the lining on both sides correspond to the high-value main peaks of the histogram. An elevation threshold is set, for example, the elevation value corresponding to the valley between the main peak and the minimum peak. Points with elevations below this threshold are classified as crack bottom point clouds, and the remaining points are temporarily considered background points. Simultaneously, after removing the crack bottom point clouds from the projected point cloud subset, points with a distance exceeding 5 cm from the crack centerline normal are also removed, resulting in the lining background point clouds of the undisturbed areas on both sides of the crack. Then, the moving least squares method is used to perform surface fitting on these background point clouds, obtaining a continuous and smooth background surface. The vertical distance from each point at the crack bottom to this background surface is calculated, yielding a series of depth sampling values. These sampled values ​​are statistically analyzed, and their arithmetic mean is taken as the average depth of the crack, while the maximum value is taken as the maximum depth of the crack. Through the above steps, the three-dimensional length, orientation, width distribution curves, average depth, and maximum depth of the crack are finally output, forming a complete three-dimensional quantification result of the crack.

[0042] In the specific implementation of step S1, the visible light camera and the lidar achieve synchronous acquisition through the same hardware trigger signal. This trigger signal is generated by the timer of the edge computing unit and sent to the exposure control terminal of the camera and the scanning start terminal of the lidar, ensuring that each frame of image and each frame of point cloud has the same timestamp. For crack mask recognition in the image domain, a deep convolutional network based on an encoder-decoder structure is adopted. This network takes the original visible light image as input, extracts multi-scale features through multi-layer downsampling, and then restores the original image resolution through upsampling and skip connections. Finally, it outputs a probability map of each pixel belonging to the crack category, which is then thresholded and binarized to obtain the crack mask. As a feasible implementation, this embodiment adopts the U-Net network structure: the encoder part contains 5 convolutional blocks, each consisting of two 3×3 convolutional layers (stride 1, padding 1) and a 2×2 max pooling layer (stride 2), with channel numbers of 64, 128, 256, 512, and 1024 respectively; the decoder part contains 4 upsampling blocks, each consisting of a 2×2 transposed convolution (stride 2) and two 3×3 convolutional layers, and is concatenated with the feature maps of the corresponding layers of the encoder through skip connections; the last layer uses a 1×1 convolution to map the number of channels to two (crack class and non-crack class), and uses the Softmax activation function to output a probability map. During training, a weighted sum of the Dice loss function and cross-entropy loss is used, and the training dataset contains no less than two thousand tunnel lining images with manually labeled crack masks. In actual inference, the probability threshold is set to 0.5, and the probability is... Figure 2 The crack mask is obtained by quantification. The process of establishing the absolute coordinate system of the tunnel is as follows: using real-time localization and mapping algorithms, the three-dimensional point cloud collected by lidar is used as the main observation data, and the three-axis angular velocity and linear acceleration of the carrier output by the inertial measurement unit are used as motion priors. The two information are fused by extended Kalman filtering or graph optimization methods to estimate the six-degree-of-freedom pose of the carrier in the tunnel space in real time.

[0043] Specifically, the following steps are employed: First, geometric feature points, including planar points and edge points, are extracted from each frame of the LiDAR point cloud. Then, the feature points of the current frame are matched with those of the previous frame or a local map, and the relative pose transformation between frames is calculated using an iterative nearest-point algorithm. Simultaneously, the pose transformation is predicted and corrected using the pre-integration results of the inertial measurement unit. When the carrier moves to the vicinity of a pre-embedded target or a significant geometric structure (such as a segment joint) within the tunnel, closed-loop detection is performed, and the pose of all key frames is globally adjusted using a graph optimization algorithm to eliminate cumulative drift. Finally, each frame of point cloud is registered to the world coordinate system through the optimized pose transformation and aligned with the coordinate reference of the tunnel design drawings to form the absolute coordinate system of the tunnel.

[0044] Simultaneously with pose estimation, the current frame point cloud is registered to a unified world coordinate system through pose transformation, accumulating and generating or updating a pre-established 3D tunnel background model. This background model is aligned with the coordinate reference of the tunnel design drawings, forming an absolute tunnel coordinate system that can be used for subsequent projection. Through the above-mentioned synchronous triggering, deep learning segmentation, and multi-sensor fusion localization, a spatiotemporally consistent input data foundation is provided for subsequent steps.

[0045] In step S2, the initial value of the offset surrogate parameter is set to zero, indicating that before any iterative optimization, it is assumed by default that there is no coordinate offset between the image pixels and the point cloud space. This initial setting provides an unbiased starting point for subsequent iterations, allowing the optimization process to gradually search for the true offset from a zero-offset state. During the iterative optimization process, the range of values ​​for the offset surrogate parameter is not infinite, but is limited by two physical factors. The first factor is the extrinsic parameter calibration residual between the camera and the LiDAR. The extrinsic parameter calibration is a rigid transformation matrix obtained through a joint calibration program during system deployment. Due to factors such as the accuracy of the calibration target, ambient lighting, and algorithm convergence errors, this matrix has a known residual error range, which serves as the hard boundary for the offset surrogate parameter value. The second factor is the attitude change amplitude of the inspection vehicle during dynamic operation, specifically including the instantaneous fluctuations in roll angle, pitch angle, and yaw angle caused by track unevenness, acceleration / deceleration, and vibration. These fluctuations are converted into the projection offset range of image pixels in the tunnel coordinate system through a kinematic model. The offset boundary determined by the above two factors is embedded as a constraint in the solution process of the optimization objective function. During each iteration, when updating the offset surrogate parameter, it is checked whether it exceeds the boundary. If it does, it is clamped to the boundary value. This approach ensures the convergence of the optimization process while avoiding computational inefficiencies caused by an excessively large invalid search range or convergence to physically impossible parameter values.

[0046] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for three-dimensional quantization of tunnel cracks by integrating visible light and laser point clouds, characterized in that, Includes the following steps: S1: During the dynamic inspection of the tunnel lining, visible light images and laser point clouds are acquired simultaneously to establish the absolute coordinate system of the tunnel and identify crack masks in the image domain. S2: Construct offset proxy parameters, project the crack mask onto the absolute coordinate system of the tunnel, and obtain a subset of candidate 3D point clouds; S3: Extract the two-dimensional geometric constraint features of the crack mask, extract the local geometric features of the candidate three-dimensional point cloud subset, and optimize the offset proxy parameters iteratively to make the two-dimensional geometric constraint features and the local geometric features spatially consistent, thereby obtaining the optimized projection point cloud subset. S4: Fuse the three-dimensional coordinates of the optimized projection point cloud subset with the corresponding crack mask image features to construct a multimodal fusion feature vector; S5: Based on the multimodal fusion feature vector, calculate the three-dimensional spatial parameters of the crack, including the three-dimensional centerline length, width distribution and depth, and output structured data; The two-dimensional geometric constraint features include the local crack orientation vector, the parallelism constraint of the crack edge, and the image domain estimate of the crack width. The local geometric features include the local normal vector distribution of the point cloud, the local surface curvature, and the point cloud density gradient. The spatial consistency in step S3 is achieved by solving an optimization objective function. The optimization objective function is used to make the distribution of the point cloud normal vector in the projection area corresponding to the local direction vector of the image crack perpendicular to the local direction vector, and to make the curvature of the local surface of the point cloud in the projection area exhibit a concave abrupt change feature.

2. The method according to claim 1, characterized in that, The process of iteratively optimizing the offset proxy parameters in step S3 includes: The candidate 3D point cloud subset is back-projected onto the image domain to generate a virtual crack line map. The virtual crack line map is compared with the crack mask, and the comparison result is used as a constraint term of the optimization objective function to iteratively update the offset proxy parameter.

3. The method according to claim 2, characterized in that, The iterative optimization process employs a two-stage closed-loop verification: The first iteration solves for the first offset by constraining the perpendicular relationship between the local crack orientation and the point cloud normal vector, thus obtaining the first updated point cloud subset; The second iteration uses the difference map between the virtual crack line generated by the back projection of the first updated point cloud subset and the original crack mask as a new constraint term to solve for the second offset and obtain the second updated point cloud subset.

4. The method according to claim 3, characterized in that, Step S3 further includes setting an intersection-union ratio threshold, comparing the intersection-union ratio of the virtual crack line map and the crack mask with the intersection-union ratio threshold, and triggering the next iteration if it is lower than the threshold, until the intersection-union ratio reaches or exceeds the threshold and the final optimized projection point cloud subset is output.

5. The method according to claim 1, characterized in that, The multimodal fusion feature vector in step S4 is composed of the three-dimensional coordinates of each point in the projection point cloud subset, the corresponding crack mask pixel gray value, and the estimated local crack width.

6. The method according to claim 1, characterized in that, The calculation of the three-dimensional spatial parameters of the crack in step S5 specifically includes: The final optimized subset of projected point clouds is skeletonized and fitted with spline curves to obtain the 3D centerline curve, and the 3D length and direction are output. At equidistant sampling points on the center line, the crack width estimate in the multimodal fusion feature vector is combined with the point cloud density boundary in the normal plane to calculate the true width in three-dimensional space and output the width distribution curve. By utilizing the elevation difference between the bottom point of the crack and the background point cloud of the lining in the projection point cloud sub-set, and combining it with local surface fitting, the average depth and maximum depth of the crack are output.

7. The method according to claim 1, characterized in that, In step S1, the crack mask identification in the image domain adopts a pixel-level segmentation network based on deep learning. The synchronous acquisition adopts a unified timestamp trigger signal. The tunnel absolute coordinate system is established in real time through an instant positioning and map building algorithm that fuses laser point clouds and inertial measurement units.

8. The method according to claim 1, characterized in that, The initial value of the offset proxy parameter is zero, and its value range during the iterative optimization process is limited by the extrinsic parameter calibration residuals of the camera and lidar and the attitude change range of the inspection vehicle.

Images