Roadway surface and internal defect synchronous sensing method based on point cloud registration

Abstract

The invention provides a roadway surface and internal defect synchronous sensing method based on point cloud registration. The roadway surface and internal defect synchronous sensing method comprises the following steps: S1, multi-sensor joint calibration; s2, synchronously acquiring data; s3, data preprocessing; s4, projecting a camera image to the point cloud; s5, surface defect identification: S5.1, geometric defect identification based on point cloud; s5.2, performing visual defect identification based on the image, and marking an identification result in the point cloud; s6, integrating the geometric defect point cloud and the visual defect point cloud; s7, identifying internal defects; s8, determining an internal confirmation three-dimensional position; s9, marking an internal defect point cloud; and S10, uniformly visualizing the surface defects and the internal defects. According to the invention, through cooperative work of the laser radar, the ground penetrating radar and the camera, the same perception of tunnel defect surface and internal defects is realized through point cloud visualization.

Description

Technical Field

[0001] This invention relates to the field of mine roadway structural health monitoring and sensing technology, specifically a method for synchronous sensing of roadway surface and internal defects based on point cloud registration. Background Technology

[0002] As a key component of underground engineering structures, tunnels are subject to various factors such as ground pressure, water erosion, and human activities during long-term service, resulting in various types of defects. Common tunnel defects mainly include deformation, cracks, spalling, voids, and water seepage.

[0003] Currently, the main methods for detecting defects in tunnels include manual inspection and automated detection. Manual inspection relies on the experience and visual observation of inspectors, supplemented by simple tools (such as measuring tapes, crack gauges, and hammer probes) for recording and judgment. This method is intuitive and low-cost, but it is inefficient, highly susceptible to human subjectivity, difficult to quantify and assess defects, and has poor safety in harsh environments.

[0004] With the development of science and technology, more and more automated detection technologies are being applied to roadway defect detection, including: (1) Total station / 3D laser scanning, which is mainly used to measure the geometric deformation of roadways and can obtain high-precision 3D point cloud data, but it is difficult to identify surface texture and internal defects. (2) Photogrammetry / machine vision, which uses image processing algorithms to identify surface defects such as cracks and spalling by taking images or videos of the roadway surface, but is easily affected by lighting conditions and cannot obtain geometric deformation and internal information. (3) Ground penetrating radar (GPR) can detect structural anomalies in the roadway lining or surrounding rock, such as cavities, voids, and water-bearing areas, but is not sensitive to surface defects.

[0005] In recent years, to improve detection accuracy and range, some research and engineering practices have attempted to combine multiple sensors, such as combining lidar with cameras to address both geometric deformation and surface texture recognition, or combining ground-penetrating radar with video images to cross-validate surface and internal structural anomalies. However, these approaches are mostly based on "sensor stitching," lacking a unified spatial fusion mechanism. Different data are often processed independently, and defect identification results cannot be expressed in the same coordinate system, leading to information redundancy, fusion difficulties, and making it hard to form a complete, continuous, and interpretable tunnel structural health model.

[0006] Some existing technologies are based on SLAM (Simultaneous Localization and Mapping) to build multi-sensor fusion frameworks. SLAM primarily addresses the localization and mapping problems of mobile robots moving in unknown environments. SLAM allows mobile robots to build an environmental model and estimate their own motion while moving in the environment without any prior information. By coordinating visual SLAM, laser SLAM, and ground-penetrating radar, simultaneous localization and mapping of unmanned platforms can be achieved. These methods emphasize the real-time performance of sensor pose estimation and map building, which can alleviate the problem of insufficient coverage from a single sensor to some extent. However, these methods rely on the SLAM map as the core, with sensor data only used as auxiliary information. Defects are often labeled based on trajectories or sparse maps, making it difficult to achieve detailed 3D representations of structures. Furthermore, SLAM mapping is susceptible to cumulative errors, leading to pose drift and further reducing the fusion accuracy of multi-source data. The output map mainly serves navigation and localization, lacking a clear depiction of defect types, locations, and scales, making it difficult to meet the need for in-depth representation of the spatial distribution, morphology, and structural impact of defects. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a method for simultaneous perception of surface and internal defects in roadways based on point cloud registration. This method includes the following steps:

[0008] S1. Multi-sensor joint calibration: LiDAR, multiple cameras and multiple ground-penetrating radars are mounted on a mobile robot. The relative positions of the heterogeneous sensor systems of LiDAR, cameras and ground-penetrating radars are jointly calibrated to obtain the spatial transformation relationships between them and relative to the coordinate system of the mobile robot.

[0009] S2. Data Synchronization Acquisition: A unified time reference-driven synchronization triggering mechanism is adopted. The LiDAR point cloud is synchronously acquired through the sensors of LiDAR, camera, and ground penetrating radar. The camera image frame I and the ground penetrating radar record radar signal sequence form a profile. All data are accompanied by timestamps and spatial pose information.

[0010] S3. Data Preprocessing: After the sensors complete data acquisition, the mobile robot uploads the images, lidar point cloud and ground-penetrating radar data to a remote server for unified data preprocessing.

[0011] S4. Camera Image to Point Cloud Projection: Maps the color information of the camera image to the LiDAR point cloud, improving the surface texture representation of the point cloud and enabling the fusion of information between the LiDAR point cloud and the camera image;

[0012] S5. Surface Defect Identification: Surface defect identification is performed on a remote server, including:

[0013] S5.1 Step-by-step geometric defect identification based on point cloud;

[0014] S5.2 Image-based visual defect recognition, and the recognition results are marked in the point cloud;

[0015] S6. Integrate geometric and visual defect point clouds: Integrate geometric defect recognition and visual defect recognition point clouds in a remote server, and uniformly mark the recognition results of geometric and visual defects in the point cloud data structure;

[0016] S7. Internal Defect Identification: Identify internal defects of the ground penetrating radar in a remote server. Input the pre-processed ground penetrating radar profile into a deep learning model to identify internal defects and determine their corresponding lining surface points.

[0017] S8. Determine the three-dimensional location of the internal defect: Determine the three-dimensional location of the internal defect in the remote server. Calculate the depth d of the internal defect based on the time delay of the ground penetrating radar signal and the propagation speed of the radar wave, and determine its three-dimensional location P by combining the normal direction of the surface point.

[0018] S9. Mark internal defect point cloud: Mark internal defect point cloud on remote server, and add corresponding information to the point cloud structure of the identified internal defect area;

[0019] S10. Unified visualization of surface and internal defects: Perform unified visualization of surface and internal defects on a remote server, and overlay surface defect point clouds and internal defect primitives in a 3D visualization platform to form a fused visualization result under a unified coordinate system.

[0020] In some embodiments of this application, step S1 specifically involves: using a fixed target placed in the tunnel, and obtaining the extrinsic parameter matrix T between the lidar L and the camera C using a hand-eye calibration method. L→C extrinsic matrix T L→c Satisfy the following formula:

[0021]

[0022] Among them, R LC Let t represent the rotation matrix from the lidar coordinate system to the camera coordinate system. LC This represents the translation matrix of the lidar in the camera coordinate system;

[0023] By setting up calibration objects with obvious reflection characteristics and combining them with the lidar point cloud, the transformation matrix T from the lidar L to the body coordinate system of the mobile robot R is completed. L→R Calibration, transformation matrix T L→RSatisfy the following formula:

[0024]

[0025] Where, R LR Let t represent the rotation matrix from the lidar L coordinate system to the mobile robot's R coordinate system. LR This indicates the position of the lidar L in the body coordinate system of the mobile robot R;

[0026] Ground-penetrating radar (GPR) echo signals were tested using a flat plate of known material and thickness to determine the relationship between the signal propagation zero-point time t0 and depth. Simultaneously, the installation position and attitude of the GPR antenna were measured to obtain the transformation matrix T from the mobile robot's body coordinate system to the GPR coordinate system. R→G Transformation matrix T R→G Satisfy the following formula:

[0027]

[0028] Among them, R RG This represents the rotation matrix from the mobile robot's body coordinate system R to the ground-penetrating radar's coordinate system G; t RG This indicates the position of the ground-penetrating radar antenna G in the body coordinate system of the mobile robot R;

[0029] Multiple cameras simultaneously capture images using a shared target, and a multi-view geometric method is employed to calibrate camera C. i To camera C j The extrinsic transformation matrix, and the relative extrinsic matrix between multiple cameras C are T. Ci→Cj The relative extrinsic parameter matrix is ​​T Ci→Cj Satisfy the following formula:

[0030]

[0031] Where, R Cij， t Cij Let represent the rotation matrix and translation matrix from camera i to camera j, respectively;

[0032] Multiple ground-penetrating radar antennas are subjected to external parameter solving through structural measurements or synchronization signal responses to obtain the transformation matrix T of each ground-penetrating radar antenna relative to the coordinate system of the mobile robot. R→Gk Transformation matrix T R→Gk Satisfy the following formula:

[0033]

[0034] Where k represents the serial number of the ground-penetrating radar antenna G, R RGk Let t represent the rotation matrix of the k-th ground-penetrating radar antenna G relative to the mobile robot R. RGkIt represents the translation matrix of the k-th ground penetrating radar antenna G relative to the body of the mobile robot R.

[0035] In some embodiments of the present application, step S2 specifically includes: ensuring the consistency of multi-modal perception information in space and time, adopting a synchronous triggering mechanism driven by a unified time reference. When the main control system starts a task, it initializes the time synchronization module, uniformly schedules the acquisition nodes of lidar, camera and ground penetrating radar sensors. In each sampling period, each sensor collects data at the unified time. The lidar obtains the current frame of three-dimensional point cloud set P=(p1, p2,... pN), the camera image frame I, and the ground penetrating radar records the radar signal sequence and forms a cross-sectional view. All data are marked with accurate time, and the spatial pose of the ground penetrating radar data is obtained in combination with the positioning module, and they are uniformly cached for subsequent processing.

[0036] In some embodiments of the present application, step S4 specifically includes: expanding the point pi=[Xi, Yi, Zi] in each point cloud of the lidar L to homogeneous coordinates Pi,homog,L=[Xi, Yi, Zi, 1] T , and using the external parameter matrix T L→C between the lidar and the camera to transform it to the camera coordinate system, obtaining Pi,homog,C = T L→C ·Pi,homog,L, and then extracting the transformed three-dimensional coordinates as [Xi’, Yi’, Zi’] T . If Zi’ ≤ 0, it means the point is behind the camera or coplanar, and this point is discarded; if Zi’ > 0, use the camera internal parameter matrix K to project it onto the image plane, and then obtain the pixel coordinates (u i , v i ). The formula for the camera internal parameter matrix K to project it onto the image plane is as follows:

[0037]

[0038] where S = Zi’ is the depth factor; judge whether the pixel coordinates (u i , v i ) fall within the image range, that is, satisfy 0 < u i < W, and 0 < v i < H; if it is within the range, obtain the color information of the corresponding pixel from the image, and assign this color to the corresponding point Pi in the point cloud, and expand the point cloud structure to include texture attributes such as RGB or grayscale values; if the pixel coordinates exceed the image range or cannot be effectively registered, retain the original point cloud information or mark it as a textureless value.

[0039]

[0040] In some embodiments of the present application, step S5.1 geometric defect identification includes:​

[0041] Step S5.1.1: Surface deformation detection. Register the colored point cloud obtained in step S4 with the reference point cloud. Use a high-precision 3D point cloud registration algorithm to transform the currently scanned point cloud to the same coordinate system as the reference point cloud. The reference point cloud is obtained by scanning under the condition of the roadway health status. Calculate the distance between point clouds. For each point in the current point cloud, find its nearest neighbor in the reference point cloud and calculate the distance from the current point to its nearest neighbor in the reference point cloud. Identify the deformation area and set a deformation threshold δ.

[0042] If the distance of a point exceeds the deformation threshold δ, the point is considered to belong to the deformation region. The deformation threshold δ is determined by historical data or experience. By performing threshold segmentation on the distance map and performing connected component analysis, continuous points exceeding the threshold are clustered to identify specific deformation regions. The positive or negative value of the distance represents deformation convergence or bulging. The point cloud is labeled, and corresponding information is added to the point cloud structure of the identified deformation region.

[0043] Step S5.1.2: Identification of large-scale surface spalling and voids. Based on surface normal and curvature analysis, identification of large-scale surface spalling and voids can be achieved; details are as follows:

[0044] Estimate the surface normal of the point cloud. For each point in the colored point cloud obtained in step S4, perform principal component analysis in its local neighborhood. The eigenvector corresponding to the minimum eigenvalue of the principal component analysis is the normal direction of the surface of the neighborhood of that point.

[0045] Calculate the curvature of the point cloud, and simultaneously calculate the curvature information of the local surface from the principal component analysis results;

[0046] Abnormal regions are identified based on normals and curvature. Thresholds are set, and regions where the normal direction changes significantly from the normal direction of surrounding points, or regions where the curvature value exceeds a certain threshold, are identified as the edges of peeling or voids. Alternatively, multiple features can be combined for judgment.

[0047] Clustering is used to identify spalling and void regions. The identified outliers are clustered and analyzed to identify a spatially continuous set of outliers as an independent spalling or void defect. A subset of point clouds labeled with spalling and void regions is obtained, and the corresponding information is added to the point cloud structure of the identified large spalling and void regions.

[0048] In some embodiments of this application, step S5.2 visual defect identification includes the identification of surface cracks, small peelings, and water seepage; specifically as follows:

[0049] Step S5.2.1. Point cloud surface unfolding to generate a 2D image: Fit the center point of the cross-section along the tunnel direction to determine an approximate tunnel central axis. For each point in the colored point cloud, calculate its angle θ relative to the central axis and its height h along the axis. Use (θ,h) as the pixel coordinates of the 2D image. Based on the calculated (θ,h) coordinates, assign the color value of the corresponding 3D point to the pixel of the 2D image. Use interpolation methods to fill the holes generated by the unfolding of sparse point clouds to generate a complete 2D unfolded image. The 2D unfolded image maps the spatially aligned defect recognition results back to the 3D structure, realizing expression and analysis under a unified coordinate system.

[0050] Step S5.2.2. Image segmentation and recognition based on deep learning: Input the unfolded image from step S5.2.1 into the trained deep learning model. The model will process the image and output a segmentation mask of the same size as the original image, where different pixel values ​​represent different defect types or backgrounds.

[0051] Step S5.2.3. Post-processing: Post-process the segmentation mask output by the model to remove weak isolated noise regions, perform morphological operations to smooth the segmentation results, and connect continuous regions of the same category to form complete defect regions.

[0052] Step S5.2.4: Inverse unfolding mapping. For each pixel in the two-dimensional defect region identified in step S5.2.3, find the original three-dimensional point projected onto the pixel during the unfolding process in step S5.2.1.

[0053] Mark the point cloud by identifying the original 3D points found in the point cloud data as belonging to the corresponding visual defect type.

[0054] In some embodiments of this application, the specific process of S7 is as follows: the preprocessed radar profile image is input into the trained deep learning model, the trained deep learning model analyzes the input radar profile image, identifies specific patterns that represent internal defects, and the model outputs a list containing the identified defects, each defect having its bounding box on the radar profile image, defect type, and estimated size.

[0055] The location of the defect on the horizontal axis of the radar profile is determined, along with the corresponding bottom-penetrating radar trajectory. Combined with the time delay t on the vertical axis, the detection time of the defect is pinpointed. The positioning system provides the pose T of the mobile robot in the three-dimensional coordinate system of the tunnel. R→w (t), combined with the transformation relationship T between the ground-penetrating radar antenna and the mobile robot body G→R The global pose T of the ground-penetrating radar antenna at time t is calculated. G→T (t); T G→T (t) satisfies the following formula:

[0056] T G→T (t)=T R→T (t)·T G→R

[0057] After detecting an internal defect at time t, the corresponding tunnel lining surface point is inferred from the three-dimensional position and orientation of the antenna. The nearest point P0 is searched in the colored point cloud near the current position of the ground penetrating radar antenna as the surface mapping position of the defect reflection signal, realizing the spatial association between the internal defect and the surface structure. The normal direction of the lining surface point is determined, and the surface normal n is calculated in the local neighborhood of P in the colored point cloud.

[0058] In some embodiments of this application, step S8 specifically involves: based on the ground-penetrating radar antenna pose T... G→T Using (t) and color point cloud data, determine the lining surface point P0 corresponding to the internal defect and its normal direction n, and calculate the three-dimensional position P of the internal defect, where P satisfies the following formula:

[0059] P = P0 + n·d

[0060] Where d is the offset depth, d = v·(t-t0) / 2, v is the radar wave propagation speed, t is the time delay, and t0 is the zero point time.

[0061] In some embodiments of this application, the specific process in step S10 is as follows: based on the surface defect coloring point cloud data obtained in step S6, each point in the point cloud contains its three-dimensional coordinate color value and surface defect type label; based on the internal defect information obtained in step S9, including defect type, estimated location and size parameters, and stored in a structured manner.

[0062] Load the surface defect point cloud into the selected 3D visualization platform or library;

[0063] The display style is set according to the defect label, and different colors or highlighting effects are assigned to different types of surface defects to enhance recognizability;

[0064] The internal defect data is traversed, and appropriate geometric primitives are selected for visualization mapping according to the defect type. For example, spheres or cubes are used to approximate the model for cavities or voids, and semi-transparent volume blocks are used to represent water-bearing areas. All primitives are translated and positioned with the estimated 3D coordinates of the corresponding defect as the center, and appropriate transparency is set to ensure the mutual visibility of primitives and background point cloud structures.

[0065] By overlaying internal defect primitives onto surface point cloud data, a three-dimensional fusion visualization result is formed under a unified coordinate system, enabling the synchronous display of surface and internal defects in the tunnel structure, and providing a basis for defect assessment and repair strategy formulation.

[0066] This innovative method achieves simultaneous identification and three-dimensional localization of surface and internal defects in roadways. It unifies and overlays visual defects such as surface cracks, spalling, and water seepage identified by cameras, defects such as deformation, large spalling, and void areas identified by lidar, and internal defects identified by ground-penetrating radar into a point cloud model, forming an intuitive, continuous, and interactive digital roadway structural health model. The model clearly presents the type, spatial location, morphological characteristics, and interrelationships of defects using a unified three-dimensional coordinate system. This facilitates engineers in quickly assessing the severity and distribution patterns of defects, providing an intuitive and reliable reference for formulating subsequent repair strategies. It significantly improves the efficiency and targeted nature of roadway maintenance, laying a solid foundation for predictive maintenance and intelligent operation and maintenance. Attached Figure Description

[0067] Figure 1 This application provides a flowchart of a method for synchronously sensing surface and internal defects in roadways based on point cloud registration;

[0068] Figure 2 This is a flowchart for surface defect identification. Detailed Implementation

[0069] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0070] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0071] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0072] Please see Figure 1 , Figure 1 This application provides a flowchart of a method for simultaneous perception of surface and internal defects in roadways based on point cloud registration. The method includes the following:

[0073] Step S1: The mobile robot is equipped with a LiDAR, a camera, and a ground-penetrating radar. The LiDAR is used to acquire point cloud data of the tunnel, the camera is used to acquire image data of the tunnel, and the ground-penetrating radar is used to acquire radar waveform data.

[0074] Multi-sensor joint calibration is necessary to achieve unified registration and fusion analysis of multi-sensor data. This requires joint calibration of multiple heterogeneous sensor systems, such as LiDAR, cameras, and ground-penetrating radar (GPR), to obtain their spatial transformation relationships and relative to the coordinate system of the mobile robot. The specific calibration is as follows:

[0075] First, a fixed target is deployed in the tunnel, and the extrinsic parameter matrix T between the lidar L and the camera C is obtained using the hand-eye calibration method. L→C extrinsic matrix T L→c Satisfy the following formula:

[0076]

[0077] Among them, R LC R,t represents the rotation matrix R from the lidar coordinate system to the camera coordinate system. LC The translation matrix t represents the lidar in the camera coordinate system;

[0078] By setting up calibration objects with obvious reflection characteristics and combining them with the lidar point cloud, the transformation matrix T from the lidar L to the body coordinate system of the mobile robot R is completed. L→R Calibration, transformation matrix T L→R Satisfy the following formula:

[0079]

[0080] Where, R LR R,t represents the rotation matrix R from the lidar L to the mobile robot's R coordinate system. LR Let t represent the translation matrix of the lidar L in the body coordinate system of the mobile robot R.

[0081] It should be noted that the large index R in the matrix represents the rotation matrix, and the subscript R represents the mobile robot, T L→R In this context, R represents a mobile robot.

[0082] Ground-penetrating radar (GPR) echo signals were tested using a flat plate of known material and thickness to determine the relationship between the signal propagation zero-point time t0 and depth. Simultaneously, the installation position and attitude of the GPR antenna were measured to obtain the transformation matrix T from the mobile robot's body coordinate system to the GPR coordinate system. R→G Transformation matrix T R→G Satisfy the following formula:

[0083]

[0084] Among them, R RG This represents the rotation matrix from the mobile robot's body coordinate system R to the ground-penetrating radar's coordinate system G; t RG This indicates the position of the ground-penetrating radar antenna G in the body coordinate system of the mobile robot R;

[0085] It should be noted that the large index R in the matrix represents the rotation matrix, and the subscript R represents the mobile robot, T R→G In this context, R represents a mobile robot.

[0086] Multiple cameras simultaneously capture images using a shared target, and a multi-view geometric method is employed to calibrate camera C. i To camera C j The extrinsic transformation matrix, and the relative extrinsic matrix between multiple cameras C are T. Ci→Cj The relative extrinsic parameter matrix is ​​T Ci→Cj Satisfy the following formula:

[0087]

[0088] Where, R Cij， t Cij Let represent the rotation matrix and translation matrix from camera i to camera j, respectively;

[0089] Multiple ground-penetrating radar antennas are subjected to external parameter solving through structural measurements or synchronization signal responses to obtain the transformation matrix T of each ground-penetrating radar antenna relative to the coordinate system of the mobile robot. R→Gk Transformation matrix T R→Gk Satisfy the following formula:

[0090]

[0091] Where k represents the serial number of the ground-penetrating radar antenna G, R RGk Let t represent the rotation matrix of the k-th ground-penetrating radar antenna G relative to the mobile robot R. RGk Let G represent the translation matrix of the k-th ground-penetrating radar antenna G relative to the mobile robot R.

[0092] It should be noted that the large index R in the matrix represents the rotation matrix, and the subscript R represents the mobile robot, T R→Gk In this context, R represents a mobile robot, and Gk represents the k-th ground-penetrating radar antenna.

[0093] Step S2: Data Synchronization Acquisition. To ensure the consistency of multimodal sensing data in time and space, the system adopts a synchronization triggering mechanism based on a unified time reference. When starting the detection task, the main control system initializes the synchronization module, simultaneously triggering the acquisition nodes of various sensors such as LiDAR, camera, and GPR with the time reference signal to achieve periodic synchronous sampling.

[0094] The lidar acquires point cloud data of the current frame and adds a timestamp to obtain a three-dimensional point set P = (p1, p2, ..., pN) in the lidar coordinate system, where each point pi represents the three-dimensional coordinates [Xi, Yi, Zi]. L The camera acquires the current frame image I, which is represented as a two-dimensional pixel matrix of size W (width) x H (height), with each pixel containing color or grayscale information; GPR acquires the original radar signal sequence (A-Scan) along the movement path of the mobile robot and arranges them in order to form a two-dimensional radar profile (B-Scan).

[0095] All sensor data is appended with a precise timestamp after acquisition, and the spatial position and attitude information of the corresponding sampling point are recorded by the positioning module. This data is then sent to the corresponding buffer queue in frames for subsequent registration, fusion, and defect analysis modules to access.

[0096] Step S3: Data Preprocessing. After data acquisition, the mobile robot uploads images, LiDAR point clouds, and ground-penetrating radar data to the server via a communication module (such as an industrial Ethernet, 5G module, or WiFi unit) to perform a unified data preprocessing process, thereby improving the accuracy and robustness of subsequent defect identification and visualization analysis.

[0097] Step S3.1: In terms of image preprocessing, distortion correction is performed based on the intrinsic parameters of each camera (including focal length, principal point, and distortion coefficients) to restore the image geometry. For image sequences acquired by multiple cameras, relative pose is estimated through feature extraction and matching algorithms, and registration and stitching methods are used to construct a panoramic image of the alleyway, achieving spatial fusion of visual information. The processed camera images are then obtained.

[0098] Step S3.2: For the preprocessing of LiDAR point cloud data, firstly, noise filtering algorithms such as statistical filtering and radius filtering are used to remove outliers. Then, point cloud segmentation is performed, such as region growing segmentation methods and clustering algorithms, to segment the point cloud into areas like tunnel lining, ground, and equipment. Only the tunnel lining area is considered for defect identification, reducing computational load and improving processing efficiency. The processed LiDAR point cloud is obtained.

[0099] Step S3.3: In terms of ground penetrating radar data preprocessing, first construct a B-Scan image according to the sampling order, and successively perform operations such as zero-point correction, band-pass filtering, time-varying gain compensation, and background removal, and finally convert it into a standard grayscale image. Multiple GPR profiles are geometrically aligned and stitched based on the pose information of the mobile robot and the equipment installation parameters to construct a panoramic underground structure with spatial continuity, enhancing the recognition ability of concealed components, cavities, and structural anomalies. Obtain the processed ground penetrating radar profile.

[0100] Step S4: Project the camera image onto the point cloud through the remote server. To achieve the information fusion of lidar point cloud and camera image and enhance the surface texture expression ability of the point cloud, the camera image needs to be mapped to the lidar point cloud. Specifically as follows:

[0101] For a frame of point cloud obtained within a processed lidar scan cycle in Step S3.2, select the camera image frame with the closest processed timestamp obtained in Step S3.1 as the mapping image. For any point pi = [Xi, Yi, Zi] in the point cloud L , expand it into homogeneous coordinate form Pi,homog,L = [Xi, Yi, Zi, 1] T , and map it to the camera coordinate system through the extrinsic transformation matrix T L→C from lidar to camera, obtaining Pi,homog,C = T L→C · Pi,homog,L, and then extract the transformed three-dimensional coordinates as [Xi’, Yi’, Zi’] T .

[0102] If Zi’ ≤ 0, that is, the point is behind the camera or coplanar, discard the point; otherwise, project the three-dimensional coordinates to the image plane using the camera intrinsic matrix K, and the formula is:

[0103]

[0104] where S = Zi’ is the depth factor. Judge whether the pixel coordinates (u i , v i ) fall within the image range, that is, satisfy 0 < u i < W, and 0 < v i < H. If within the range, obtain the color information of the corresponding pixel from the image, and assign this color to the corresponding point Pi in the point cloud, expanding the point cloud structure to include texture attributes such as RGB or grayscale values. If the pixel coordinates exceed the image range or cannot be effectively registered, retain the original point cloud information or mark it as a textureless value.

[0105] Through the above projection and mapping process, image enhancement of the point cloud data is achieved, providing basic support for subsequent surface defect detection, texture analysis, defect three-dimensional positioning, and multi-modal perception.

[0106] Please see Figure 2 , Figure 2 The flowchart for surface defect identification is as follows: Step S5: Surface defect identification includes step S5.1 geometric defect identification (point cloud) and step S5.2 visual defect identification (two-dimensional image).

[0107] Step S5.1: Geometric defect identification includes step S5.1.1 surface deformation detection and step S5.1.2 identification of large surface spalling and voids.

[0108] Step S5.1.1: Surface Deformation Detection. Surface deformation detection can be achieved by comparing it with the baseline point cloud scanned in a healthy state of the tunnel. Specifically:

[0109] Step S5.1.1.1: Register the colored point cloud obtained in step S4 with the reference point cloud. Use a high-precision 3D point cloud registration algorithm to transform the currently scanned point cloud to the same coordinate system as the reference point cloud, which is obtained by scanning under the condition of the roadway health status.

[0110] Step S5.1.1.2: Calculate the distance between point clouds. For each point in the current point cloud, find its nearest neighbor in the reference point cloud and calculate the distance from the current point to its nearest neighbor in the reference point cloud.

[0111] Step S5.1.1.3: Identify deformable regions and set a deformation threshold δ. If the distance to a point exceeds δ, the point is considered to potentially belong to a deformable region. The deformation threshold δ is determined by historical data or experience. Specific deformable regions can be identified by thresholding the distance map and performing connected component analysis to cluster consecutive points exceeding the threshold. The sign of the distance indicates whether the deformation is converging or bulging.

[0112] Step S5.1.1.4: Mark the point cloud. Add relevant information to the point cloud structure of the identified deformed region.

[0113] Step S5.1.2: Identification of large-scale surface spalling and voids. Identification of large-scale surface spalling and voids can be achieved based on surface normal and curvature analysis. Specifically:

[0114] Step S5.1.2.1: Estimate the surface normal of the point cloud. For each point in the colored point cloud obtained in step S4, perform principal component analysis (PCA) in its local neighborhood. The eigenvector corresponding to the smallest eigenvalue of the principal component analysis is the normal direction of the surface of the neighborhood of that point.

[0115] Step S5.1.2.2: Calculate the curvature of the point cloud, and at the same time calculate the curvature information of the local surface from the PCA results.

[0116] Step S5.1.2.3: Identify abnormal regions based on normals and curvature, and set thresholds. Regions where the normal direction changes significantly from the normal direction of surrounding points, or regions where the curvature value exceeds a certain threshold, may be the edges of peeling or voids. Multiple features can also be combined for judgment. For example, find regions where the normal changes significantly and the point cloud density suddenly decreases. The threshold is determined by historical data or experience.

[0117] Step S5.1.2.4: Cluster identification of spalling / void regions. The identified outliers are clustered to identify spatially continuous sets of outliers as independent spalling or void defects. Finally, a subset of point clouds labeled with spalling and void regions is obtained.

[0118] Step S5.1.2.5: Mark the point cloud. Add relevant information to the point cloud structure of the identified large-scale erosion and void areas.

[0119] Step S5.2: Visual defect identification includes the identification of surface cracks, small peelings, and water seepage.

[0120] Deep learning-based image segmentation and recognition can identify surface cracks, small peelings, and water seepage. Specifically:

[0121] Step S5.2.1: Unfold the point cloud surface to generate a 2D image. Fit the center point of the cross-section along the tunnel direction to determine an approximate tunnel central axis. For each point in the colored point cloud, calculate its degree θ relative to the central axis and its height h along the axis. Use (θ, h) as the pixel coordinates of the 2D image. Based on the calculated (θ, h) coordinates, assign the color value of the corresponding 3D point to the pixel of the 2D image. Use interpolation methods to fill the holes generated by the unfolding of the sparse part of the point cloud to generate a complete 2D unfolded image. The 2D unfolded image can map the spatially aligned defect recognition results back to the 3D structure, realizing expression and analysis under a unified coordinate system.

[0122] Step S5.2.2: Input the unfolded image from step S5.2.1 into the trained deep learning model (such as U-Net, Ma step SkR-CNN). The model will process the image and output one or more segmentation masks of the same size as the original image, where different pixel values ​​represent different defect types or backgrounds. The trained deep learning model uses the unfolded images generated by the labeled colored point clouds of surface cracks, small peelings, and water seepage for identification.

[0123] Step S5.2.3: Post-process the segmentation mask output by the model to remove small isolated noise regions, perform morphological operations to make the segmentation results smoother, and connect continuous regions of the same category to form complete defect regions.

[0124] Step S5.2.4: Inverse unfolding mapping. For each pixel in the two-dimensional defect region identified in step S5.2.3, find the original three-dimensional point projected onto the pixel during the unfolding process in step S5.2.1.

[0125] Step S5.2.4: Mark the point cloud. Mark the original 3D points found in the point cloud data as belonging to the corresponding visual defect type.

[0126] Step S6: Integrate the point clouds of geometric and visual defect identification. The point clouds of deformed, large-scale spalling, and void areas identified by geometric defects, and the point clouds of cracks, small-scale spalling, and water seepage areas identified by visual defects, are uniformly labeled in the corresponding point cloud data structure. A single point may belong to multiple defect types simultaneously.

[0127] Step S7: Internal Defect Identification. For ground-penetrating radar internal defect identification, the pre-processed radar profile image is input into a trained deep learning model. The trained deep learning model analyzes the input radar profile image, identifies specific patterns representing internal defects, and outputs a list containing the identified defects. Each defect has its bounding box on the radar profile image, defect type, and estimated size.

[0128] Identify the lining surface point corresponding to the defect. Determine the corresponding GPR trajectory position based on the defect's location on the horizontal axis of the radar profile, and combine this with the time delay t on the vertical axis to pinpoint the detection time of the defect occurrence. The positioning system provides the pose T of the mobile robot in the three-dimensional coordinate system of the tunnel. R→w (t), combined with the transformation relationship T of the GPR antenna relative to the mobile robot body G→R The global pose T of the GPR antenna at time t is calculated. G→T (t)=T R→T (t)·T G→R .

[0129] After an internal defect is detected at time t, the corresponding point on the tunnel lining surface is deduced based on the antenna's three-dimensional position and orientation. The nearest point P0 is searched in the color point cloud near the current position of the GPR antenna and used as the surface mapping position of the defect reflection signal, thus realizing the spatial correlation between the internal defect and the surface structure.

[0130] Determine the normal direction of the points on the lining surface, and calculate the surface normal n in the local neighborhood of P in the colored point cloud.

[0131] Step S8: Determine the three-dimensional location of the internal defect. The three-dimensional location P of the internal defect is obtained by offsetting its corresponding lining surface point P0 along the inward-pointing normal direction n by a depth d, as shown in the following formula:

[0132] P = P0 + n·d

[0133] Where d is the offset depth, d = v·(t-t0) / 2, v is the radar wave propagation speed, t is the time delay, and t0 is the zero point time.

[0134] Step S9: Mark the internal defect point cloud. Add relevant information to the point cloud structure of the identified internal defect region.

[0135] Step S10: Unified visualization of surface and internal defects. First, based on the surface defect colored point cloud data obtained in step S6, each point in the point cloud contains its three-dimensional coordinates (X,Y,Z), color values ​​(R,G,B), and a surface defect type label.

[0136] Furthermore, the internal defect information obtained in step S9 includes defect type (such as void, void, water content, etc.), estimated location (X,Y,Z) and size parameters (such as radius, length, width and height, etc.), and is stored in a structured manner.

[0137] Load the surface defect point cloud into the selected 3D visualization platform or library (such as Open3D, PCL, or VTK). Set the display style according to the defect label, and assign different colors or highlight effects to different types of surface defects. For example, use red to color cracks to enhance their recognizability.

[0138] The internal defect data is traversed, and appropriate geometric primitives are selected for visualization mapping according to the defect type: spheres or cubes are used to approximate the modeling of voids or cavities, and semi-transparent volume blocks are used to represent water-bearing areas; all primitives are translated and positioned with the estimated 3D coordinates of the corresponding defect as the center, and appropriate transparency is set to ensure the mutual visibility of primitives and background point cloud structures.

[0139] Finally, by overlaying the aforementioned internal defect primitives onto the surface point cloud data, a three-dimensional fusion visualization result is formed under a unified coordinate system, realizing the synchronous display of surface and internal defects of the tunnel structure, and providing an intuitive and accurate reference for subsequent defect assessment and repair strategy formulation.

[0140] This application provides a method for synchronously sensing tunnel defects by fusing camera images and ground-penetrating radar (GPR) data, using LiDAR point clouds as a unified spatial reference framework. This overcomes the limitations of traditional manual inspection methods, which are heavily reliant on manual inspections, highly subjective, and difficult to quantify comprehensively. It also overcomes the shortcomings of existing sensor stitching schemes, such as low fusion levels and isolated data. Compared to SLAM-based multi-sensor fusion technology, this invention does not rely on real-time mapping and pose inference. Through high-precision extrinsic parameter calibration and spatial registration, it achieves seamless integration of multimodal data in a unified point cloud coordinate system. This avoids registration drift caused by accumulated errors in synchronous positioning and mapping methods, ensuring the spatial consistency and stability of defect identification and localization.

[0141] This innovative method achieves simultaneous identification and three-dimensional localization of surface and internal defects in roadways. It unifies and overlays visual defects such as surface cracks, spalling, and water seepage identified by cameras, defects such as deformation, large spalling, and void areas identified by lidar, and internal defects identified by ground-penetrating radar into a point cloud model, forming an intuitive, continuous, and interactive digital roadway structural health model. The model clearly presents the type, spatial location, morphological characteristics, and interrelationships of defects using a unified three-dimensional coordinate system. This facilitates engineers in quickly assessing the severity and distribution patterns of defects, providing an intuitive and reliable reference for formulating subsequent repair strategies. It significantly improves the efficiency and targeted nature of roadway maintenance, laying a solid foundation for predictive maintenance and intelligent operation and maintenance.

[0142] The present invention and its embodiments have been described above. This description is not restrictive, and the accompanying drawings are only one embodiment of the present invention; the actual structure is not limited thereto. In conclusion, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the invention, such designs should fall within the protection scope of the present invention.

Claims

1. A method for synchronously perceiving a roadway surface and internal defects based on point cloud registration, characterized in that, Comprise: S1. Multi-sensor joint calibration: A laser radar, multiple cameras and multiple ground penetrating radars are carried on a mobile robot, the relative positions of the heterogeneous sensor system of the laser radar, camera and ground penetrating radar are jointly calibrated, and the spatial transformation relationship between each other and relative to the mobile robot coordinate system is obtained; S2. Data synchronous acquisition: a synchronous trigger mechanism driven by a unified time reference is adopted, laser radar point cloud sets, camera image frames I, and ground penetrating radar recorded radar signal sequences are synchronously collected by the three sensors, and profile graphs are formed, all data are attached with time stamps and spatial pose information; S3. Data preprocessing: after the sensor completes data acquisition, the mobile robot uploads the image, laser radar point cloud and ground penetrating radar data to the remote server, and uniformly executes data preprocessing operation; S4. Camera image to point cloud projection: the color information of the camera image is mapped to the laser radar point cloud, the surface texture expression ability of the point cloud is improved, and the information fusion of the laser radar point cloud and the camera image is realized; S5. Surface defect identification: surface defect identification is carried out in the remote server, including: S5.1 Step geometry defect identification based on point cloud; S5.2 Visual defect identification based on image, and marking the identification result in the point cloud; S6. Integrate geometry defect and visual defect point cloud: integrate geometry defect identification and visual defect identification point cloud in the remote server, and mark the identification results of geometry defect and visual defect in the point cloud data structure; S7. Internal defect identification: internal defect identification is carried out in the remote server, the preprocessed ground penetrating radar profile graph is input into the deep learning model, the internal defect is identified, and the corresponding lining surface point is determined; S8. Determine the three-dimensional position of internal confirmation: determine the three-dimensional position of internal defect in the remote server, calculate the depth d of internal defect according to the time delay of ground penetrating radar signal and the speed of radar wave propagation, and determine its three-dimensional position P combined with the normal direction of surface point; S9. Mark internal defect point cloud: mark internal defect point cloud in the remote server, add corresponding information in the point cloud structure of the identified internal defect area part; S10. Surface defect and internal defect unified visualization: surface defect and internal defect unified visualization is carried out in the remote server, surface defect point cloud and internal defect primitive are superimposed and displayed in the three-dimensional visualization platform, and fusion visualization result in unified coordinate system is formed.

2. The method for synchronously perceiving the roadway surface and internal defects based on point cloud registration according to claim 1, characterized in that, A specific process of the step S1 is as follows: a fixed target is arranged in a tunnel site, and an extrinsic parameter matrix T between the laser radar L and the camera C is obtained through a hand-eye calibration method L→C The extrinsic parameter matrix T L→c satisfies the following formula: wherein R LC represents a rotation matrix of the LiDAR coordinate system to the camera coordinate system, t LC represents a translation matrix of the LiDAR in the camera coordinate system; By setting the calibration object with obvious reflection characteristics, combined with the laser radar point cloud, the transformation matrix T of the laser radar L to the movable robot R body coordinate system is completed L→R Calibration, transformation matrix T L→R Satisfies the following formula: wherein R LR represents a rotation matrix of the laser radar L to the movable robot R coordinate system, t LR represents the position of the laser radar L in the movable robot R body coordinate system; The ground penetrating radar selects a flat plate with known material and thickness to test the echo signal, and determines the signal propagation zero time t0 and the depth conversion relationship; meanwhile, the installation position and attitude of the ground penetrating radar antenna are measured, and the transformation matrix T of the movable robot body coordinate system to the ground penetrating radar coordinate system is obtained R→G The transformation matrix T R→G satisfies the following formula: wherein R RG represents a rotation matrix of the mobile robot R body frame to the ground penetrating radar G frame; t RG represents a translation matrix of the ground penetrating radar antenna G in the mobile robot R body frame; The multiple cameras are synchronously photographed through the common view target, and a multi-view geometry method is used to calibrate the camera C i The extrinsic transformation matrix of the camera C j is T Ci→Cj The relative extrinsic matrix between the multiple cameras C is T Ci→Cj The relative extrinsic matrix satisfies the following formula: wherein, R Cij， t Cij Ri,jand ti,jdenote the rotation and translation matrices from camera i to camera j, respectively; The plurality of ground penetrating radar antennas solve the external parameter by structure measurement or synchronous signal response, and obtain a transformation matrix T of each ground penetrating radar antenna relative to the movable robot body coordinate system R→Gk The transformation matrix T R→Gk satisfies the following formula: wherein k denotes the index of the ground penetrating radar antenna G, R RGk represents the rotation matrix of the k-th ground penetrating radar antenna G with respect to the mobile robot R body, t RGk represents the translation matrix of the k-th ground penetrating radar antenna G with respect to the mobile robot R body.

3. The method for synchronously perceiving the surface and internal defects of a roadway based on point cloud registration according to claim 1, characterized in that, The step S2 specifically comprises: ensuring the consistency of multi-modal perception information in space and time, adopting a synchronous triggering mechanism driven by a unified time reference, initializing a time synchronization module by the master control system at task start, uniformly scheduling the collection nodes of the laser radar, camera and ground penetrating radar sensors, in each sampling period, each sensor collects data according to the unified time, the laser radar obtains the current frame three-dimensional point cloud set P=(p1, p2,...pN), the camera image frame I, and the ground penetrating radar records the radar signal sequence and constitutes a profile, all data are marked with accurate time, and the spatial pose of the ground penetrating radar data is obtained in combination with the positioning module, and the data are uniformly cached for subsequent processing.

4. The method for synchronously perceiving the surface and internal defects of a roadway based on point cloud registration according to claim 1, characterized in that, The step S4 specifically comprises: the laser radar Pi = [Xi, Yi, Zi] in each point cloud L Extended to homogeneous coordinates Pi, homog, L = [Xi, Yi, Zi, 1] T , using the extrinsic matrix T between the LiDAR and the camera L→C Transform to the camera coordinate system, get Pi, homog, C = T L→C · Pi, homog, L, and then extract the transformed three-dimensional coordinates as [Xi', Yi', Zi'] T , if Zi'≤0, the point is located behind the camera or coplanar, then discard the point; if Zi'>0, use the camera intrinsic matrix K to project it to the image plane, and then obtain the pixel coordinates (u i , v i ), the camera intrinsic matrix K projects it to the image plane formula as follows: where S = Zi' is the depth factor; determine whether the pixel coordinates (u i ,v i ) fall within the image range, i.e. satisfy 0<u i <W, and 0<v i <H; if within the range, obtain the color information of the corresponding pixel from the image and assign the color to the corresponding point Pi in the point cloud, and expand the point cloud structure to include texture attributes such as RGB or grayscale values; if the pixel coordinates are out of the image range or cannot be effectively registered, keep the original point cloud information or mark it as a non-texture value.

5. The method for registration of point cloud based tunnel surface and internal defect synchronous perception according to claim 1, characterized in that, The step S5.1 geometric defect identification comprises: Step S5.1.1: surface deformation detection, the colored point cloud obtained in step S4 is registered with the reference point cloud, a high-precision three-dimensional point cloud registration algorithm is used to transform the current scanned point cloud to the same coordinate system as the reference point cloud, which is scanned under the healthy condition of the roadway; the distance between the point clouds is calculated, for each point in the current point cloud, the nearest neighbor point in the reference point cloud is found to calculate the distance from the current point to the nearest neighbor point in the reference point cloud; a deformation threshold δ is set; If the distance of a point exceeds the deformation threshold δ, it is considered that the point belongs to the deformation area, and the deformation threshold δ is determined by historical data or experience; the distance graph is threshold segmented, and connected region analysis is performed to cluster the continuous points exceeding the threshold, and the specific deformation area is identified; the positive and negative of the distance represent deformation convergence or bulging; the point cloud is marked, and the corresponding information is added to the point cloud structure in the identified deformation area part; Step S5.1.2: surface large-scale spalling and cavity identification, based on surface normal and curvature analysis, surface large-scale spalling and cavity identification can be realized; specifically as follows: Estimating the surface normal of the point cloud, for each point in the colored point cloud obtained in step S4, principal component analysis is performed in its local neighborhood, and the feature vector corresponding to the minimum eigenvalue of the principal component analysis is the normal direction of the surface of the point neighborhood; Calculate the curvature of the point cloud, and calculate the curvature information of the local surface from the principal component analysis result; Identify abnormal areas based on the normal and curvature, set a threshold, the area where the normal direction changes greatly from the normal direction of the surrounding points, or the area where the curvature value exceeds a certain threshold, is the edge of spalling or cavity, or, multiple features are combined for judgment; Cluster to identify spalling and cavity areas, cluster analysis is performed on the identified abnormal points, and a continuous abnormal point set in space is identified as an independent spalling or cavity defect; obtain the point cloud subset marked with the spalling and cavity area, and add the corresponding information to the point cloud structure in the identified large-scale spalling and cavity identification area part.

6. The method for registration of point cloud based tunnel surface and internal defect synchronous perception according to claim 1, characterized in that, The step S5.2 visual defect identification comprises identification of surface cracks, small-scale spalling and water seepage; specifically as follows: Step S5.2.

1. Point cloud surface unfolding to generate a two-dimensional image: fitting the cross-section center points along the tunnel direction to determine an approximate tunnel center axis, for each point in the colored point cloud, calculating its angle θ relative to the center axis and height h along the axis; taking (θ, h) as the pixel coordinates of the two-dimensional image, assigning the color value of the corresponding three-dimensional point to the pixel of the two-dimensional image according to the calculated (θ, h) coordinates, filling the holes generated by the sparse part of the point cloud unfolding using an interpolation method to generate a complete two-dimensional unfolding image; the two-dimensional unfolding image maps the spatially aligned defect recognition result back to the three-dimensional structure, realizing expression and analysis in a unified coordinate system; Step S5.2.

2. Image segmentation recognition based on deep learning: input the unfolding image of step S5.2.1 into the trained deep learning model, the model will process the image and output a segmentation mask with the same size as the original image, where different pixel values represent different defect types or backgrounds; Step S5.2.

3. Post-processing: post-processing the segmentation mask output by the model to remove weak isolated noise regions, performing morphological operations to smooth the segmentation result, and connecting continuous regions of the same class to form complete defect regions; Step S5.2.4: inverse unfolding mapping, for each pixel in the two-dimensional defect region identified in step S5.2.3, find the original three-dimensional point projected to this pixel in the unfolding process in step S5.2.1; Label the point cloud, mark the found original three-dimensional point in the point cloud data as belonging to the corresponding visual defect type.

7. The method of claim 1, wherein, The specific process of step S7 is: input the preprocessed radar profile image into the trained deep learning model, the trained deep learning model will analyze the input radar profile image and identify the specific patterns representing internal defects therein, the model outputs a list containing the identified defects, each defect has its bounding box on the radar profile image, defect type and estimated size; According to the position of the defect on the horizontal axis of the radar profile, the corresponding sounding radar track position is determined, and the sounding time when the defect occurs is located in combination with the time t delay on the vertical axis; the pose T of the movable robot in the three-dimensional coordinate system of the roadway is provided by the positioning system R→w (t), the transformation relationship T G→R of the ground penetrating radar antenna relative to the movable robot body is combined, and the global pose T G→T of the ground penetrating radar antenna at time t is calculated G→T (t); T G→T (t) satisfies the following formula: T G→T (t) = T R→T (t) · T G→R After detecting the internal defect at time t, the corresponding tunnel lining surface point of the defect is back calculated according to the three-dimensional position and attitude of the antenna; search for the nearest point P0 in the color point cloud near the current position of the ground penetrating radar antenna as the surface mapping position of the defect reflection signal, realize the spatial correlation between the internal defect and the surface structure; determine the normal direction of the lining surface point, calculate the surface normal n in the local neighborhood of P in the colored point cloud.

8. The method for registration of point cloud based tunnel surface and internal defect synchronous perception according to claim 1, characterized in that, The specific process of the step S8 is: determining the lining surface point P0 corresponding to the internal defect and the normal direction n of the lining surface point P0 according to the ground penetrating radar antenna pose T G→T (t) and the color point cloud data, determining the three-dimensional position P of the internal defect, P satisfies the following formula: P = P0 + n·d Where d is the offset depth, d = v·(t-t0) / 2, v is the radar wave propagation speed, t is the time delay, and t0 is the zero time.

9. The method for registration of point cloud based tunnel surface and internal defect synchronous perception according to claim 1, characterized in that, The specific process in step S10 is: according to the surface defect colored point cloud data obtained in step S6, each point in the point cloud contains its three-dimensional coordinate color value and surface defect type label; according to the internal defect information obtained in step S9, including defect type, estimated position and size parameters, and stored in a structured manner; Load the surface defect point cloud into the selected three-dimensional visualization platform or library; According to the defect label, set the display style, assign different colors or highlight effects to different types of surface defects to enhance the recognition; The internal defect data is traversed, and appropriate geometric primitives are selected for visualization mapping according to the defect type: a sphere or a cube is used for hollow or empty modeling, and a semi-transparent volume block is used for the water-containing area; all primitives are translated and positioned with the estimated three-dimensional coordinates of the corresponding defects as the center, and appropriate transparency is set to ensure the mutual visibility of the primitives and the background point cloud structure; By superimposing the internal defect primitives onto the surface point cloud data, a three-dimensional fusion visualization result in a unified coordinate system is formed, realizing the synchronous display of the surface and internal defects of the roadway structure, and providing a basis for defect evaluation and repair strategy formulation.

Images