A tunnel point cloud continuous registration and splicing method, medium and equipment

Abstract

The application discloses a tunnel point cloud continuous registration and splicing method, medium and equipment, and the method steps are as follows: a target displacement of a registration point is accurately collected by a visual camera to generate a two-dimensional identity code; point cloud data of a full-curved surface multi-point position of a tunnel is collected, target feature points are identified, and a target space identity code is generated; based on the two-dimensional identity code and the space identity code of each target, point cloud data collected at the multi-point position is spliced in time and space; and after the spliced point cloud data is fitted, tunnel displacement monitoring is performed. The medium and the equipment are used to implement the method. The application has the beneficial effects that: by using visual monitoring, the target displacement of the registration point is accurately obtained, the registration point is used for registration of point cloud data, the number of target points of machine vision can be greatly reduced, and the difficulty of using point cloud can be greatly reduced.

Description

Technical Field

[0001] This application relates to the field of tunnel construction technology, and in particular to a method, medium and equipment for continuous registration and splicing of tunnel point clouds. Background Technology

[0002] Laser-based 3D point cloud technology, due to its non-contact, high-precision, and high-density data acquisition capabilities, has been widely applied in tunnel engineering for tasks such as structural inspection, deformation monitoring, and construction quality assessment. However, in practical applications, especially during tunnel construction or long-term operational monitoring, the processing of point cloud data still faces several key technical challenges, hindering its automation and intelligent development.

[0003] Point cloud registration is a fundamental step in achieving global tunnel modeling and deformation analysis. Tunnel environments are characterized by their elongated structure, high surface feature redundancy, and poor lighting conditions, making feature-point-based registration algorithms prone to getting trapped in local optima. Current engineering practices commonly rely on traditional data processing methods such as filtering and denoising, and least-squares plane fitting. These methods are effective for small-scale, short-distance registration (e.g., between adjacent stations), but as the distance between collection points increases (e.g., exceeding 20 meters), the overlap between point clouds decreases, spatial geometric constraints weaken, leading to a significant decrease in the convergence of registration algorithms and severe accumulation of registration errors.

[0004] Meanwhile, the dynamic changes in the lining during construction further complicate point cloud processing. During tunnel construction, the lining surface often undergoes multiple re-spraying and leveling processes, leading to continuous changes in its geometry. For displacement monitoring or over- and under-excavation analysis using the lining as a reference, these human-induced or process-induced changes are directly recorded in the point cloud, but they are not actual displacements caused by structural stress or geological processes. Traditional point cloud processing methods struggle to distinguish between "actual structural displacement" and "appearance changes caused by construction operations," resulting in distorted displacement analysis results based on point clouds and severely impacting the assessment of tunnel structural safety. If the lining is directly used as a reference surface during registration, its own geometric changes will introduce systematic registration errors. Summary of the Invention

[0005] One objective of this application is to provide a method for continuous registration and stitching of tunnel point clouds that can solve at least one of the defects in the aforementioned background art.

[0006] Another objective of this application is to provide a computer-readable storage medium capable of implementing a method for continuous registration and stitching of tunnel point clouds that addresses at least one of the deficiencies in the aforementioned background art.

[0007] Another objective of this application is to provide an electronic device capable of implementing a method for continuous registration and stitching of tunnel point clouds that addresses at least one of the deficiencies in the aforementioned background art.

[0008] To achieve at least one of the above objectives, one aspect of this application provides a method for continuous registration and stitching of tunnel point clouds, comprising the following steps: S100: Use a vision camera to collect the displacement of all targets installed at each registration point in the tunnel, generate a two-dimensional identification code for each target containing angle labels, image coordinates and projection invariants, and extract the centroid coordinates of each target. S200: Collect point cloud data of the entire curved surface of the tunnel from multiple points. Based on the matching of the center point of each target and the centroid coordinates of each target in the point cloud data, determine the corresponding position of each target in the point cloud data and obtain the spatial identity code of each target, including the center point coordinates of the target and the three-dimensional Euclidean distance between the center point of the target and the center points of all other targets. S300: Based on the two-dimensional and spatial identification codes of each target, spatiotemporal splicing is performed on point cloud data collected from multiple locations, including spatial splicing of point cloud data from different locations at the same time, and temporal splicing of point cloud data from different times. S400: Performs surface fitting on the stitched point cloud data and obtains the displacement matrix of the target at different locations over time, which is used to monitor the displacement deformation of the entire tunnel.

[0009] Preferably, the acquisition of the two-dimensional identity code of each target in step S100 includes the following process: using a trained YOLO model to locate the image coordinate position of the target from the image data acquired by the vision camera, and assigning an angle label to each target according to the rotation angle relationship between each target and the vision camera; calculating the line cross ratio of each target as its projection invariant.

[0010] Preferably, when performing step S100, for the target installed in the tunnel, highly reflective feature patches are selected from the tunnel image data collected by the vision camera, and the target is made based on these features; when extracting the centroid coordinates of the target, the target is calculated in grayscale and weighted to obtain the centroid coordinates.

[0011] Preferably, in step S200, the specific process of matching the center point of each target with the centroid coordinates of each target is as follows: using the FPFH algorithm to perform coarse registration of the point cloud data of multiple points, and then using the local ICP algorithm to perform fine registration of the local point cloud near the target to achieve spatial compensation of the point cloud data; performing target boundary fitting on the registered point cloud data to extract the corner points of each target, and calculating the center point set of all targets based on the obtained corner points; enumerating and arranging the center point set to obtain multiple center point sets with different orders; using the PnP algorithm and the principle of minimizing reprojection error, the center point sets with different orders are projected in two dimensions and matched with the target centroid coordinates with the fixed order in step S100.

[0012] Preferably, when the visual camera identifies an unfamiliar target based on the beam crossover ratio, the following self-healing mechanism will be executed: S201: Calculate the current centroid coordinates of the targets whose cross-ratio of the remaining wire bundles has not changed, and compare them with the recorded historical centroid coordinates; S202: If the difference between the current centroid coordinates of the other targets and the historical values ​​exceeds the set error threshold, it is determined that the visual camera has changed pose, and step S203 is executed; otherwise, it is determined that the visual camera has not changed pose, and step S204 is executed. S203: Based on the current pose of the visual camera, perform a two-dimensional projection on the center point of the unfamiliar target in the point cloud data and determine whether it can coincide with the centroid coordinates of the unfamiliar target; if it can coincide, proceed to step S205, otherwise proceed to step S206. S204: Calculate whether the centroid coordinates of the unfamiliar target can be matched with the historical centroid coordinates; if they can be matched with the historical centroid coordinates, proceed to step S205; otherwise, proceed to step S206. S205: Determine the correspondence between the position of the unfamiliar target and the old target. The unfamiliar target inherits the two-dimensional identification code of the old target and updates the harness crossover ratio. S206: Add the center point of the unfamiliar target to the center point set and re-sort and match locally.

[0013] Preferably, when performing spatial stitching of point cloud data in step S300, the point cloud data of adjacent points are shared by at least some targets; when performing temporal stitching of point cloud data in step S300, the center point coordinates and centroid coordinates of the target at the previous moment are used to predict the center point coordinates of the target at the current moment through PnP mapping; the actual center point coordinates of the target at the current moment are collected, and the error between the predicted coordinates and the actual coordinates is calculated iteratively using the least squares method until the error approaches zero, thus completing the stitching in the time domain.

[0014] Preferably, step S400 includes the following process: using a greedy triangle fitting algorithm to construct a triangular mesh surface from the point cloud data at each time step, and calculating the normal vector of each vertex of the triangular mesh surface; determining the start time and the target time, taking the vertex of the triangular mesh surface at the start time as the reference vertex, drawing a ray along the normal vector direction of the reference vertex to the triangular mesh surface at the target time, and calculating the intersection point of the ray with the triangular mesh surface at the target time; taking the distance from the reference vertex to the intersection point as the normal displacement of the reference vertex between the start time and the target time, and obtaining the displacement matrix characterizing the tunnel displacement deformation by summarizing the normal displacements of all reference vertices at different target times.

[0015] Preferably, the triangular mesh surface includes a vertex set and a triangular patch set. Each triangular patch in the triangular patch set is composed of the indices of three vertices in the vertex set. When calculating the normal vector of a vertex, the index contains all triangular patches of that vertex. The unit normal vector of each triangular patch is calculated and weighted according to the area of ​​each triangular patch to obtain the normal vector of that vertex. When calculating the intersection point of the ray and the triangular mesh surface at the target time, the parametric equation of the ray is substituted into the plane equation of the triangular patch corresponding to the triangular mesh surface at the target time to solve for the distance parameter value. The centroid coordinate method is used to check whether the intersection point is located inside the triangular patch. If there are multiple intersection points inside the triangular patch, the intersection point corresponding to the smallest positive distance parameter value is taken.

[0016] Another aspect of this application provides a computer-readable storage medium storing a computer program; when the computer program is executed by a processor, it implements the above-described method for continuous registration and stitching of tunnel point clouds.

[0017] Another aspect of this application provides an electronic device, including a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program to implement the above-described tunnel point cloud continuous registration and stitching method.

[0018] Compared with the prior art, the beneficial effects of this application are as follows: (1) This application combines visual monitoring with point cloud registration. The visual camera only needs to acquire the precise displacement of a small number of targets to drive the temporal stitching and spatial registration of the entire point cloud data, which greatly reduces the number of targets required, significantly improves the on-site survival rate of targets and the continuous monitoring capability of the system, thereby enhancing the adaptability in the harsh environment of the tunnel.

[0019] (2) This application utilizes visual monitoring to obtain the displacement information of the registration point target in advance, and uses this information directly for the registration of the point cloud, so that the point cloud registration process does not need to perform absolute positioning in advance, nor does it need to rely on complex iterative algorithms such as iterative nearest point for global convergence, which can greatly reduce the computational complexity and time cost of point cloud data processing. Attached Figure Description

[0020] Figure 1 This is a schematic diagram illustrating the overall working steps of this application; Figure 2 This is a schematic diagram illustrating the specific workflow of the self-healing mechanism implemented in this application. Detailed Implementation

[0021] The present application will now be further described in conjunction with specific embodiments. It should be noted that, in the description of this specification, the use of terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicates that the specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms should not be construed as necessarily referring to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0022] In the description of this application, it should be noted that the terms "center", "lateral", "longitudinal", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc., which indicate the orientation and positional relationship based on the orientation or positional relationship shown in the accompanying drawings, are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and should not be construed as limiting the specific protection scope of this application.

[0023] It should be noted that the terms "first," "second," etc., in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0024] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0025] In this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can include direct contact between the first and second features, or contact between the first and second features through another feature between them. Furthermore, "above," "over," and "on top" of the second feature includes the first feature being directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature includes the first feature being directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0026] The terms “comprising” and “having”, and any variations thereof, in the specification and claims of this application are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0027] One preferred embodiment of this application, such as Figure 1 As shown, a method for continuous registration and stitching of tunnel point clouds includes the following steps: S100: Use a vision camera to collect the displacement of all targets installed at intervals in the tunnel, generate a two-dimensional identification code for each target containing angle labels, image coordinates and projection invariants, and extract the centroid coordinates of each target.

[0028] It should be noted that in the tunnel construction and operation environment, the lighting conditions are poor and there is a lot of background interference, making it difficult for traditional machine vision methods to reliably identify multiple targets. At the same time, when multiple targets look similar, the lack of a unique identifier can easily lead to data association errors in subsequent displacement monitoring.

[0029] This step generates a two-dimensional identification code for each target, containing angle labels, image coordinates, and projection invariants, thereby achieving unique target identification and robust tracking and avoiding identity confusion. Combined with sub-pixel level extraction of centroid coordinates, it provides a high-precision two-dimensional reference point for subsequent point cloud registration, laying the foundation for the accuracy of displacement monitoring.

[0030] S200: Collects point cloud data of the entire curved surface of the tunnel from multiple points. Based on the matching of the center point of each target in the point cloud data with the centroid coordinates of each target, the corresponding position of each target in the point cloud data is determined, and the spatial identity code of each target is obtained, including the center point coordinates of the target and the three-dimensional Euclidean distance between the center point of the target and the center points of all other targets.

[0031] It should be noted that laser scanning of a single point can produce point cloud holes due to target occlusion, resulting in inaccurate extraction of the target center. At the same time, the two-dimensional target centroid acquired by the visual camera and the target point cloud clusters in the three-dimensional point cloud are in different coordinate systems. How to automatically and accurately establish the correspondence between the two is a key challenge for point cloud registration.

[0032] This step effectively eliminates blind spots caused by single-point scanning by acquiring and matching point clouds of the entire curved surface of the tunnel from multiple locations. Based on the matching of the target's spatial center point and the two-dimensional centroid coordinates, the corresponding positions of each target in the point cloud data are determined, and a spatial identification code containing the coordinates of the spatial center point and the three-dimensional Euclidean distance between targets is generated, providing a three-dimensional spatial reference and geometric constraints for subsequent spatiotemporal stitching.

[0033] S300: Based on the two-dimensional and spatial identification codes of each target, it performs spatiotemporal global stitching of point cloud data collected from multiple locations, including spatial stitching of point cloud data from different locations at the same time, and temporal stitching of point cloud data from different times.

[0034] It should be understood that in long-distance longitudinal monitoring of tunnels, point clouds collected from different scanning points need to be unified to the same coordinate system, and point clouds collected at different times also need to consider the alignment of the time dimension. Traditional methods rely on absolute positioning equipment (such as total stations) or global feature matching, which are difficult to implement in tunnels and are prone to cumulative errors.

[0035] This step, based on the two-dimensional and spatial identification codes of each target, achieves spatiotemporal full-domain stitching: spatial stitching merges the point clouds of multiple sites at the same time into a complete tunnel cross-section point cloud, eliminating stitching misalignment; temporal stitching aligns point cloud data from different times to a unified time series, providing a continuous and consistent spatiotemporal data foundation for deformation monitoring.

[0036] S400: Performs surface fitting on the stitched point cloud data and obtains the displacement matrix of the target at different locations over time, which is used to monitor the displacement deformation of the entire tunnel.

[0037] It should be understood that the stitched point cloud is a discrete set of points. Directly comparing discrete points is difficult to reflect the continuous deformation of the tunnel lining and is easily affected by noise. Traditional deformation calculation methods often cannot accurately obtain the normal displacement perpendicular to the lining surface, which is a key indicator for assessing tunnel safety.

[0038] This step involves surface fitting of the stitched point cloud data, transforming the discrete point cloud into a continuous surface model, which can more realistically reflect the geometry of the tunnel lining. Based on this, displacement and deformation monitoring can be performed to obtain the complete deformation field of the entire tunnel cross section, providing intuitive and reliable monitoring data for tunnel safety assessment.

[0039] Understandably, compared to traditional methods, this application combines visual monitoring with point cloud registration. The visual camera only needs to acquire the precise displacement of a small number of targets to drive the temporal stitching and spatial registration of the entire point cloud data, significantly reducing the number of targets required and greatly improving the on-site survival rate of targets and the continuous monitoring capability of the system, thereby enhancing its adaptability to the harsh environment of tunnels. Simultaneously, by using visual monitoring to pre-acquire the displacement information of the registration point targets and directly using this information for point cloud registration, the point cloud registration process does not require absolute positioning in advance, nor does it rely on complex iterative algorithms such as iterative nearest point for global convergence, which can significantly reduce the computational complexity and time cost of point cloud data processing.

[0040] To facilitate understanding of the technical solution of this application, the specific implementation process of each step will be described in detail below.

[0041] In a specific embodiment, when performing step S100, obtaining the two-dimensional identification code of the target includes the following steps: S110: Multiple specially designed targets are installed at certain intervals along the longitudinal direction of the tunnel, such as 5-10m. These targets have high reflectivity, such as circular or specific polygonal patches made of reflective film or reflective paint.

[0042] S120: A fixedly installed vision camera (including an infrared fill light, a black and white industrial camera, and an infrared filter) continuously acquires image data of the tunnel interior wall; among them, the infrared filter can effectively filter out ambient stray light and highlight highly reflective patches.

[0043] S130: The trained YOLO model is used to coarsely locate highly reflective patches in the image data to obtain the bounding box of each target. At the same time, since each target is assigned a specific rotation angle relative to the camera optical axis (e.g., 0°, 45°, 90°, etc.) during installation, the YOLO model identifies this rotation angle during detection and assigns a unique angle label T to the target.

[0044] S140: Utilizing the principle of line bundle cross-ratio invariants in projective geometry, five feature points (e.g., five corner points or the center of internal sub-patterns) are extracted for each target region, ensuring that no three points are collinear, and the line bundle cross-ratio CR is calculated. The specific calculation process of the line bundle cross-ratio CR is well known to those skilled in the art, and therefore will not be described in detail here.

[0045] Understandably, since the cross ratio (CR) remains constant under affine transformation and perspective projection, even if the target undergoes some deformation or a change in viewing angle, its CR value remains essentially constant. When the vision camera is first powered on, it iterates through all targets in the image, assigns labels to all targets recognized by the YOLO model, and calculates the cross ratio (CR) for each target, generating a unique two-dimensional identification code P for each target. R : [T, (u0, v0), CR]; where (u0, v0) represents the coordinates of the center point of the bounding box of the target, i.e., the image coordinates of the target.

[0046] In a specific example, for the specially designed target in step S110, highly reflective patches can be selected from the tunnel image data acquired by the vision camera, and these features can be used to create the target. Therefore, during target identification, based on the combination of an infrared supplementary light, a monochrome industrial camera, and an infrared filter used in step S120, infrared light can penetrate some dust, and the monochrome camera is more sensitive to infrared light. The infrared filter removes visible light interference, resulting in only the highly reflective target appearing as a bright patch in the image data, while the background is almost entirely black, effectively simplifying image processing.

[0047] In a specific example, when performing step S100, the centroid coordinates of the target can be extracted using a gray-scale weighted centroid method based on the target's features. This involves calculating the centroid coordinates by performing gray-scale calculations on the target and then weighting the results. c v c The specific calculation expression is as follows: , .

[0048] In the formula, I(u, v) represents the pixel gray value corresponding to the coordinate point (u, v) in the image coordinate system.

[0049] In a specific embodiment, when executing step S200, since the lidar emits point-like divergence, data is collected only at one point. This may cause the tunnel contour to be obscured by the target, resulting in blank areas on the contour surface. Therefore, multi-point measurements are needed to perform point cloud compensation on the tunnel inner wall contour. The specific process for point cloud compensation is as follows: S210: Use the FPFH algorithm to extract front-end features from point cloud data at multiple locations to complete coarse registration.

[0050] Understandably, to eliminate obstructions, the lidar scans the tunnel wall at at least two different points (e.g., on the left and right sides of the same cross-section) to acquire multiple sets of raw point cloud data. Using two sets of raw point cloud data P... A and P B For example, the Fast Point Feature Histogram (FPFH) algorithm is used to extract two sets of point cloud data P. A and P B By performing initial alignment on the local geometric features of the point cloud data P, the point cloud data can be transformed. B Roughly transformed to P A In the coordinate system.

[0051] S220: The local ICP algorithm is used to perform fine registration of the local point cloud near the target, thereby achieving spatial compensation of the point cloud data.

[0052] Understandably, the target centroid will shift in coordinates depending on the angle at which the point cloud data is acquired; therefore, based on the coarse registration, fine registration is performed only on the local point cloud near each target using the Iterative Closest Point (ICP) algorithm. Continuing with the two sets of point cloud data P mentioned above... A and P B For example, the objective function corresponding to the ICP algorithm is: .

[0053] In the formula, N represents the total number of point pairs corresponding to the two local point clouds used for registration; R represents a 3×3 rotation matrix used to rotate the source point cloud so that its orientation matches that of the target point cloud; t represents a 3×1 translation matrix used to translate the source point cloud; P A,i and P B,i Represents point cloud data P A and P B The i-th point pair used for registration in the corresponding local point cloud.

[0054] Based on the objective function described above, the optimal rotation matrix R and translation vector t are calculated by minimizing the sum of squared distances between point pairs in two local point clouds. This optimal transformation is then applied to all point cloud data to achieve accurate registration and fusion of multi-point point cloud data, forming complete tunnel cross-section point cloud data.

[0055] In a specific embodiment, when performing step S200, the specific process of matching the center point of each target with the centroid coordinates of each target based on the point cloud data of precise registration and fusion is as follows: S230: Fit the target boundary to the registered point cloud data to extract the corner points of each target, and calculate the set of spatial center points of all targets based on the obtained corner points.

[0056] Specifically, in the registered point cloud data, Euclidean distance-based clustering or DBSCAN density clustering algorithms are used to automatically segment the point cloud clusters corresponding to each target based on spatial distance or density distribution. For each point cloud cluster, the outermost boundary points of the point cloud cluster are extracted using the convex hull algorithm or Alpha Shapes algorithm to form a boundary line; straight line segments of the boundary line are fitted, and the intersection points of adjacent straight line segments are calculated as the corner points of the target; based on at least three detected corner points, the geometric center of the target is calculated, obtaining the spatial center point P(X, Y, Z) of the target in the spatial coordinate system; by summing up the spatial center points of all targets, the set of spatial center points B = { P1(X1, Y1, Z1), P2(X2, Y2, Z2), ..., P...} is obtained. n (X n Y n Z n )}; where P i (X i Y i Z i Let ) represent the spatial center point corresponding to the i-th target, i = {1, 2, ..., n}, and n represent the total number of targets.

[0057] S240: Enumerate and arrange the set of spatial center points to obtain multiple sets of spatial center points with different orders.

[0058] It should be noted that the point cloud clusters extracted from the registered point cloud data by the clustering algorithm are unordered, meaning that the extracted point cloud map cannot be directly correlated with the target installation position captured by the visual camera. Therefore, it is necessary to enumerate all possible sortings of the spatial center point set to facilitate determining in the next step that the unique spatial center point sorting corresponds exactly to the target installation position captured by the visual camera.

[0059] S250: Using the PnP algorithm and the principle of minimizing reprojection error, the spatial center point sets of different sortings are projected in two dimensions and matched with the target centroid coordinates of the fixed sorting in step S100.

[0060] Specifically, for all n points in the set B of spatial center points, a full permutation can be performed to get B. nThe system generates a set of three distinct ordered spatial points. Simultaneously, it sorts all target positions acquired by the visual camera in step S100 and uses the PnP algorithm to solve for a virtual camera pose. For each ordered spatial point set, based on the obtained camera pose and combined with the known camera intrinsic parameter matrix, it reprojects all 3D points within the ordered spatial point set onto the image plane to obtain 2D projected coordinates. The reprojection error E between the 2D projected coordinates and the centroid coordinates of all targets in step S100 is calculated, with the specific expression as follows: .

[0061] In the formula, D i This represents the centroid coordinates of the i-th target. Let represent the two-dimensional projected coordinates of the i-th target in an ordered set of spatial points, and n represent the total number of targets.

[0062] Based on the number of ordered spatial points, we can obtain B. n For each reprojection error E, the sorting of the ordered spatial point set corresponding to the smallest reprojection error E is matched with the sorting of the target acquired by the visual camera.

[0063] Understandably, after a successful match, the three-dimensional Euclidean distances ΔL1, ΔL2, ..., ΔL' between all target space center points and the remaining target space center points are calculated. n-1 This allows us to obtain the spatial identity code P of the target. K : [P(X, Y, Z), ΔL1, ΔL2,…, ΔL n-1 After assigning the two-dimensional identification code to the target, the unique identification code P of the target can be obtained: [T, (u0, v0), CR, P(X, Y, Z), ΔL1, ΔL2, ..., ΔL n-1 ].

[0064] It should be understood that during tunnel construction, targets are prone to damage or degradation of surface reflectivity due to grout coverage, dust pollution, mechanical impact, or natural aging. This can cause vision cameras to be unable to stably acquire the line cross ratio (CR) feature of the target, resulting in the loss or invalidation of the target's two-dimensional identification code. To restore monitoring capabilities, engineers typically replace or reinstall new targets. Based on the newly installed targets, the vision camera may recognize unfamiliar line cross ratios. For targets corresponding to unfamiliar line cross ratios, position matching is required to ensure the accuracy of point cloud registration. This embodiment provides a self-healing mechanism for targets corresponding to unfamiliar line cross ratios; for ease of understanding, a detailed description follows.

[0065] In a specific embodiment, such as Figure 2 As shown, when the visual camera identifies an unfamiliar target based on the beam crossover ratio, the following self-healing mechanism will be executed: S201: Calculate the current centroid coordinates of the targets whose cross-ratios of the remaining wire bundles have not changed, and compare them with the recorded historical centroid coordinates.

[0066] It should be understood that the vision camera is mounted on a trolley. During tunnel construction, the trolley moves synchronously with the vision camera, causing its pose to change relative to the movement. Therefore, the purpose of this step is to determine whether the vision camera's current pose has changed. Specifically, for targets whose harness cross-ratio has not changed (i.e., targets with known positions), we can determine whether the vision camera's pose has changed based on whether its centroid coordinates have changed.

[0067] S202: If the difference between the current centroid coordinates of the other targets and the historical values ​​exceeds the set error threshold, it is determined that the visual camera has changed pose, and step S203 is executed; otherwise, it is determined that the visual camera has not changed pose, and step S204 is executed.

[0068] It should be understood that the specific value of the error threshold can be selected according to the actual needs of those skilled in the art; for example, it can be 0.1m. There are several ways to determine if a visual camera has experienced a change in pose. It can be that the average value (i.e., average distance) of the position change of the current centroid coordinates of targets at known positions relative to historical values ​​exceeds the error threshold, or that the position change of the current centroid coordinates of targets at known positions relative to historical values ​​exceeds the error threshold for more than a certain number (e.g., 50%) of targets at known positions.

[0069] S203: Based on the current pose of the visual camera, perform a two-dimensional projection on the center point of the unfamiliar target in the point cloud data and determine whether it can coincide with the centroid coordinates of the unfamiliar target; if it can coincide, proceed to step S205, otherwise proceed to step S206.

[0070] Specifically, based on the current pose estimation of the visual camera (which can be solved in real-time using the PnP algorithm through the 3D-2D correspondence of existing targets), the spatial center point corresponding to the unfamiliar target in the point cloud data is reprojected onto the image plane to obtain the projected coordinates. It is then determined whether these projected coordinates fall within the bounding box of the unfamiliar target detected by YOLO, or whether the distance between these coordinates and the centroid coordinates of the unfamiliar target is less than a set pixel threshold (e.g., 5 pixels). If these conditions are met, it indicates that the unfamiliar target actually corresponds to an existing target with an existing identification code. If the projected coordinates do not match the position of the unfamiliar target in the image, it means that after the change in the visual camera pose, the unfamiliar target cannot establish a correspondence with the spatial center point of any existing target in the point cloud, and in this case, it is determined as a mismatch.

[0071] S204: Calculate whether the centroid coordinates of the unfamiliar target can be matched with the historical centroid coordinates; if they can be matched with the historical centroid coordinates, proceed to step S205; otherwise, proceed to step S206.

[0072] Specifically, if the pose of the visual camera remains unchanged, the current centroid coordinates of the unfamiliar target can be directly matched with the historical centroid coordinate sequence of all previously recorded targets. The specific method is as follows: calculate the Euclidean distance (pixel distance) between the current centroid of the unfamiliar target and the most recently recorded centroid of each previously recorded target. If the minimum distance is less than a set threshold (e.g., 3 pixels), the unfamiliar target is considered to coincide with the corresponding previously recorded target (i.e., replaced in situ); otherwise, it is considered a target at a completely new location.

[0073] S205: Determine if the position of the unfamiliar target corresponds to that of the old target. The unfamiliar target inherits the two-dimensional identification code of the old target and updates the harness crossover ratio.

[0074] Specifically, when an unfamiliar target is determined to correspond to the position of an old target, the unfamiliar target can inherit the angle label, historical image coordinates, and spatial identity information from the old target's two-dimensional identification code. Simultaneously, the CR value of the corresponding entry in the two-dimensional identification code is updated using the currently calculated new harness crossover ratio. Furthermore, if the camera pose changes, the coordinates of the spatial center point corresponding to the target also need to be updated to ensure the continuity of monitoring data, enabling seamless integration of the historical displacement data of the old target with the subsequent data of the new target.

[0075] S206: Add the spatial center point of the unfamiliar target to the spatial center point set and re-sort and match locally.

[0076] Specifically, if an inheritance relationship cannot be established through any of the above methods, the unfamiliar target is determined to be a completely new target installed in a non-original location (or the original target has been permanently lost). In this case, the spatial center point coordinates of the new target can be added to the spatial center point set in step S200, and the CR value and centroid coordinates of the new target can be recorded as a new two-dimensional identification code. Since the positional relationships of known targets whose cross-sectional ratio has not changed are always preserved, the updated spatial center point set can be locally rearranged; that is, only the newly added spatial center points are rearranged, while the remaining spatial center points with determined positional relationships inherit the previous matching relationships. Then, using the PnP algorithm and the principle of minimizing reprojection error, a local match is performed with the unfamiliar target whose position is determined in the two-dimensional identification code set acquired by the visual camera at the current moment, thereby achieving the redetering of the one-to-one correspondence between all targets.

[0077] Understandably, the self-healing mechanism of this application can accurately distinguish between camera pose changes and actual target changes, avoiding misjudgments caused by minute camera displacements, preventing invalid updates or erroneous deletions of the identity database, and ensuring the stability of long-term monitoring. Simultaneously, it enables in-situ target replacement with identity inheritance: when a new target is installed at the same location after an old target is damaged, the system can automatically identify the overlapping positions, allowing the new target to inherit the old target's identity code, maintaining the temporal continuity of deformation monitoring data without manual recalibration. Furthermore, it supports dynamic expansion of targets installed in non-in-situ locations: when a new target is installed at a new location, the system automatically adds it to the spatial center point set and re-matches globally, seamlessly expanding the monitoring network to adapt to dynamic changes during tunnel construction. Additionally, it enables adaptive handling of camera pose changes: when camera pose changes cause CR value drift, the system uses point cloud projection for verification, updating camera parameters and restoring matching without manual intervention, enhancing the system's robustness to environmental disturbances.

[0078] In a specific embodiment, when executing step S300, the spatial stitching of point cloud data is essentially the fusion of point cloud data from adjacent points. To ensure the accuracy of spatial stitching, the point cloud data from adjacent points must share at least some targets, for example, at least 3 to 5 identical common targets. The specific process for spatial stitching of point cloud data from adjacent points is as follows: initial alignment is performed based on the macroscopic geometric features of the point cloud; ICP iteration is performed only based on the local point cloud near the shared target to optimize the transformation matrix; the point cloud in the overlapping area is downsampled (e.g., voxel filtering) to eliminate redundant data and make the point cloud distribution uniform; finally, the coordinates of the extracted three-dimensional spatial center point of the shared target are directly used to solve for the precise rigid body transformation parameters using the least squares method, and the point cloud of subsequent points is forcibly aligned to the coordinate system of the reference point to complete the seamless stitching of the entire circumferential section.

[0079] In a specific embodiment, when executing step S300, the goal of temporal stitching of the point cloud data is to align the full-section point cloud data M1 corresponding to the previous moment with the full-section point cloud data M2 corresponding to the current moment. The specific steps are as follows: using the spatial center point coordinates and centroid coordinates of the target at the previous moment, the pose change of the visual camera is calculated using the PnP algorithm, thereby predicting the spatial center point coordinates of the target at the current moment. The actual spatial center point coordinates of the target at the current moment are collected, and the error between the predicted coordinates and the actual coordinates is iteratively calculated using the least squares method until the error approaches zero, completing the temporal stitching.

[0080] In a specific embodiment, when performing step S400, the displacement is calculated using a surface reconstruction and normal projection intersection method based on a greedy triangle fitting algorithm; the specific process is as follows: S410: A greedy triangle fitting algorithm is used to construct a triangular mesh surface from the point cloud data at each time step, and the normal vector of each vertex of the triangular mesh surface is calculated.

[0081] Understandably, the greedy triangle fitting algorithm projects a 3D point cloud onto a 2D plane, performs Delaunay triangulation within the plane, and then maps it back to 3D space to generate the desired triangular mesh surface. The generated triangular mesh surface M... t Including vertex set V t and the set of triangular facets F t , where the set of triangular facets F t Each triangular facet f in t It consists of the indices of three vertices in the vertex set.

[0082] When calculating the normal vector of a vertex, the index contains all triangle faces of that vertex. The unit normal vector of each triangle face is calculated, and then a weighted average is taken according to the area of ​​each triangle face to obtain the normal vector of the vertex. For ease of understanding, the specific calculation process will be described in detail below.

[0083] For any vertex v ti Search for all triangle faces containing the given vertex; calculate the unit normal vector of each triangle face, and take a weighted average of the areas of all triangle faces to obtain the average unit normal vector as the normal vector of the vertex; the specific calculation expression is as follows: .

[0084] In the formula, n fti,k V represents the unit normal vector of the k-th triangular facet in the set of triangular faces. ti,k1 V ti,k2 V ti,k3 Let n represent the three vertices of the k-th triangular facet that makes up the triangular facet. ti Let A represent the average unit normal vector, N represent the total number of triangular faces in the set of triangular faces, and A represent the average unit normal vector. ti,k This represents the area of ​​the k-th triangular facet in the triangular facet.

[0085] S420: Determine the start time and target time. Take the vertex of the triangular mesh surface at the start time as the reference vertex, draw a ray along the normal vector direction of the reference vertex to the triangular mesh surface at the target time, and calculate the intersection point of the ray and the triangular mesh surface at the target time.

[0086] Understandably, when calculating the intersection point of the ray and the triangular mesh surface at the target time, the parametric equation of the ray is substituted into the plane equation of the triangular patch corresponding to the triangular mesh surface at the target time to solve for the distance parameter value. The centroid coordinate method is used to check whether the intersection point is located inside the triangular patch. If there are multiple intersection points inside the triangular patch, the intersection point corresponding to the smallest positive distance parameter value is taken.

[0087] Specifically, from the reference vertex V t0 Starting from, along its normal vector n t0 Draw a ray in the direction of s, and the expression for the parametric equation r(s) of the ray is: r(s) = V t0 +s·n t0 , s≥0.

[0088] In the formula, s represents the distance moved from the starting point along the ray direction.

[0089] The plane equation of the triangular facet is expressed as: (xa)·n f =0.

[0090] In the formula, x represents the position vector of any point on the plane, a represents the position vector of a known point on the triangular patch, and n f This represents the unit normal vector of the triangular facet.

[0091] Substituting the parametric equation r(s) into the plane equation, we can obtain the expression for the distance parameter s as follows: .

[0092] The expression for the intersection point equation is: .

[0093] In the formula, q represents the coordinates (three-dimensional coordinates) of the intersection point of the ray and the plane containing the triangular facet.

[0094] After obtaining the intersection equation, the centroid coordinate method can be used to check whether the obtained intersection point is located inside the triangular facet. Since the ray may have multiple intersection points with the triangular mesh surface (e.g., passing through the arch and the invert), the minimum value of s > 0 among all valid intersection points can be taken as the final displacement.

[0095] S430: The distance from the reference vertex to the intersection point is taken as the normal displacement of the reference vertex between the start time and the target time. By summing up the normal displacements of all reference vertices at different target times, the displacement matrix representing the tunnel displacement deformation is obtained.

[0096] Another aspect of this application provides a computer-readable storage medium, in a preferred embodiment of which a computer program is stored on the storage medium; when the computer program is executed by a processor, it implements the above-described method for continuous registration and stitching of tunnel point clouds.

[0097] Another aspect of this application provides an electronic device, in one preferred embodiment of which includes a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program to implement the above-described tunnel point cloud continuous registration and stitching method.

[0098] The basic principles, main features, and advantages of this application have been described above. Those skilled in the art should understand that this application is not limited to the above embodiments. The embodiments and descriptions in the specification are merely the principles of this application. Various changes and modifications can be made to this application without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claims. The scope of protection claimed by this application is defined by the appended claims and their equivalents.

Claims

1. A method for continuous registration and stitching of tunnel point clouds, characterized in that, Includes the following steps: S100: Use a vision camera to collect the displacement of all targets installed at each registration point in the tunnel, generate a two-dimensional identification code for each target containing angle labels, image coordinates and projection invariants, and extract the centroid coordinates of each target. S200: Collect point cloud data of the entire curved surface of the tunnel from multiple points. Based on the matching of the center point of each target and the centroid coordinates of each target in the point cloud data, determine the corresponding position of each target in the point cloud data and obtain the spatial identity code of each target, including the center point coordinates of the target and the three-dimensional Euclidean distance between the center point of the target and the center points of all other targets. S300: Based on the two-dimensional and spatial identification codes of each target, spatiotemporal splicing is performed on point cloud data collected from multiple locations, including spatial splicing of point cloud data from different locations at the same time, and temporal splicing of point cloud data from different times. S400: Performs surface fitting on the stitched point cloud data and obtains the displacement matrix of the target at different locations over time, which is used to monitor the displacement deformation of the entire tunnel. The beam crossover ratio of each target is calculated as its projection invariant. When the visual camera recognizes an unfamiliar target based on the beam crossover ratio, the following self-healing mechanism will be executed: S201: Calculate the current centroid coordinates of the targets whose cross-ratio of the remaining wire bundles has not changed, and compare them with the recorded historical centroid coordinates; S202: If the difference between the current centroid coordinates of the other targets and the historical values ​​exceeds the set error threshold, it is determined that the visual camera has changed pose, and step S203 is executed; otherwise, it is determined that the visual camera has not changed pose, and step S204 is executed. S203: Based on the current pose of the visual camera, perform a two-dimensional projection on the center point of the unfamiliar target in the point cloud data and determine whether it can coincide with the centroid coordinates of the unfamiliar target; if it can coincide, proceed to step S205, otherwise proceed to step S206. S204: Calculate whether the centroid coordinates of the unfamiliar target can be matched with the historical centroid coordinates; if they can be matched with the historical centroid coordinates, proceed to step S205; otherwise, proceed to step S206. S205: Determine the correspondence between the position of the unfamiliar target and the old target. The unfamiliar target inherits the two-dimensional identification code of the old target and updates the harness crossover ratio. S206: Add the center point of the unfamiliar target to the center point set and re-sort and match locally.

2. The tunnel point cloud continuous registration and stitching method as described in claim 1, characterized in that, The trained YOLO model is used to locate the image coordinates of the target from the image data acquired by the vision camera, and angle labels are assigned to each target according to the rotation angle relationship between the target and the vision camera.

3. The method for continuous registration and stitching of tunnel point clouds as described in claim 1, characterized in that, When performing step S100, for the target installed in the tunnel, highly reflective feature patches are selected from the tunnel image data collected by the vision camera, and the target is made based on these features; when extracting the centroid coordinates of the target, the target is calculated in grayscale and weighted to obtain the centroid coordinates.

4. The method for continuous registration and stitching of tunnel point clouds as described in any one of claims 1-3, characterized in that, In step S200, the specific process of matching the center point of each target with the centroid coordinates of each target is as follows: The FPFH algorithm is used to perform coarse registration of point cloud data at multiple locations, and then the local ICP algorithm is used to perform fine registration of local point clouds near the target, thereby achieving spatial compensation of point cloud data. The registered point cloud data is fitted with the target boundary to extract the corner points of each target, and the set of center points of all targets is calculated based on the obtained corner points. Enumerate and arrange the set of center points to obtain multiple sets of center points with different sorts; Using the PnP algorithm and the principle of minimizing reprojection error, the set of center points with different sortings is projected in two dimensions and matched with the target centroid coordinates with the fixed sorting in step S100.

5. The method for continuous registration and stitching of tunnel point clouds as described in claim 1, characterized in that, When performing spatial stitching of point cloud data in step S300, the point cloud data of adjacent points must share at least a portion of the target data. When performing temporal stitching of point cloud data in step S300, the center point coordinates and centroid coordinates of the target at the previous moment are used to predict the center point coordinates of the target at the current moment through PnP mapping. The actual center point coordinates of the target at the current moment are collected, and the error between the predicted coordinates and the actual coordinates is calculated iteratively using the least squares method until the error approaches zero, thus completing the splicing of the time domain.

6. The method for continuous registration and stitching of tunnel point clouds as described in claim 1, characterized in that, Step S400 includes the following process: A greedy triangle fitting algorithm is used to construct a triangular mesh surface from the point cloud data at each time step, and the normal vector of each vertex of the triangular mesh surface is calculated. Determine the start time and the target time. Take the vertex of the triangular mesh surface at the start time as the reference vertex, draw a ray along the normal vector direction of the reference vertex to the triangular mesh surface at the target time, and calculate the intersection point of the ray and the triangular mesh surface at the target time. The distance from the reference vertex to the intersection point is taken as the normal displacement of the reference vertex between the starting time and the target time. By summing up the normal displacements of all reference vertices at different target times, the displacement matrix characterizing the tunnel displacement deformation is obtained.

7. The tunnel point cloud continuous registration and stitching method as described in claim 6, characterized in that, A triangular mesh surface consists of a set of vertices and a set of triangular patches. Each triangular patch in the set of triangular patches is composed of the indices of three vertices in the set of vertices. When calculating the normal vector of a vertex, the index contains all triangle faces of that vertex, the unit normal vector of each triangle face is calculated, and a weighted average is taken according to the area of ​​each triangle face to obtain the normal vector of the vertex. When calculating the intersection point of the ray and the triangular mesh surface at the target time, the parametric equation of the ray is substituted into the plane equation of the triangular patch corresponding to the triangular mesh surface at the target time to solve for the distance parameter value. The centroid coordinate method is used to check whether the intersection point is located inside the triangular patch. If there are multiple intersection points inside the triangular patch, the intersection point corresponding to the smallest positive distance parameter value is taken.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program; when the computer program is executed by a processor, it implements the tunnel point cloud continuous registration and stitching method as described in any one of claims 1-7.

9. An electronic device, characterized in that, It includes a processor and a memory; the memory is used to store a computer program, and the processor is used to execute the computer program to implement the tunnel point cloud continuous registration and stitching method as described in any one of claims 1-7.

Images