Structured light point cloud stitching method and medium for medical navigation

Abstract

This invention discloses a structured light point cloud stitching method and medium for medical navigation. The method includes: acquiring current frame point cloud data; aligning the current frame point cloud data into a point cloud model to obtain the pose corresponding to the current frame point cloud data; acquiring a pose map; establishing relative pose constraints corresponding to the current frame point cloud data based on the pose map, the pose corresponding to the current frame point cloud data, the current frame point cloud data, historical frame point cloud data, and a preset pose constraint algorithm, until relative pose constraints are established between all point cloud data; acquiring a global pose map; obtaining an optimized pose corresponding to the point cloud data based on the global pose map and a preset global pose optimization algorithm; performing pose transformation on the point cloud data based on the optimized pose corresponding to the point cloud data; and stitching the pose-transformed point cloud data to obtain the point cloud stitching result. This method solves the problem of uneven accuracy in traditional methods, achieving high global consistency, uniform accuracy across the entire area, and requiring no manual intervention.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and computer vision technology, and in particular to a structured light point cloud stitching method and medium for medical navigation. Background Technology

[0002] In neuronavigation systems, precise registration of the patient's preoperative medical imaging data (such as CT and MRI) with the actual physical space during surgery is a crucial prerequisite for achieving accurate positioning and guidance. This process heavily relies on constructing a high-precision, highly complete three-dimensional surface model of the head.

[0003] Currently, the following two methods are mainly used in existing technologies to obtain head surface point cloud data: Contact probe scanning: This method uses an optical positioning system to track the tip of a handheld probe, allowing the operator to collect spatial coordinates point by point on the patient's affected area. However, this method has inherent drawbacks: First, the physical contact between the probe and soft tissue causes skin deformation, introducing systematic errors; second, manual operation makes it difficult to avoid shaking and slippage, resulting in a large amount of noise in the collected data; finally, point-by-point collection is inefficient, making it difficult to complete full coverage of the affected area (especially the top of the head, back of the head, etc.) within a reasonable time.

[0004] Multi-frame stitching based on structured light: As a non-contact solution, structured light technology can quickly acquire high-density local point clouds. By moving the camera to acquire data from multiple perspectives and performing registration and stitching, a complete model of the affected area can theoretically be reconstructed. However, existing multi-frame stitching methods are mainly divided into two categories, each with significant shortcomings: one type of method relies on an external optical positioning system to provide the coordinate transformation relationship between each frame of point cloud, which not only increases system complexity and cost but also imposes stringent requirements on time synchronization and system calibration, resulting in poor overall robustness; the other type of method uses iterative frame-by-frame registration (such as the classic ICP algorithm) for incremental stitching. This type of method suffers from serious error accumulation problems. As the number of stitched frames increases, the drift error continues to amplify, causing distortion of the final generated model on a global scale, especially in the back of the head and top of the head regions far from the initial scanning position, where geometric accuracy drops sharply.

[0005] In summary, existing technologies cannot achieve uniform, non-cumulative drift-free multi-frame point cloud stitching across the entire affected area under non-contact conditions and without external positioning assistance. Summary of the Invention

[0006] The main objective of this invention is to provide a structured light point cloud stitching method for medical navigation, aiming to solve the above-mentioned technical problems.

[0007] This invention provides a structured light point cloud stitching method and medium for medical navigation, aiming to solve the technical problem that it is impossible to achieve uniform accuracy and no cumulative drift in multi-frame point cloud stitching under non-contact conditions without external positioning assistance.

[0008] To achieve the aforementioned objectives, the first aspect of this invention proposes a structured light point cloud stitching method for medical navigation, comprising: Obtain the point cloud data of the current frame; Align the current frame point cloud data with the point cloud model to obtain the pose corresponding to the current frame point cloud data; wherein, the point cloud model is generated based on historical frame point cloud data; A pose graph is obtained, and relative pose constraints corresponding to the current frame point cloud data are established based on the pose graph, the pose corresponding to the current frame point cloud data, the current frame point cloud data, the historical frame point cloud data, and a preset pose constraint algorithm; wherein, the nodes in the pose graph represent the poses corresponding to the point cloud data, and the edges in the pose graph represent the relative pose constraints between the point cloud data. Until the relative pose constraints between all the point cloud data are established, a global pose graph is obtained, and the optimized pose corresponding to the point cloud data is obtained according to the global pose graph and the preset global pose optimization algorithm. The point cloud data is transformed into a pose based on the optimized pose corresponding to the point cloud data, and the transformed point cloud data is then stitched together to obtain the point cloud stitching result.

[0009] Further, aligning the current frame point cloud data into the point cloud model to obtain the pose corresponding to the current frame point cloud data includes: Obtain the corresponding point in the point cloud model for each point in the current frame point cloud data and the normal vector of the plane containing the corresponding point; Based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm, the pose corresponding to the current frame point cloud data is obtained; wherein, the preset point cloud alignment algorithm is used to minimize the distance error between each point and the tangent plane of the corresponding point.

[0010] Further, obtaining the pose corresponding to the current frame point cloud data based on the current frame point cloud data, the corresponding point of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm includes: Based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm, the pose corresponding to the current frame point cloud data is obtained. The error function in the preset point cloud alignment algorithm is:

[0011] Indicates the first Frame point cloud data, the first Frame point cloud data refers to the current frame point cloud data. Indicates the first Frame point cloud data The first in One point, Indicates the first The corresponding point of each point in the point cloud model Indicates the first The normal vector of the plane containing the corresponding points of each point. Indicates the first The pose corresponding to the point cloud data of the first frame, the The pose corresponding to the point cloud data of the frame is the pose corresponding to the point cloud data of the current frame. This represents the distance from each point to the tangent plane of the corresponding point. The error is represented by a pre-defined point cloud alignment algorithm. The pose that minimizes the error.

[0012] Further, obtaining the pose corresponding to the current frame point cloud data based on the current frame point cloud data, the corresponding point of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm includes: Based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm, the pose corresponding to the current frame point cloud data is obtained. The optimized error function in the preset point cloud alignment algorithm is as follows:

[0013]

[0014] Indicates the first Frame point cloud data, the first Frame point cloud data refers to the current frame point cloud data. Indicates the first Frame point cloud data The first in One point, Indicates the first The corresponding point of each point in the point cloud model Indicates the first The normal vector of the plane containing the corresponding points of each point. Indicates the first The pose corresponding to the point cloud data of the first frame, the The pose corresponding to the point cloud data of the frame is the pose corresponding to the point cloud data of the current frame. This represents the distance from each point to the tangent plane of the corresponding point. for Simplified representation, Indicates the distance threshold. The value represents the optimized error, obtained based on a preset point cloud alignment algorithm. The pose that minimizes the error after optimization.

[0015] Further, the relative pose constraint includes adjacent frame pose constraints and loop closure pose constraints. The step of obtaining the pose graph, and establishing the relative pose constraint corresponding to the current frame point cloud data based on the pose graph, the pose corresponding to the current frame point cloud data, the current frame point cloud data, the historical frame point cloud data, and a preset pose constraint algorithm, includes the following steps: The pose corresponding to the current frame point cloud data is added to the pose graph to establish the adjacent frame pose constraint corresponding to the current frame point cloud data; wherein, the adjacent frame pose constraint is the relative pose constraint between the current frame point cloud data and the previous frame point cloud data. Based on the current frame point cloud data, the historical frame point cloud data, and the preset loop closure detection algorithm, establish the loop closure pose constraint corresponding to the current frame point cloud data.

[0016] Further, the step of establishing the loop closure pose constraint corresponding to the current frame point cloud data based on the current frame point cloud data, the historical frame point cloud data, and a preset loop closure detection algorithm includes: Spatial overlap detection is performed on the current frame point cloud data and the historical frame point cloud data to obtain spatial overlap information; If the spatial overlap information exceeds a preset spatial overlap threshold, a loop pose constraint is established between the current frame point cloud data and the corresponding historical frame point cloud data. Further, the step of obtaining a global pose map, and obtaining an optimized pose corresponding to the point cloud data based on the global pose map and a preset global pose optimization algorithm, includes: Obtain the global pose graph; Based on the poses corresponding to all point cloud data in the global pose graph and the relative pose constraints between all point cloud data, the derived pose constraints corresponding to the relative pose constraints are obtained. Based on the relative pose constraints and the derived pose constraints corresponding to the relative pose constraints, the inter-frame pose error is calculated, and the inter-frame pose error is vectorized to obtain the vectorized result. Based on the vectorization processing result and the preset global pose optimization objective function, the optimized pose corresponding to the point cloud data is obtained.

[0017] Further, the step of obtaining the derived pose constraint corresponding to the relative pose constraint based on the poses corresponding to all the point cloud data in the global pose graph and the relative pose constraints between all the point cloud data includes: From the global pose graph Get the first Frame point cloud data and the first Relative pose constraints between frame point cloud data and from the global pose graph Obtain the first Pose corresponding to frame point cloud data and the first Pose corresponding to frame point cloud data ; According to the first Pose corresponding to frame point cloud data and the first Pose corresponding to frame point cloud data The relative pose constraint is obtained. Corresponding derivation of pose constraints ;in, , The global pose graph Nodes in The global pose graph represents the pose corresponding to the point cloud data. The edge in This represents the relative pose constraints between the point cloud data.

[0018] Further, the step of calculating the inter-frame pose error based on the relative pose constraints and the derived pose constraints corresponding to the relative pose constraints, and vectorizing the inter-frame pose error to obtain the vectorized result, includes: According to the first Frame point cloud data and the first Relative pose constraints between frame point cloud data and the relative pose constraint Corresponding derivation of pose constraints The inter-frame pose error is obtained. ;in, , express The reverse; For the inter-frame pose error Perform vectorization processing to obtain the vectorized result. ;in, , This indicates the inter-frame pose error. Perform a mapping from Lie groups to Lie algebras.

[0019] Further, obtaining the optimized pose corresponding to the point cloud data based on the vectorization processing result and the preset global pose optimization objective function includes: Based on the vectorization processing result and the preset global pose optimization objective function, the optimized pose corresponding to the point cloud data is obtained; The preset global pose optimization objective function is:

[0020] Represents the global pose graph The Middle The pose corresponding to the frame point cloud data express The reverse, Represents the global pose graph The Middle The pose corresponding to the frame point cloud data Represents the global pose graph The Middle Frame point cloud data and the first Relative pose constraints between frame point cloud data Represents the relative pose constraint Corresponding derivation of pose constraints Indicates the first Frame point cloud data and the first The inter-frame pose error between frame point cloud data. This indicates the inter-frame pose error. Perform a mapping from Lie groups to Lie algebras. The vectorization result, the global pose graph Nodes in The global pose graph represents the pose corresponding to the point cloud data. The edge in This represents the relative pose constraints between the point cloud data. Indicates the first The node corresponding to the frame point cloud data and the first The edges between nodes corresponding to the frame point cloud data belong to The first Optimized pose corresponding to frame point cloud data The pose that minimizes the objective function value of global pose optimization.

[0021] A second aspect of this invention provides a structured light point cloud stitching device for medical navigation, comprising: The acquisition unit is used to acquire point cloud data of the current frame; The first processing unit is used to align the current frame point cloud data into the point cloud model to obtain the pose corresponding to the current frame point cloud data; wherein the point cloud model is generated based on historical frame point cloud data; The second processing unit is used to acquire a pose graph and, based on the pose graph, the pose corresponding to the current frame point cloud data, the current frame point cloud data, the historical frame point cloud data, and a preset pose constraint algorithm, establish a relative pose constraint corresponding to the current frame point cloud data. The third processing unit is used to establish relative pose constraints between all the point cloud data, obtain a global pose map, and obtain the optimized pose corresponding to the point cloud data based on the global pose map and a preset global pose optimization algorithm. The fourth processing unit is used to perform pose transformation on the point cloud data according to the optimized pose corresponding to the point cloud data, and to stitch the pose-transformed point cloud data to obtain the point cloud stitching result.

[0022] A third aspect of the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the structured light point cloud stitching method for medical navigation as described in the first aspect above.

[0023] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the structured light point cloud stitching method for medical navigation described in the first aspect above.

[0024] In this embodiment, non-contact structured light acquisition is used, avoiding soft tissue deformation and artificial jitter noise caused by traditional probe contact, providing high-quality, high-density raw local point cloud data for subsequent processing. By aligning each frame to the cumulative point cloud model generated based on historical frames, rather than just aligning it to the previous frame, the propagation of local errors is effectively suppressed. Point-to-surface ICP and robust kernel functions are used to enhance the stability and noise resistance of registration. While establishing constraints on adjacent frames, this scheme introduces constraints on non-adjacent frames through loop closure detection, and uses a global pose graph optimization algorithm to adjust the pose of all frames simultaneously, making the entire point cloud model self-consistently closed on a global scale. This mechanism fundamentally solves the error accumulation problem and ensures the global geometric consistency of the reconstructed model. Using all constraints in the pose graph (including adjacent frame constraints and loop closure constraints), the optimal pose of each frame is solved in one go by minimizing the weighted error sum. This step improves the local registration results to a globally consistent level, completely eliminating the drift phenomenon of frame-by-frame stitching. Based on the optimized pose, all frame point clouds are transformed to a unified coordinate system and fused. Combined with post-processing filtering, a complete, continuous, non-redundant, and high-precision 3D head surface model is generated, which can be directly used for spatial registration in medical navigation. This method is entirely based on the geometric information of the point cloud itself, requiring no external optical positioning system, and exhibits strong system robustness. The introduction of global optimization and loop closure detection fundamentally solves the problem of uneven accuracy in traditional methods. The final output point cloud model has significant advantages such as high global consistency, uniform accuracy across the entire region, and no need for manual intervention, providing a reliable 3D spatial reference for transcranial magnetic stimulation and neurosurgical navigation. Attached Figure Description

[0025] Figure 1 This is a schematic flowchart of the structured light point cloud stitching method for medical navigation provided in the first embodiment of this application; Figure 2 This is a schematic flowchart of steps S21 to S22 of the structured light point cloud stitching method for medical navigation provided in the first embodiment of this application; Figure 3 This is a schematic flowchart of steps S31 to S32 of the structured light point cloud stitching method for medical navigation provided in the first embodiment of this application; Figure 4 This is a schematic flowchart of steps S321 to S322 in the structured light point cloud stitching method for medical navigation provided in the first embodiment of this application; Figure 5 This is a schematic flowchart of steps S41 to S44 of the structured light point cloud stitching method for medical navigation provided in the first embodiment of this application; Figure 6 This is a schematic flowchart of steps S421 to S422 in the structured light point cloud stitching method for medical navigation provided in the first embodiment of this application; Figure 7 This is a schematic flowchart of steps S431 to S432 in the structured light point cloud stitching method for medical navigation provided in the first embodiment of this application; Figure 8 This is a schematic diagram of a structured light point cloud stitching device for medical navigation provided in the second embodiment of this application; Figure 9 This is a schematic diagram of a computer device provided in the third embodiment of the present invention.

[0026] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0028] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this specification means the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any modules and all combinations of one or more associated listed items.

[0029] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0030] Reference Figure 1 The first embodiment of this application provides a structured light point cloud stitching method for medical navigation, including the following steps S1-S6: S1: Obtain the point cloud data of the current frame.

[0031] In this embodiment, the current frame point cloud data is point cloud data acquired through structured light. Specifically, it is a set of three-dimensional spatial points reconstructed using triangulation after a single image of the patient's affected area is taken with a structured light camera. Each point contains coordinates (x, y, z) in the camera's local coordinate system.

[0032] For example, if point cloud data of the head needs to be acquired, a structured light camera can slowly move around the patient's head. The camera acquires data continuously at a fixed frame rate (e.g., 30 frames per second). Each frame describes only a local region of the head (e.g., the left temporal region or a local area of ​​the top of the head). During acquisition, it is possible to ensure that there is more than 30% spatial overlap between adjacent frames for subsequent registration.

[0033] In transcranial magnetic stimulation navigation, a point cloud frame directly in front of the patient's face is first acquired as frame p1. Then, the camera is slowly moved towards the top of the head to acquire the second frame p2. At this time, p2 includes part of the forehead and the starting area of ​​the top of the head, and overlaps with p1.

[0034] The structured light non-contact acquisition method avoids soft tissue deformation and human-induced jitter noise caused by traditional probe contact, and quickly obtains high-density local point clouds, providing high-quality raw data for subsequent full head reconstruction.

[0035] S2: Align the current frame point cloud data to the point cloud model to obtain the pose corresponding to the current frame point cloud data; wherein, the point cloud model is generated based on historical frame point cloud data.

[0036] Aligning the current frame point cloud data with the point cloud model involves matching the currently acquired point cloud data with the existing point cloud model. An optimization method is used to find a rigid body transformation (rotation + translation) that maximizes the fit between the transformed current frame point cloud and the surface shape of the point cloud model. The rigid body transformation parameters, which transform the current frame point cloud from its local coordinate system to the global reference coordinate system, are then calculated; these parameters represent the pose of the current frame point cloud data.

[0037] The point cloud model is generated based on historical frame point cloud data. Historical frame point cloud data refers to one or more frames of point cloud data that have been acquired, processed, and whose poses have been determined before the current frame. It typically includes the first frame, the second frame, ... up to the (i-1)th frame.

[0038] In one embodiment, S2 may include S21~S22, such as Figure 2 As shown, S21~S22 are as follows: S21: Obtain the corresponding point in the point cloud model for each point in the current frame point cloud data and the normal vector of the plane where the corresponding point is located.

[0039] Obtain the corresponding point in the point cloud model and the normal vector of the plane containing each point in the current frame point cloud data.

[0040] In this context, for a given point in the current frame's point cloud, the nearest 3D point in the point cloud model is found through nearest neighbor search. Corresponding points represent matching pairs in two sets of point clouds that describe the same physical surface location. The normal vector represents the direction vector (usually a unit vector) of the local tangent plane containing the corresponding point in the point cloud model. It can be obtained by fitting principal component analysis (PCA) of all points in the neighborhood of that point, reflecting the surface orientation at that location.

[0041] Current frame point cloud data P i = {p1, p2, ..., p m}, for P i Each point p in k Search for its nearest neighbor q in the point cloud model k Search methods can employ spatial index structures such as KD-trees (K-Dimensional Trees) or octrees.

[0042] Get q k Then, q is further calculated. k The normal vector n of the local plane k You can choose q. k For all points within the neighborhood radius (e.g., the k=20 nearest points within a radius r = 1mm), calculate the covariance matrix of these points, take the eigenvector corresponding to the smallest eigenvalue as the normal vector, and normalize it.

[0043] By establishing a correspondence between each current frame point and the point cloud model, and obtaining the normal vector at the corresponding point, the necessary geometric information is provided for subsequent point-to-surface error calculation. The introduction of the normal vector allows the registration to consider not only the point position but also the local surface orientation, making it more stable and faster to converge than simple point-to-point matching.

[0044] S22: Based on the current frame point cloud data, the corresponding point of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm, the pose corresponding to the current frame point cloud data is obtained; wherein, the preset point cloud alignment algorithm is used to minimize the distance error between each point and the tangent plane of the corresponding point.

[0045] In this embodiment, a preset point cloud alignment algorithm is used to minimize the distance error between each point and the tangent plane of the corresponding point. This preset point cloud alignment algorithm can be a point-to-plane ICP algorithm.

[0046] Based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane containing the corresponding point, and a preset point cloud alignment algorithm, the pose corresponding to the current frame point cloud data is obtained. After finding the corresponding points of each point in the current frame in the point cloud model and the normal vector of the plane containing that corresponding point in step S21, a preset point cloud alignment algorithm is used to continuously adjust the pose of the current frame point cloud, minimizing the vertical distance from each point in the current frame to the tangent plane containing its corresponding point after pose transformation, ultimately solving for the optimal pose.

[0047] In one implementation, a point cloud alignment algorithm based on minimizing the distance error from a point to a tangent plane is employed. Through mathematical modeling, the registration problem between the current frame point cloud and the point cloud model is transformed into a nonlinear least squares optimization problem, which solves for the optimal pose that minimizes the error. S22 may specifically include: obtaining the pose corresponding to the current frame point cloud data based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane containing the corresponding points, and a preset point cloud alignment algorithm; The error function in the preset point cloud alignment algorithm is:

[0048] Indicates the first Frame point cloud data, the first Frame point cloud data refers to the current frame point cloud data. Indicates the first Frame point cloud data The first in One point, Indicates the first The corresponding point of each point in the point cloud model Indicates the first The normal vector of the plane containing the corresponding points of each point. Indicates the first The pose corresponding to the point cloud data of the first frame, the The pose corresponding to the point cloud data of the frame is the pose corresponding to the point cloud data of the current frame. This represents the distance from each point to the tangent plane of the corresponding point. The error is represented by a pre-defined point cloud alignment algorithm. The pose that minimizes the error.

[0049] Specifically, this embodiment uses the standard point-to-plane ICP algorithm, which includes the current frame point cloud data, the corresponding points on the map, the normal vectors of the corresponding points on the map plane, and a "preset point cloud alignment algorithm". The goal is to find an optimal pose that minimizes the error function.

[0050] In another implementation, S22 may specifically include: obtaining the pose corresponding to the current frame point cloud data based on the current frame point cloud data, the corresponding point of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm; The optimized error function in the preset point cloud alignment algorithm is as follows:

[0051]

[0052] Indicates the first Frame point cloud data, the first Frame point cloud data refers to the current frame point cloud data. Indicates the first Frame point cloud data The first in One point, Indicates the first The corresponding point of each point in the point cloud model Indicates the first The normal vector of the plane containing the corresponding points of each point. Indicates the first The pose corresponding to the point cloud data of the first frame, the The pose corresponding to the point cloud data of the frame is the pose corresponding to the point cloud data of the current frame. This represents the distance from each point to the tangent plane of the corresponding point. for Simplified representation, Indicates the distance threshold. The value represents the optimized error, obtained based on a preset point cloud alignment algorithm. The pose that minimizes the error after optimization.

[0053] In this embodiment, a preset point cloud alignment algorithm is introduced. Specifically, it can be a robust kernel function, which constructs an error equation using the geometric relationship between points and planes. The Huber kernel function is used to robustly process the error equation, preventing incorrect matching points from interfering with positioning accuracy.

[0054] In this embodiment, the distance error from a point to the tangent plane is used instead of a simple point-to-point distance error. This makes the registration process less sensitive to uneven sampling density in the point cloud model, resulting in faster convergence. The tangent plane information utilizes the orientation features of local surfaces, which better conforms to the geometric constraints of real curved surfaces than simple point position matching, thus achieving higher registration accuracy. This error metric is robust to noise and outliers because outliers often cannot form stable tangent plane constraints, thus having a smaller impact on the optimization process. By iteratively optimizing to gradually approach the optimal pose, it can handle cases with large initial pose deviations, ensuring a high registration success rate.

[0055] S3: Obtain the pose graph, and establish the relative pose constraints corresponding to the current frame point cloud data based on the pose graph, the pose corresponding to the current frame point cloud data, the current frame point cloud data, the historical frame point cloud data, and the preset pose constraint algorithm; wherein, the nodes in the pose graph represent the poses corresponding to the point cloud data, and the edges in the pose graph represent the relative pose constraints between the point cloud data.

[0056] Obtain the pose graph, which is a graph data structure used to represent the spatial constraints between point clouds across multiple frames. Nodes in the pose graph represent the poses of the point cloud data, and edges in the pose graph represent the relative pose constraints between the point cloud data.

[0057] Using the pose corresponding to the current frame point cloud data, the original point cloud data of the current frame, the point cloud data of all historical frames, and the preset pose constraint algorithm, relative pose constraints between the current frame and related frames are established, and these constraints are added to the pose graph in the form of edges.

[0058] Specifically, these relative pose constraints are not restricted here. For example, they can include two types: adjacent frame pose constraints with the previous frame and loop pose constraints with historical non-adjacent frames.

[0059] In one embodiment, the relative pose constraint includes adjacent frame pose constraint and loop closure pose constraint, and S3 may include S31~S32, such as Figure 3 As shown, S31~S32 are as follows: S31: Add the pose corresponding to the current frame point cloud data to the pose graph and establish the adjacent frame pose constraint corresponding to the current frame point cloud data; wherein, the adjacent frame pose constraint is the relative pose constraint between the current frame point cloud data and the previous frame point cloud data.

[0060] Obtain the pose of the current frame, that is, the pose corresponding to the point cloud data of the current frame (the i-th frame).

[0061] The pose corresponding to the current frame's point cloud data is encapsulated into a node. If the pose graph is currently empty (i.e., i=1), an empty graph is initialized, and the currently encapsulated node is added as the first node. Typically, the pose of the first frame is set to the identity matrix, serving as the global reference coordinate system.

[0062] Obtain the node and pose of the previous frame. If i>1, obtain the node and pose of the previous frame from the pose graph.

[0063] The adjacent frame pose constraint is the relative pose constraint between the current frame point cloud data and the previous frame point cloud data. It refers to the relative pose relationship established between two temporally consecutive frames (frame i-1 and frame i). This constraint reflects the motion estimation of the camera at consecutive moments and is usually obtained by directly registering the original point clouds of two adjacent frames.

[0064] After obtaining the current frame pose, it is added as a new node to the pose graph. The current frame pose is packaged into a node and assigned a unique identifier, for example, the node number is equal to the frame number. If the pose graph is still empty (i.e., the current frame is the first frame), an empty pose graph is created first, and then the first frame node is added to it.

[0065] Determine if the current frame is the first frame. If not, establish an edge between the current frame node and the previous frame node. The edge starts at the previous frame node and ends at the current frame node. The observation value of the edge is the relative motion from the pose of the previous frame to the pose of the current frame. The edge information matrix is ​​set based on the overlap and registration error of the point clouds in the two frames. The larger the overlap area and the smaller the registration error, the larger the value of the information matrix, indicating that the constraint is more reliable.

[0066] The newly created current frame node and the edge connecting the current frame to the previous frame are added to the pose graph's data structure. The pose graph maintains a list of nodes and a list of edges for easy access by subsequent global optimization algorithms.

[0067] After the addition is complete, the number of nodes in the pose graph increases by one, and the number of edges also increases by one. At this point, the pose graph contains pose nodes from the first frame to the current frame, as well as constraint edges between all adjacent frames.

[0068] By constructing a pose graph and establishing constraints between adjacent frames, the local motion relationships between frames are explicitly modeled, providing a basic topological structure for subsequent global optimization. Adjacency constraints ensure pose continuity over short periods, avoiding abrupt local changes.

[0069] S32: Based on the current frame point cloud data, the historical frame point cloud data, and the preset loop closure detection algorithm, establish the loop closure pose constraint corresponding to the current frame point cloud data.

[0070] Using the currently acquired local point cloud data and the previously acquired historical point cloud data, a preset loop closure detection algorithm is used to determine whether there is spatial overlap between the two. If there is significant overlap, a loop closure pose constraint is established between the current frame and the corresponding historical frame.

[0071] This constraint can span multiple intermediate frames, directly associating the current frame with a historical frame, thereby providing crucial information for subsequent global optimization to correct accumulated errors.

[0072] When the camera returns to a previously scanned area during the scanning process, it is necessary to identify this "loop" phenomenon and record the spatial relationship between the current frame and a previous frame.

[0073] In this embodiment, the pose of each frame is used as a node, and the relative poses between adjacent frames are used as edges to construct a graph structure, providing a clear mathematical framework for subsequent global optimization. Adjacent frame constraints ensure the continuity and smoothness of local motion. By detecting the spatial overlap between the current frame and historical non-adjacent frames, loop closure constraints are automatically established. This constraint spans multiple intermediate frames, forming a closed-loop structure, fundamentally suppressing error accumulation in long sequence splicing, significantly improving accuracy far from the starting region, and achieving a uniform distribution of accuracy.

[0074] In one embodiment, S32 may include S321~S322, such as Figure 4 As shown, S321~S322 are as follows: S321: Perform spatial overlap detection on the current frame point cloud data and the historical frame point cloud data to obtain spatial overlap information.

[0075] Spatial overlap detection is performed on the current frame point cloud data and historical frame point cloud data to obtain spatial overlap information. Spatial overlap detection needs to determine whether the current frame point cloud and a certain historical frame point cloud share a common physical region in 3D space, and the degree of overlap. Overlap detection is usually performed in the same global reference coordinate system, so both frame point clouds need to be transformed into this coordinate system. Spatial overlap information is data describing the degree of overlap between the two frame point clouds, which can be quantitative indicators such as the area ratio of the overlapping region, the number of overlapping points, the distance between the centers of the two frame point clouds, or the overlap rate obtained through registration.

[0076] Acquire the point cloud data and its pose for the current frame. The point cloud data for the current frame is the original locally acquired point cloud, located in the camera's local coordinate system.

[0077] Acquire historical frame point cloud data and their poses. Historical frame point cloud data refers to the point cloud data of one or all frames that have been acquired and processed before the current frame. Each historical point cloud has its own corresponding pose.

[0078] Perform a spatial uniform transformation. To determine whether two point cloud frames overlap spatially, they need to be transformed to the same coordinate system. A global reference coordinate system is typically chosen as the uniform space. The specific steps are as follows: Transform the current frame point cloud data according to its pose to obtain a point cloud in the global coordinate system. Transform the historical frame point cloud data according to its pose to obtain a point cloud in the global coordinate system.

[0079] Calculate spatial overlap information. Under a unified coordinate system, there are several methods to quantify the degree of overlap between two point cloud frames: Center distance method: Calculate the Euclidean distance between the center point of the current frame point cloud and the center point of the historical frame point cloud. If the distance is less than a preset threshold (e.g., 5 cm), it is considered that there may be overlap.

[0080] Bounding box intersection method: Calculate whether the axis-aligned bounding boxes of two point clouds intersect, and calculate the proportion of the intersection volume to their respective volumes.

[0081] Nearest neighbor point statistics method: For each point after the current frame is transformed, search for the nearest neighbor point in the point cloud after the historical frames are transformed, and count the number of points whose distance is less than a certain threshold (e.g., 1 mm). The proportion of this number to the total number of points in the current frame is the overlap rate.

[0082] Registration error method: Directly perform fast ICP registration between the current frame and historical frames, and record the root mean square error after registration. The smaller the error, the higher the degree of overlap.

[0083] Finally, spatial overlap information is output. The calculated quantitative indicators (such as overlap rate, center distance, registration error, etc.) are used as spatial overlap information.

[0084] S322: If the spatial overlap information exceeds a preset spatial overlap threshold, then establish a loop pose constraint between the current frame point cloud data and the corresponding historical frame point cloud data.

[0085] The spatial overlap information is compared with a preset spatial overlap threshold. The preset threshold is a constant set by the system in advance, such as an overlap rate threshold of 30% or a center distance threshold of 5 cm. If the spatial overlap information is greater than or equal to the threshold, a valid loop closure candidate is considered to have been detected.

[0086] To prevent false matches, a precise registration can be performed between the candidate current frame and historical frames. Specifically, point-to-surface ICP registration is performed between the original point clouds of the current frame and the original point clouds of the historical frames, and the precise relative poses and root mean square error (RMSE) of the registration are calculated. If the RMSE is less than a preset verification threshold (e.g., 0.3 mm), it is considered a valid loop closure; otherwise, the loop closure is discarded.

[0087] Calculate the relative pose. Using the registration results from the geometric verification above, obtain the relative pose transformation from the historical frame to the current frame. This relative pose describes: starting from the pose of the historical frame, what rotations and translations are needed to reach the pose of the current frame.

[0088] Establish loop closure pose constraint edges. In the pose graph, find the nodes corresponding to historical frames and the nodes corresponding to the current frame, and add an edge between these two nodes. The observation value of this edge is the relative pose calculated in the previous step. At the same time, assign a higher information matrix weight to this edge, usually higher than the weight of the adjacent frame constraint, because the loop closure constraint has a stronger corrective effect on global optimization.

[0089] In this embodiment, by detecting spatial overlap and establishing loop closure constraints, a closed-loop structure is formed, effectively eliminating drift errors in long sequence stitching. This allows non-facial regions such as the back of the head and top of the head to be bound to precise facial regions through loop closures, achieving consistent accuracy across the entire head. Based entirely on the geometric information of the point cloud itself, without relying on an optical positioning system, the system is simplified and highly robust. Automatic loop closure identification and constraint establishment require no manual intervention, improving operational efficiency. As a strong constraint, the loop closure constraint guides global optimization to quickly converge to a more accurate pose.

[0090] S4: Until the relative pose constraints between all the point cloud data are established, a global pose graph is obtained, and the optimized pose corresponding to the point cloud data is obtained according to the global pose graph and the preset global pose optimization algorithm.

[0091] Obtain the global pose graph, which is a complete pose graph containing all nodes of all acquired frames and all established constraint edges (including adjacent frame constraints and loop closure constraints). The global pose graph contains the nodes and edges corresponding to all point cloud data.

[0092] Based on the global pose graph and the preset global pose optimization algorithm, the optimized pose corresponding to the point cloud data is obtained. The preset global pose optimization algorithm is used to minimize the global inter-frame pose error.

[0093] In one implementation, the global pose graph includes all nodes and edges corresponding to the point cloud data. A preset global pose optimization algorithm is used to minimize the global inter-frame pose error. Using the constructed global pose graph, the preset global pose optimization algorithm performs a one-time global adjustment of the pose of each frame's point cloud, minimizing the overall error of all inter-frame constraints, thus obtaining the optimized pose. Specifically, based on the relative positional relationships between all adjacent frames and loopback frames, the most reasonable position and orientation of each frame are calculated backwards, eliminating cumulative drift.

[0094] In one implementation, the step of acquiring a global pose map and obtaining the optimized pose corresponding to the point cloud data based on the global pose map and a preset global pose optimization algorithm may include S41~S44, such as... Figure 5 As shown, S41~S44 are as follows: S41: Obtain the global pose graph.

[0095] S42: Based on the poses corresponding to all the point cloud data in the global pose graph and the relative pose constraints between all the point cloud data, obtain the derived pose constraints corresponding to the relative pose constraints.

[0096] Based on the poses of all point cloud data in the global pose graph and the relative pose constraints between all point cloud data, the derived pose constraints corresponding to the relative pose constraints are obtained. For each constraint edge in the pose graph (i.e., the relative pose observation between two nodes), a "derived" relative pose is calculated by using the current poses of these two nodes (the current estimated values ​​in the optimization process).

[0097] In one embodiment, S42 may include S421~S422, such as Figure 6 As shown, S421~S422 are as follows: S421: From the global pose graph Get the first Frame point cloud data and the first Relative pose constraints between frame point cloud data and from the global pose graph Obtain the first Pose corresponding to frame point cloud data and the first Pose corresponding to frame point cloud data .

[0098] From the global pose graph Get the first Frame point cloud data and the first Relative pose constraints between frame point cloud data The global pose graph G is a data structure, denoted as G(G). Global pose graph Nodes in The global pose graph represents the pose corresponding to the point cloud data. The edge in This represents the relative pose constraints between the point cloud data. Each node V i The pose T of the point cloud corresponding to the i-th frame i Each edge connects node m and node n, and includes a relative pose constraint. .

[0099] S422: According to the above... Pose corresponding to frame point cloud data and the first Pose corresponding to frame point cloud data The relative pose constraint is obtained. Corresponding derivation of pose constraints ;in, , The global pose graph Nodes in The global pose graph represents the pose corresponding to the point cloud data. The edge in This represents the relative pose constraints between the point cloud data.

[0100] Using the first Pose corresponding to frame point cloud data and the Pose corresponding to frame point cloud data This leads to the derivation that, under the current estimate, from the th Frame to the The relative pose that a frame should have is denoted as .

[0101] in, , The global pose graph Nodes in The global pose graph represents the pose corresponding to the point cloud data. The edge in This represents the relative pose constraints between the point cloud data.

[0102] If the current estimate and If it is completely accurate, then the derivation is... It should equal If there is an error, the two will differ, and this derivation of pose constraints will be incorrect. With relative pose constraints The difference between them represents the error that needs to be optimized in subsequent steps.

[0103] In this embodiment, a unified and clear error calculation framework is established, so that adjacent frame constraints and loop closure constraints are processed in the same way; the comparison between the derived pose and the observed value has a clear geometric meaning and can intuitively reflect the deviation of the current pose estimation; it naturally supports iterative optimization, and dynamically adjusts the derived value as the node pose is updated, guiding the algorithm to converge stably; it has high computational efficiency, requires no additional storage, and each side operation is independent of each other, making it easy to parallelize and accelerate.

[0104] S43: Calculate the inter-frame pose error based on the relative pose constraints and the derived pose constraints corresponding to the relative pose constraints, and vectorize the inter-frame pose error to obtain the vectorized result.

[0105] The relative pose constraint and the derived pose constraint corresponding to the relative pose constraint are compared, and the difference between the two (i.e., the inter-frame pose error) is calculated. Then, this difference is converted from the form of rigid body transformation matrix into vector form to obtain the vectorized processing result.

[0106] In one embodiment, S43 may include S431-S432, such as... Figure 7 As shown, S431~S432 are as follows: S431: According to the aforementioned... Frame point cloud data and the first Relative pose constraints between frame point cloud data and the relative pose constraint Corresponding derivation of pose constraints The inter-frame pose error is obtained. ;in, , express The reverse.

[0107] For each constraint edge in the pose graph, the original observed relative pose is... According to the current pose Estimated relative pose By comparing the two, we obtain a matrix representing the difference between them; this matrix is ​​the inter-frame pose error. .in, , express The reverse.

[0108] Multiply the derived pose by the inverse of the original observed pose. If the current pose estimate is perfectly accurate, the derived pose should equal the observed pose, in which case the error is an identity matrix (i.e., no error). If there is a bias, the error matrix will deviate from the identity matrix, and the degree of deviation reflects the magnitude of the error in the current estimate.

[0109] S432: Regarding the inter-frame pose error Perform vectorization processing to obtain the vectorized result. ;in, , This indicates the inter-frame pose error. Perform a mapping from Lie groups to Lie algebras.

[0110] Logarithmic mapping is used to convert inter-frame pose errors Mapping to the corresponding Lie algebra vector yields the vectorized result. The first three components of this vector represent rotational error (usually in radians), and the last three components represent translational error (usually in millimeters). , This indicates the inter-frame pose error. Perform a mapping from Lie groups to Lie algebras.

[0111] After vectorization, the matrix error, which was originally difficult to calculate directly, becomes an ordinary vector that can be added, subtracted, have its norm calculated, and multiplied by weights.

[0112] In this embodiment, the vector error obtained by logarithmic mapping approximates the linear error in Euclidean space under small error conditions, and the function is smooth and differentiable, which is beneficial to the rapid convergence of the optimization algorithm. The six-dimensional vector simultaneously contains rotation and translation errors, which do not need to be processed separately, simplifying the construction of the optimization objective function. The vectorized result can be directly multiplied with the information matrix to achieve differentiated weighting for different constraints (such as adjacent frames and loop-back frames), thereby increasing the contribution of high-confidence constraints.

[0113] S44: Based on the vectorization processing result and the preset global pose optimization objective function, obtain the optimized pose corresponding to the point cloud data.

[0114] The vectorized result is substituted into the preset global pose optimization objective function, and the set of poses that minimizes the objective function value is solved by mathematical optimization method. This set of poses is the optimized pose corresponding to the point cloud data.

[0115] Specifically, the optimal pose is solved by a preset global pose optimization objective function. S54 may include: obtaining the optimized pose corresponding to the point cloud data based on the vectorization processing result and a preset global pose optimization objective function; The preset global pose optimization objective function is:

[0116] Represents the global pose graph The Middle The pose corresponding to the frame point cloud data express The reverse, Represents the global pose graph The Middle The pose corresponding to the frame point cloud data Represents the global pose graph The Middle Frame point cloud data and the first Relative pose constraints between frame point cloud data Represents the relative pose constraint Corresponding derivation of pose constraints Indicates the first Frame point cloud data and the first The inter-frame pose error between frame point cloud data. This indicates the inter-frame pose error. Perform a mapping from Lie groups to Lie algebras. The vectorization result, the global pose graph Nodes in The global pose graph represents the pose corresponding to the point cloud data. The edge in This represents the relative pose constraints between the point cloud data. Indicates the first The node corresponding to the frame point cloud data and the first The edges between nodes corresponding to the frame point cloud data belong to The first Optimized pose corresponding to frame point cloud data The pose that minimizes the objective function value of global pose optimization.

[0117] S5: Perform pose transformation on the point cloud data according to the optimized pose corresponding to the point cloud data, and stitch the pose-transformed point cloud data together to obtain the point cloud stitching result.

[0118] The point cloud data is transformed into a pose based on the optimized pose. Pose transformation is the process of transforming the original i-th frame point cloud from the camera's local coordinate system to the global reference coordinate system using the optimized pose. For each 3D point in the original i-th frame point cloud, its coordinates are multiplied by the optimized pose corresponding to that frame (rotation matrix plus translation vector) to obtain the new coordinates of that point in the global coordinate system.

[0119] For each frame of the original point cloud, perform the following operations sequentially: Retrieve the original point cloud data for that frame, where the coordinates of each point are local coordinates relative to the location of the structured light camera at that time. Retrieve the optimized pose corresponding to that frame, which contains a rotation matrix and a translation vector. For each point in that frame, transform it using the optimized pose: new coordinates = rotation matrix multiplied by the original coordinates, plus the translation vector. Store the transformed points in a new point cloud set. At this point, the point cloud for that frame has been moved to the correct position in the global reference coordinate system.

[0120] In this embodiment, non-contact structured light acquisition is used, avoiding soft tissue deformation and artificial jitter noise caused by traditional probe contact, providing high-quality, high-density raw local point cloud data for subsequent processing. By aligning each frame to the cumulative point cloud model generated based on historical frames, rather than just aligning it to the previous frame, the propagation of local errors is effectively suppressed. Point-to-surface ICP and robust kernel functions are used to enhance the stability and noise resistance of registration. While establishing constraints on adjacent frames, this scheme introduces constraints on non-adjacent frames through loop closure detection, and uses a global pose graph optimization algorithm to adjust the pose of all frames simultaneously, making the entire point cloud model self-consistently closed on a global scale. This mechanism fundamentally solves the error accumulation problem and ensures the global geometric consistency of the reconstructed model. Using all constraints in the pose graph (including adjacent frame constraints and loop closure constraints), the optimal pose of each frame is solved in one go by minimizing the weighted error sum. This step improves the local registration results to a globally consistent level, completely eliminating the drift phenomenon of frame-by-frame stitching. Based on the optimized pose, all frame point clouds are transformed to a unified coordinate system and fused. Combined with post-processing filtering, a complete, continuous, non-redundant, and high-precision 3D head surface model is generated, which can be directly used for spatial registration in medical navigation. This method is entirely based on the geometric information of the point cloud itself, requiring no external optical positioning system, and exhibits strong system robustness. The introduction of global optimization and loop closure detection fundamentally solves the problem of uneven accuracy in traditional methods. The final output point cloud model has significant advantages such as high global consistency, uniform accuracy across the entire region, and no need for manual intervention, providing a reliable 3D spatial reference for transcranial magnetic stimulation and neurosurgical navigation.

[0121] Reference Figure 8 The second embodiment of the present invention also provides a structured light point cloud stitching device for medical navigation, comprising: The first processing unit 10 is used to align the current frame point cloud data into the point cloud model to obtain the pose corresponding to the current frame point cloud data; wherein the point cloud model is generated based on historical frame point cloud data; The second processing unit 20 is used to acquire a pose map and, based on the pose map, the pose corresponding to the current frame point cloud data, the current frame point cloud data, the historical frame point cloud data, and a preset pose constraint algorithm, establish a relative pose constraint corresponding to the current frame point cloud data. The third processing unit 30 is used to establish relative pose constraints between all the point cloud data, obtain a global pose map, and obtain the optimized pose corresponding to the point cloud data according to the global pose map and a preset global pose optimization algorithm. The fourth processing unit 40 is used to perform pose transformation on the point cloud data according to the optimized pose corresponding to the point cloud data, and to stitch the pose-transformed point cloud data to obtain a point cloud stitching result.

[0122] Further, the first processing unit is specifically used for: Obtain the corresponding point in the point cloud model for each point in the current frame point cloud data and the normal vector of the plane containing the corresponding point; Based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm, the pose corresponding to the current frame point cloud data is obtained; wherein, the preset point cloud alignment algorithm is used to minimize the distance error between each point and the tangent plane of the corresponding point.

[0123] Furthermore, the first processing unit is specifically used for: Based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm, the pose corresponding to the current frame point cloud data is obtained. The error function in the preset point cloud alignment algorithm is:

[0124] Indicates the first Frame point cloud data, the first Frame point cloud data refers to the current frame point cloud data. Indicates the first Frame point cloud data The first in One point, Indicates the first The corresponding point of each point in the point cloud model Indicates the first The normal vector of the plane containing the corresponding points of each point. Indicates the first The pose corresponding to the point cloud data of the first frame, the The pose corresponding to the point cloud data of the frame is the pose corresponding to the point cloud data of the current frame. This represents the distance from each point to the tangent plane of the corresponding point. The error is represented by a pre-defined point cloud alignment algorithm. The pose that minimizes the error.

[0125] Furthermore, the first processing unit is specifically used for: Based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm, the pose corresponding to the current frame point cloud data is obtained. The optimized error function in the preset point cloud alignment algorithm is as follows:

[0126]

[0127] Indicates the first Frame point cloud data, the first Frame point cloud data refers to the current frame point cloud data. Indicates the first Frame point cloud data The first in One point, Indicates the first The corresponding point of each point in the point cloud model Indicates the first The normal vector of the plane containing the corresponding points of each point. Indicates the first The pose corresponding to the point cloud data of the first frame, the The pose corresponding to the point cloud data of the frame is the pose corresponding to the point cloud data of the current frame. This represents the distance from each point to the tangent plane of the corresponding point. for Simplified representation, Indicates the distance threshold. The value represents the optimized error, obtained based on a preset point cloud alignment algorithm. The pose that minimizes the error after optimization.

[0128] Furthermore, the relative pose constraint includes adjacent frame pose constraints and loop closure pose constraints, and the second processing unit is specifically used for: The pose corresponding to the current frame point cloud data is added to the pose graph to establish the adjacent frame pose constraint corresponding to the current frame point cloud data; wherein, the adjacent frame pose constraint is the relative pose constraint between the current frame point cloud data and the previous frame point cloud data. Based on the current frame point cloud data, the historical frame point cloud data, and the preset loop closure detection algorithm, establish the loop closure pose constraint corresponding to the current frame point cloud data.

[0129] Furthermore, the second processing unit is specifically used for: Spatial overlap detection is performed on the current frame point cloud data and the historical frame point cloud data to obtain spatial overlap information; If the spatial overlap information exceeds a preset spatial overlap threshold, a loop pose constraint is established between the current frame point cloud data and the corresponding historical frame point cloud data.

[0130] Furthermore, the third processing unit is specifically used for: Obtain the global pose graph; Based on the poses corresponding to all point cloud data in the global pose graph and the relative pose constraints between all point cloud data, the derived pose constraints corresponding to the relative pose constraints are obtained. Based on the relative pose constraints and the derived pose constraints corresponding to the relative pose constraints, the inter-frame pose error is calculated, and the inter-frame pose error is vectorized to obtain the vectorized result. Based on the vectorization processing result and the preset global pose optimization objective function, the optimized pose corresponding to the point cloud data is obtained.

[0131] Furthermore, the third processing unit is specifically used for: From the global pose graph Get the first Frame point cloud data and the first Relative pose constraints between frame point cloud data and from the global pose graph Obtain the first Pose corresponding to frame point cloud data and the first Pose corresponding to frame point cloud data ; According to the first Pose corresponding to frame point cloud data and the first Pose corresponding to frame point cloud data The relative pose constraint is obtained. Corresponding derivation of pose constraints ;in, , The global pose graph Nodes in The global pose graph represents the pose corresponding to the point cloud data. The edge in This represents the relative pose constraints between the point cloud data.

[0132] Furthermore, the third processing unit is specifically used for: According to the first Frame point cloud data and the first Relative pose constraints between frame point cloud data and the relative pose constraint Corresponding derivation of pose constraints The inter-frame pose error is obtained. ;in, , express The reverse; For the inter-frame pose error Perform vectorization processing to obtain the vectorized result. ;in, , This indicates the inter-frame pose error. Perform a mapping from Lie groups to Lie algebras.

[0133] Furthermore, the third processing unit is specifically used for: Based on the vectorization processing result and the preset global pose optimization objective function, the optimized pose corresponding to the point cloud data is obtained; The preset global pose optimization objective function is:

[0134] Represents the global pose graph The Middle The pose corresponding to the frame point cloud data express The reverse, Represents the global pose graph The Middle The pose corresponding to the frame point cloud data Represents the global pose graph The Middle Frame point cloud data and the first Relative pose constraints between frame point cloud data Represents the relative pose constraint Corresponding derivation of pose constraints Indicates the first Frame point cloud data and the first The inter-frame pose error between frame point cloud data. This indicates the inter-frame pose error. Perform a mapping from Lie groups to Lie algebras. The vectorization result, the global pose graph Nodes in The global pose graph represents the pose corresponding to the point cloud data. The edge in This represents the relative pose constraints between the point cloud data. Indicates the first The node corresponding to the frame point cloud data and the first The edges between nodes corresponding to the frame point cloud data belong to The first Optimized pose corresponding to frame point cloud data The pose that minimizes the objective function value of global pose optimization.

[0135] Reference Figure 9The third embodiment of the present invention also provides a computer device, the internal structure of which can be as follows: Figure 9 As shown. Figure 9 As shown, the computer device 9 in this embodiment includes: a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and executable on the processor 90, such as a structured light point cloud stitching program for medical navigation. When the processor 90 executes the computer program 92, it implements the steps in the various embodiments of the structured light point cloud stitching method for medical navigation described above, for example... Figure 1 Steps 101 to 105 are shown. Alternatively, when the processor 90 executes the computer program 92, it implements the functions of each module in the above-described device embodiments, for example... Figure 8 The functions of modules 10 to 50 are shown.

[0136] For example, the computer program 92 can be divided into one or more modules / units, which are stored in the memory 91 and executed by the processor 90 to complete this application. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 92 in the computer device 9. For example, the computer program 92 can be divided into an acquisition unit, a first processing unit, a second processing unit, a third processing unit, and a fourth processing unit; The acquisition unit is used to acquire point cloud data of the current frame; The first processing unit is used to align the current frame point cloud data into the point cloud model to obtain the pose corresponding to the current frame point cloud data; wherein the point cloud model is generated based on historical frame point cloud data; The second processing unit is used to acquire a pose graph and, based on the pose graph, the pose corresponding to the current frame point cloud data, the current frame point cloud data, the historical frame point cloud data, and a preset pose constraint algorithm, establish a relative pose constraint corresponding to the current frame point cloud data. The third processing unit is used to establish relative pose constraints between all the point cloud data, obtain a global pose map, and obtain the optimized pose corresponding to the point cloud data based on the global pose map and a preset global pose optimization algorithm. The fourth processing unit is used to perform pose transformation on the point cloud data according to the optimized pose corresponding to the point cloud data, and to stitch the pose-transformed point cloud data to obtain the point cloud stitching result.

[0137] The computer device may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art will understand that... Figure 9This is merely an example of computer device 9 and does not constitute a limitation on computer device 9. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device may also include input / output devices, network access devices, buses, etc.

[0138] The processor 90 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0139] The memory 91 can be an internal storage unit of the computer device 9, such as a hard disk or RAM. The memory 91 can also be an external storage device of the computer device 9, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the computer device 9 can include both internal and external storage units. The memory 91 is used to store the computer program and other programs and data required by the computer device. The memory 91 can also be used to temporarily store data that has been output or will be output.

[0140] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0141] This application also provides a network device, which includes: at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, wherein the processor executes the computer program to implement the steps in any of the above method embodiments.

[0142] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0143] This application provides a computer program product that, when run on a mobile terminal, enables the mobile terminal to implement the steps described in the above-described method embodiments.

[0144] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0145] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0146] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0147] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0148] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0149] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0150] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A structured light point cloud stitching method for medical navigation, characterized in that, The method includes: Obtain the point cloud data of the current frame; Align the current frame point cloud data with the point cloud model to obtain the pose corresponding to the current frame point cloud data; wherein, the point cloud model is generated based on historical frame point cloud data; A pose graph is obtained, and relative pose constraints corresponding to the current frame point cloud data are established based on the pose graph, the pose corresponding to the current frame point cloud data, the current frame point cloud data, the historical frame point cloud data, and a preset pose constraint algorithm; wherein, the nodes in the pose graph represent the poses corresponding to the point cloud data, and the edges in the pose graph represent the relative pose constraints between the point cloud data. Until the relative pose constraints between all the point cloud data are established, a global pose graph is obtained, and the optimized pose corresponding to the point cloud data is obtained according to the global pose graph and the preset global pose optimization algorithm. The point cloud data is transformed into a pose based on the optimized pose corresponding to the point cloud data, and the transformed point cloud data is then stitched together to obtain the point cloud stitching result.

2. The structured light point cloud stitching method for medical navigation according to claim 1, characterized in that, Aligning the current frame point cloud data into the point cloud model to obtain the pose corresponding to the current frame point cloud data includes: Obtain the corresponding point in the point cloud model for each point in the current frame point cloud data and the normal vector of the plane containing the corresponding point; Based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm, the pose corresponding to the current frame point cloud data is obtained; wherein, the preset point cloud alignment algorithm is used to minimize the distance error between each point and the tangent plane of the corresponding point.

3. The structured light point cloud stitching method for medical navigation according to claim 2, characterized in that, The step of obtaining the pose corresponding to the current frame point cloud data based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane containing the corresponding point, and a preset point cloud alignment algorithm includes: Based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm, the pose corresponding to the current frame point cloud data is obtained. The error function in the preset point cloud alignment algorithm is: Indicates the first Frame point cloud data, the first Frame point cloud data refers to the current frame point cloud data. Indicates the first Frame point cloud data The first in One point, Indicates the first The corresponding point of each point in the point cloud model Indicates the first The normal vector of the plane containing the corresponding points of each point. Indicates the first The pose corresponding to the point cloud data of the first frame, the The pose corresponding to the point cloud data of the frame is the pose corresponding to the point cloud data of the current frame. This represents the distance from each point to the tangent plane of the corresponding point. The error is represented by a pre-defined point cloud alignment algorithm. The pose that minimizes the error.

4. The structured light point cloud stitching method for medical navigation according to claim 2, characterized in that, The step of obtaining the pose corresponding to the current frame point cloud data based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane containing the corresponding point, and a preset point cloud alignment algorithm includes: Based on the current frame point cloud data, the corresponding points of each point in the point cloud model, the normal vector of the plane where the corresponding point is located, and a preset point cloud alignment algorithm, the pose corresponding to the current frame point cloud data is obtained. The optimized error function in the preset point cloud alignment algorithm is as follows: Indicates the first Frame point cloud data, the first Frame point cloud data refers to the current frame point cloud data. Indicates the first Frame point cloud data The first in One point, Indicates the first The corresponding point of each point in the point cloud model Indicates the first The normal vector of the plane containing the corresponding points of each point. Indicates the first The pose corresponding to the point cloud data of the first frame, the The pose corresponding to the point cloud data of the frame is the pose corresponding to the point cloud data of the current frame. This represents the distance from each point to the tangent plane of the corresponding point. for Simplified representation, Indicates the distance threshold. The value represents the optimized error, obtained based on a preset point cloud alignment algorithm. The pose that minimizes the error after optimization.

5. The structured light point cloud stitching method for medical navigation according to any one of claims 1 to 4, characterized in that, The relative pose constraints include adjacent frame pose constraints and loop closure pose constraints. The step of obtaining the pose map, and establishing the relative pose constraints corresponding to the current frame point cloud data based on the pose map, the pose corresponding to the current frame point cloud data, the current frame point cloud data, historical frame point cloud data, and a preset pose constraint algorithm, includes the following steps: The pose corresponding to the current frame point cloud data is added to the pose graph to establish the adjacent frame pose constraint corresponding to the current frame point cloud data; wherein, the adjacent frame pose constraint is the relative pose constraint between the current frame point cloud data and the previous frame point cloud data. Based on the current frame point cloud data, the historical frame point cloud data, and the preset loop closure detection algorithm, establish the loop closure pose constraint corresponding to the current frame point cloud data.

6. The structured light point cloud stitching method for medical navigation according to claim 5, characterized in that, The step of establishing the loop closure pose constraint corresponding to the current frame point cloud data based on the current frame point cloud data, the historical frame point cloud data, and a preset loop closure detection algorithm includes: Spatial overlap detection is performed on the current frame point cloud data and the historical frame point cloud data to obtain spatial overlap information; If the spatial overlap information exceeds a preset spatial overlap threshold, a loop pose constraint is established between the current frame point cloud data and the corresponding historical frame point cloud data.

7. The structured light point cloud stitching method for medical navigation according to any one of claims 1 to 4, characterized in that, The step of obtaining the global pose map and, based on the global pose map and a preset global pose optimization algorithm, obtaining the optimized pose corresponding to the point cloud data includes: Obtain the global pose graph; Based on the poses corresponding to all point cloud data in the global pose graph and the relative pose constraints between all point cloud data, the derived pose constraints corresponding to the relative pose constraints are obtained. Based on the relative pose constraints and the derived pose constraints corresponding to the relative pose constraints, the inter-frame pose error is calculated, and the inter-frame pose error is vectorized to obtain the vectorized result. Based on the vectorization processing result and the preset global pose optimization objective function, the optimized pose corresponding to the point cloud data is obtained.

8. The structured light point cloud stitching method for medical navigation according to claim 7, characterized in that, The step of obtaining the derived pose constraint corresponding to the relative pose constraint based on the poses corresponding to all point cloud data in the global pose graph and the relative pose constraints between all point cloud data includes: From the global pose graph Get the first Frame point cloud data and the first Relative pose constraints between frame point cloud data and from the global pose graph Obtain the first Pose corresponding to frame point cloud data and the first Pose corresponding to frame point cloud data ; According to the first Pose corresponding to frame point cloud data and the first Pose corresponding to frame point cloud data The relative pose constraint is obtained. Corresponding derivation of pose constraints ;in, , The global pose graph Nodes in The global pose graph represents the pose corresponding to the point cloud data. The edge in This represents the relative pose constraints between the point cloud data.

9. The structured light point cloud stitching method for medical navigation according to claim 7, characterized in that, The step involves calculating the inter-frame pose error based on the relative pose constraints and the derived pose constraints corresponding to the relative pose constraints, and then vectorizing the inter-frame pose error to obtain the vectorized result, including: According to the first Frame point cloud data and the first Relative pose constraints between frame point cloud data and the relative pose constraint Corresponding derivation of pose constraints The inter-frame pose error is obtained. ;in, , express The reverse; For the inter-frame pose error Perform vectorization processing to obtain the vectorized result. ;in, , This indicates the inter-frame pose error. Perform a mapping from Lie groups to Lie algebras.

10. The structured light point cloud stitching method for medical navigation according to claim 7, characterized in that, The step of obtaining the optimized pose corresponding to the point cloud data based on the vectorization processing result and the preset global pose optimization objective function includes: Based on the vectorization processing result and the preset global pose optimization objective function, the optimized pose corresponding to the point cloud data is obtained; The preset global pose optimization objective function is: Represents the global pose graph The Middle The pose corresponding to the frame point cloud data express The reverse, Represents the global pose graph The Middle The pose corresponding to the frame point cloud data Represents the global pose graph The Middle Frame point cloud data and the first Relative pose constraints between frame point cloud data Represents the relative pose constraint Corresponding derivation of pose constraints Indicates the first Frame point cloud data and the first The inter-frame pose error between frame point cloud data. This indicates the inter-frame pose error. Perform a mapping from Lie groups to Lie algebras. The vectorization result, the global pose graph Nodes in The global pose graph represents the pose corresponding to the point cloud data. The edge in This represents the relative pose constraints between the point cloud data. Indicates the first The node corresponding to the frame point cloud data and the first The edges between nodes corresponding to the frame point cloud data belong to The first Optimized pose corresponding to frame point cloud data The pose that minimizes the objective function value of global pose optimization.

Images