A method and system for reconstructing skeleton models based on point cloud

Abstract

The present invention discloses a method and system for reconstructing a skeletal model based on a point cloud. The method comprises the following steps: S1, collecting point cloud information of key points of a patient's skeleton and coarsely registering it with a standard skeletal model; S2, collecting point cloud information of the patient's skeletal surface to obtain a patient's skeletal point cloud, and finely registering it with a standard skeletal model based on the coarse registration in S1; S3, elastically registering the patient's skeletal point cloud finely registered in S2 with the standard skeletal model so that the standard skeletal model fits the patient's skeletal point cloud; and S4, reconstructing a skeletal model of the patient's skeleton based on the registration results in S3. The present invention eliminates the need to collect CT images, significantly reducing surgical time and improving surgical efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of medical robots, and in particular to a point cloud-based skeletal model reconstruction method and system. Background Art

[0002] Currently, joint surgery, especially knee surgery, generally requires obtaining imaging data of the patient's joints, such as CT and MRI images. CT images offer high resolution and 3D modeling capabilities, facilitating preoperative surgical planning. Therefore, CT images are often used to obtain information about the patient's bones. Based on this information, doctors then formulate surgical plans.

[0003] Preoperative planning for joint replacement surgery requires reconstructing a 3D skeletal model of the joint based on CT images. This model is then used to guide preoperative planning, including osteotomy location, prosthesis type, and placement. Currently, domestic manufacturers rely on CT for preoperative planning and intraoperative navigation. However, preoperative CT reconstruction of the 3D skeletal model can take a long time, and the 3D skeletal model may differ from the actual bone and may contain missing bones. Furthermore, 3D skeletal model reconstruction leads to high costs for patients, increased radiation exposure, and difficulty in reimbursement. Summary of the Invention

[0004] Purpose of the invention: In view of the above-mentioned shortcomings, the present invention proposes a bone model reconstruction method and system based on point cloud, which can greatly reduce the operation time and improve the operation efficiency.

[0005] Technical solution:

[0006] A method for reconstructing a skeleton model based on point cloud, comprising:

[0007] S1. Collect the point cloud information of the patient's skeleton key points and perform rough registration with the standard skeleton model;

[0008] S2. Collecting point cloud information of the patient's bone surface to obtain the patient's bone point cloud, and performing fine registration with the standard bone model based on the coarse registration in S1;

[0009] S3, elastically registering the patient's bone point cloud and the standard bone model after precise registration in S2, so that the standard bone model fits the patient's bone point cloud;

[0010] S4. Reconstruct the bone model of the patient's bones according to the registration result of S3.

[0011] Specifically, in S2, after the patient's bone point cloud is collected, the step of completing the patient's bone point cloud is also included.

[0012] More specifically, the point cloud completion includes:

[0013] Calculate the point cloud boundary of the patient's bone point cloud, calculate the distance from the patient's bone key points obtained by S1 to the point cloud boundary, set the interval parameters accordingly and divide the distance, the corresponding segmentation point is the completion point, count all the completion points and the bone point cloud collected by S2 to obtain the final point cloud of the patient's bone.

[0014] Furthermore, the point cloud boundary of the patient's bone point cloud is calculated using a longitude and latitude scanning method, a grid division method, a normal estimation method or alpha shapes.

[0015] Specifically, the coarse registration of S1 includes:

[0016] The RANSAC algorithm is used to rigidly align the patient's bone key points with the corresponding points on the standard bone model.

[0017] Specifically, the fine registration in S2 includes:

[0018] S21, using the standard bone model as the source point cloud and the patient's bone point cloud as the target point cloud, obtain the initial transformation matrix of the source point cloud and the target point cloud according to the coarse registration in S1, and transform the source point cloud accordingly;

[0019] S22, searching for the nearest neighbor points between the transformed source point cloud and the target point cloud, and calculating the distance between each nearest neighbor point, and taking the sum of the distances as the distance error;

[0020] S23. Determine whether the distance error exceeds a set threshold. If not, complete the fine registration. If so, calculate and update the transformation matrix between the source point cloud and the target point cloud based on the nearest neighbor points between the transformed source point cloud and the target point cloud, and transform the source point cloud accordingly, and return to S22.

[0021] Specifically, before the elastic registration, the patient's bone point cloud and the standard bone model are meshed.

[0022] Specifically, in S3, the elastic registration of the patient's bone point cloud and the standard bone model after the S2 precise registration is as follows: the standard bone model is used as the source point cloud, the patient's bone point cloud is used as the target point cloud, and the registration is performed through the NICP algorithm based on the S2 precise registration.

[0023] More specifically, the NICP algorithm includes:

[0024] S31, searching for the nearest neighbor point between the target point cloud and the source point cloud based on the precise registration in S2;

[0025] S32, calculating the distance between the corresponding two points in each pair of nearest neighbor points, and determining whether each distance is less than a set threshold;

[0026] If it is less than , then the pair of nearest neighbor points is considered to be successfully registered, and the positions of the two corresponding points remain unchanged;

[0027] Otherwise, the transformation matrix between the corresponding points is decomposed by Cholesky to obtain the corresponding rotation matrix of the pair of nearest neighbor points, and the corresponding points in the source point cloud are rotated and the source point cloud is updated. The number of iterations is increased by 1.

[0028] S33, determine whether the number of iterations exceeds the set number P;

[0029] If yes, the iteration ends, and the final updated source point cloud is obtained, completing the elastic registration;

[0030] Otherwise, search for the nearest neighbor point between the updated source point cloud and the target point cloud, and return to S32.

[0031] A point cloud-based skeleton model reconstruction system, comprising:

[0032] Point cloud acquisition unit, used for collecting point clouds on bones;

[0033] Optical tracking system, used to obtain the position information of the point cloud acquisition unit in real time;

[0034] The processing unit obtains the position information of the point cloud acquisition unit obtained by the optical tracking system, and then obtains the point cloud information on the skeleton, and implements the standard skeleton model reconstruction method as described above.

[0035] Specifically, the processing unit obtains the point cloud information on the skeleton including:

[0036] The point cloud acquisition unit is provided with at least three non-collinear tracers, and collects point clouds by abutting the bone surface with their tips;

[0037] The optical tracking system acquires the position information of the tracer ball in real time;

[0038] The processing unit calculates the position information of the tip of the point cloud acquisition unit based on the position information of the tracer obtained by the optical tracking system, and further obtains the point cloud information on the corresponding bone.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] 1. The present invention can directly collect bone surface information during surgery through an optical tracking system, and reconstruct a 3D bone model of the bone through point cloud registration technology. It can directly perform intraoperative planning and navigation without the need to collect CT images, greatly reducing the time of the surgical process and improving surgical efficiency.

[0041] 2. The present invention completes point clouds for skeletal areas where point clouds are difficult to collect, ensuring the integrity and authenticity of the patient's skeletal point clouds.

[0042] 3. The present invention addresses the situation where there are some surface point mismatches or large errors between the patient's bone point cloud and the standard bone model, or there are situations where the standard bone model and the patient's bone point cloud are not well matched, and there are large deviations at some points. The present invention aligns the corresponding points in the patient's bone point cloud and the standard bone model to ensure the fit between the standard bone model and the collected patient's bone point cloud, ensure the accuracy of the alignment, and further improve the accuracy of the patient's bone model reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 Flowchart of the point cloud-based skeleton model reconstruction method of the present invention;

[0044] Figure 2 This is an example diagram of a point cloud model of a patient's bones in an embodiment of the present invention;

[0045] Figure 3 This is an example diagram of point cloud completion in an embodiment of the present invention, where: Figure 3 (a) is an example image before point cloud completion. Figure 3 (b) is an example image after point cloud completion;

[0046] Figure 4 This is an example of precise registration of the patient's bone point cloud with the standard bone model, where: Figure 4 (a) is an example of the rough registration of the patient's bone point cloud and the standard bone model. Figure 4 (b) An example diagram of the process of precise registration of the patient's bone point cloud with the standard bone model; Figure 4 (c) is an example image after precise registration of the patient’s bone point cloud with the standard bone model.

[0047] Figure 5 This is an example of elastic registration between the patient's bone point cloud and the standard bone model, where: Figure 5 (a) For Figure 4 (c) is an example of the elastic registration process based on the precise registration. Figure 5 (b) is an example image after elastic registration of the patient’s bone point cloud with the standard bone model. DETAILED DESCRIPTION

[0048] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments.

[0049] The present invention is based on the point cloud skeleton model reconstruction method, such as Figure 1 As shown, the steps include:

[0050] S1. Collect the point cloud information of the patient's skeleton key points and perform rough registration with the standard skeleton model;

[0051] Specifically:

[0052] S11. Click the key points of the patient's bones with a probe, identify the position information of the tracer ball through the optical tracking system, and then identify the position information of the key points of the patient's bones, so as to obtain the point cloud information of the key points of the patient's bones.

[0053] In the present invention, point cloud information of key points is collected by a point cloud collection unit such as a probe, and at least three non-collinear tracer balls are set on the probe. The key points of the patient's bones are points with anatomical significance on the patient's bones. For example, three key points are collected on the femur: the intercondylar fossa, the lateral condyle and the medial condyle; and three key points on the tibia: the proximal medial tangent point of the tibia, the proximal lateral tangent point of the tibia and the tibial tuberosity.

[0054] In the present invention, soft tissue may exist at certain key points on the patient's bones. In this case, it is necessary to puncture the soft tissue with a probe and collect point cloud information corresponding to the key points.

[0055] S12, performing coarse registration of the patient's skeleton with the standard skeleton model;

[0056] Based on the point cloud information of the patient's bone key points obtained in S11, combined with the corresponding points on the standard bone model, the patient's bone and the standard bone model are roughly aligned, so that the collected patient's bone key points and the corresponding points on the standard bone model coincide with each other. In the present invention, because the corresponding points involved in the alignment are only key points, the accuracy is not high, so they generally cannot completely coincide. However, this step is only to roughly align the patient's bone with the standard bone model to provide an initial alignment basis for subsequent fine alignment. Therefore, the rough alignment of the present invention only needs to make the collected patient's bone key points and the corresponding points on the standard bone model coincide as much as possible, as long as the set conditions are met. For example, the sum of the distances between each bone key point and the corresponding point on the standard bone model or the average distance is less than the set value. Among them, the standard bone model adopts the average bone model of the set number of people within the set age range.

[0057] Specifically, the present invention can perform rigid registration between the patient's skeletal key points and the corresponding points on the standard skeletal model through the RANSAC (Random Sample Consensus) algorithm.

[0058] S2. Collect point cloud information of the patient's bone surface to obtain the patient's bone point cloud, and perform fine registration with the standard bone model based on the rough registration in S1;

[0059] The probe is used to collect the point cloud information of the patient's bone surface except for the key points to obtain the patient's bone point cloud, such as Figure 2 During the acquisition process, the probe needs to be placed close to the bone surface and kept moving on the bone surface to ensure that the shape of the acquired point cloud is consistent with the actual bone.

[0060] In the present invention, for some bone parts where it is difficult for the probe to deeply collect point clouds, such as the area where the soft tissue is wrapped around the medial and lateral condyles of the femur, point cloud completion may be required, such as Figure 3 As shown in (a), after acquiring the patient's bone point cloud, the point cloud completion step of the patient's bone point cloud is also included;

[0061] Specifically, the point cloud boundary of the collected patient bone point cloud is calculated, and the distance between each key point and each boundary point is calculated accordingly, specifically the distance between the key point on the patient's bone with soft tissue obtained above and each boundary point, thereby setting the corresponding interval parameter between each key point and each boundary point, and dividing the distance according to the interval parameter. The corresponding segmentation point is the completion point, and the final point cloud of the patient's bone point cloud is obtained by counting all the completion points and the collected patient bone point cloud, thereby completing the point cloud completion of the patient's bone point cloud. Figure 3 (b) shown.

[0062] The present invention can use longitude and latitude scanning method, grid division method, normal estimation method, alpha shapes and other algorithms to calculate the point cloud boundary of the patient's bone point cloud. Preferably, the present invention uses a boundary extraction method based on normal estimation of point cloud data.

[0063] For example, taking the endocardium and endocardium as an example, after the endocardium and endocardium points are collected, the area from the endocardium and endocardium key points to the middle of the bone surface point cloud needs to be completed. It is necessary to calculate the actual point cloud boundary of the patient by any of the aforementioned methods for calculating the point cloud boundary of the patient's bone point cloud, and then calculate the distance from the endocardium and endocardium key points to the boundary points of the patient's actual point cloud. Then, the corresponding interval parameters are set accordingly, so that a new point cloud is generated in the area from the key point to the edge of the point cloud, and then spliced ​​with the actual point cloud to form the final point cloud of the patient's bone.

[0064] In the present invention, the ICP algorithm is used to accurately align the patient's bone point cloud with the standard bone model. Figure 5 The specific steps are as follows:

[0065] S21, using the standard bone model as the source point cloud and the patient's bone point cloud as the target point cloud, obtain the initial transformation matrix of the source point cloud and the target point cloud according to the rough registration of S1, and transform the source point cloud accordingly, as follows: Figure 4 (a)

[0066] S22, searching for the nearest neighbor point between the transformed source point cloud and the target point cloud using a nearest neighbor search algorithm, calculating the distance between each point in the transformed source point cloud and its corresponding nearest neighbor point in the target point cloud, and then calculating the sum of the distances of all nearest neighbor points as the distance error;

[0067] S23, determining whether the distance error exceeds a set threshold;

[0068] If not, the current transformation matrix is ​​used as the transformation matrix between the patient's final bone point cloud and the standard bone model to complete the precise registration, such as Figure 4 (c)

[0069] If so, the transformation matrix between the transformed source point cloud and the target point cloud is calculated and updated based on the nearest neighbor point between the two, and the source point cloud is transformed accordingly, such as Figure 4 As shown in (b), return to S22.

[0070] In this step, the patient's bone point cloud is aligned with the standard bone model. In this process, the patient's bone point cloud and the standard bone model are aligned as a whole. The purpose is to align and fit them as much as possible without changing the shape of the patient's bone point cloud. Compared with the aforementioned coarse registration, the registration accuracy is higher, and the patient's bone point cloud and the standard bone model are also more closely aligned than the aforementioned coarse registration.

[0071] S3, based on the precise registration in S2, elastically register the patient's bone point cloud with the standard bone model;

[0072] In the present invention, after the precise registration of S2, there may be some point mismatches or large errors, or the standard bone model may not fit the patient's bone point cloud well enough, and some points may have large deviations. Therefore, the present invention elastically registers the patient's bone point cloud and the standard bone model based on the precise registration of S2, so that the standard bone model fits the patient's bone point cloud, and then the standard bone model is transformed into the shape of the actual patient's bone through adjustment.

[0073] Furthermore, before elastic registration, the patient's bone point cloud and the standard bone model can be meshed, and of course, the two can also be directly elastically registered.

[0074] In the present invention, the patient's bone point cloud and the standard bone model can be meshed by using the Alpha reconstruction method, Poisson reconstruction method, rolling ball method reconstruction method and other methods to iterate the points in the patient's bone point cloud and the standard bone model until the entire point cloud is converted into a mesh model. Preferably, the present invention uses the rolling ball method reconstruction to mesh the patient's bone point cloud and the standard bone model.

[0075] In the present invention, illustratively, the NICP algorithm is used to elastically register the patient's bone point cloud with the standard bone model, specifically including:

[0076] S31, using the standard skeleton model as the source point cloud and the patient's skeleton point cloud as the target point cloud, based on the S2 precise registration, searching for the nearest neighbor points between the gridded patient's skeleton point cloud and the standard skeleton model using the nearest neighbor search algorithm;

[0077] S32, calculating the distance between the corresponding two points in each pair of nearest neighbor points, and determining whether each distance is less than a set threshold;

[0078] If it is less than , then the pair of nearest neighbor points is considered to be successfully registered, and the positions of the two corresponding points remain unchanged;

[0079] Otherwise, the transformation matrix between the corresponding points is decomposed by Cholesky to obtain the corresponding rotation matrix in the nearest neighbor point, and the corresponding points in the source point cloud are rotated and the source point cloud is updated, such as Figure 5 As shown in (a), the number of iterations is increased by 1; wherein, in the first iteration, the transformation matrix between corresponding points is the transformation matrix obtained by S22 precise registration;

[0080] S33, determine whether the number of iterations exceeds the set number P; if so, end the iteration, obtain the final updated source point cloud, and complete the elastic registration, such as Figure 5 (b); otherwise, search for the nearest neighbor point between the updated source point cloud and the target point cloud, and return to S32.

[0081] S4. Reconstruct the bone model of the patient's bones according to the registration result of S3.

[0082] Through the above-mentioned elastic registration, without changing the patient's bone point cloud, the standard bone model is transformed to fit the patient's bone point cloud, so that it is basically consistent with the actual shape of the patient's bone.

[0083] The present invention also provides a point cloud-based skeleton model reconstruction system, comprising:

[0084] The point cloud acquisition unit may be a probe having a detection tip and at least three non-collinear tracer balls. The point cloud acquisition unit acquires point clouds by contacting the bone surface with the detection tip.

[0085] Optical tracking system, real-time acquisition of the position information of the tracer ball on the point cloud acquisition unit;

[0086] The processing unit obtains the position information of the tracer ball on the point cloud acquisition unit obtained by the optical tracking system. Therefore, when the point cloud acquisition unit performs point cloud acquisition, the position information of the corresponding detection tip can be obtained through the position information of the tracer ball, and then the position information of the corresponding point on the bone surface can be obtained, and then the point cloud information can be obtained, and the point cloud information can be aligned with the standard bone model, and the reconstruction of the patient's standard bone model can be completed according to the alignment.

[0087] Specifically, the processing unit obtains the position information of the tracer ball on the point cloud acquisition unit obtained by the optical tracking system, thereby obtaining the position information of the corresponding detection tip through the position information of the tracer ball when the point cloud acquisition unit performs point cloud acquisition, and then obtaining the position information of the corresponding point on the bone surface, and then obtaining point cloud information.

[0088] The point cloud acquisition unit first collects the key points of the bone by contacting the key points of the patient's bone with its detection tip. The processing unit obtains the position information of the corresponding key points through the above processing, and performs coarse alignment with the standard bone model based on this; then, the point cloud acquisition unit collects the point cloud information of the bone surface, and the processing unit calculates the point cloud boundary based on the point cloud information of the bone surface, and calculates the distance from each key point to the point cloud boundary, and sets the interval parameters accordingly to complete the point cloud to obtain the final point cloud of the patient's bone; wherein, the standard bone model adopts the average bone model of a set number of people within a set age range.

[0089] The processing unit uses the ICP algorithm to precisely align the final point cloud of the patient's bones with the standard bone model, and based on this precise alignment, uses the NICP algorithm to elastically align the final point cloud of the patient's bones with the standard bone model, so that the standard bone model fits the patient's bone point cloud. Finally, the bone model of the patient's bones is reconstructed based on the elastic alignment result.

[0090] In the present invention, the calculation of point cloud boundaries, point cloud completion, ICP algorithm and NICP algorithm can all be consistent with the aforementioned standard skeleton model reconstruction method.

[0091] The present invention does not require the acquisition of CT images for the reconstruction of the patient's standard bone model. The optical tracking system can directly acquire the point cloud information of the bone surface during the operation to obtain the patient's bone point cloud. The three-dimensional reconstruction of the patient's bone model is completed by aligning it with the point cloud of the standard bone model. After the reconstruction is completed, planning and navigation are directly performed, which not only shortens the surgical process, improves surgical efficiency, but also greatly reduces the patient's medical costs. In addition, in the process of acquiring the patient's bone point cloud, the present invention performs point cloud completion for the bone parts that are difficult to acquire point clouds, ensuring the integrity and authenticity of the patient's bone point cloud. After alignment, elastic alignment is performed for the situation where there are some surface point mismatches or large errors between the patient's bone point cloud and the standard bone model. That is, the corresponding points in the patient's bone point cloud and the standard bone model are all aligned to ensure the fit between the standard bone model and the acquired patient's bone point cloud, ensure the accuracy of the alignment, and further improve the accuracy of the patient's bone model reconstruction.

[0092] The preferred embodiments of the present invention are described in detail above, but the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations (such as quantity, shape, position, etc.) can be made to the technical solution of the present invention, and these equivalent transformations all fall within the scope of protection of the present invention.

Claims

1. A method for reconstructing a skeleton model based on point cloud, characterized in that: include: S1. Collect the point cloud information of the patient's skeleton key points and perform rough registration with the standard skeleton model; S2. Collecting point cloud information of the patient's bone surface to obtain the patient's bone point cloud, and performing fine registration with the standard bone model based on the coarse registration in S1; S3, using the standard skeleton model as the source point cloud and the patient's skeleton point cloud as the target point cloud, perform elastic registration based on the precise registration in S2, so that the standard skeleton model fits the patient's skeleton point cloud, specifically: S31, searching for the nearest neighbor point between the target point cloud and the source point cloud based on the precise registration in S2; S32, calculating the distance between the corresponding two points in each pair of nearest neighbor points, and determining whether each distance is less than a set threshold; If it is less than , then the pair of nearest neighbor points is considered to be successfully registered, and the positions of the corresponding two points remain unchanged; Otherwise, the transformation matrix between the corresponding points is decomposed by Cholesky to obtain the corresponding rotation matrix of the pair of nearest neighbor points, and the corresponding points in the source point cloud are rotated and the source point cloud is updated. The number of iterations is increased by 1. S33, determine whether the number of iterations exceeds the set number P; If yes, the iteration ends, and the final updated source point cloud is obtained, completing the elastic registration; Otherwise, search for the nearest neighbor point between the updated source point cloud and the target point cloud, and return to S32; S4. Reconstruct the bone model of the patient's bones according to the registration result of S3.

2. The skeleton model reconstruction method according to claim 1, characterized in that: In the above S2, after the patient's bone point cloud is collected, the step of completing the patient's bone point cloud is also included.

3. The skeleton model reconstruction method according to claim 2, characterized in that: The point cloud completion includes: Calculate the point cloud boundary of the patient's bone point cloud, calculate the distance from the patient's bone key points obtained by S1 to the point cloud boundary, set the interval parameters accordingly and divide the distance, the corresponding segmentation point is the completion point, count all the completion points and the bone point cloud collected by S2 to obtain the final point cloud of the patient's bone.

4. The skeleton model reconstruction method according to claim 3, characterized in that: The point cloud boundary of the patient's bone point cloud is calculated using a longitude and latitude scanning method, a grid division method, a normal estimation method or alpha shapes.

5. The skeleton model reconstruction method according to claim 1, characterized in that: The coarse registration of S1 includes: The RANSAC algorithm is used to rigidly align the patient's bone key points with the corresponding points on the standard bone model.

6. The skeleton model reconstruction method according to claim 1, characterized in that: The fine registration in S2 includes: S21, using the standard bone model as the source point cloud and the patient's bone point cloud as the target point cloud, obtaining the initial transformation matrix of the target point cloud and the source point cloud according to the coarse registration in S1, and transforming the source point cloud accordingly; S22, searching for the nearest neighbor points between the transformed source point cloud and the target point cloud, and calculating the distance between each nearest neighbor point, and taking the sum of the distances as the distance error; S23. Determine whether the distance error exceeds a set threshold. If not, complete the fine registration. If so, calculate and update the transformation matrix between the source point cloud and the target point cloud based on the nearest neighbor points between the transformed source point cloud and the target point cloud, and transform the source point cloud accordingly, and return to S22.

7. The skeleton model reconstruction method according to claim 1, characterized in that: Prior to the elastic registration, the patient's bone point cloud and the standard bone model are meshed.

8. A point cloud-based skeleton model reconstruction system, characterized in that: include: Point cloud acquisition unit, used for collecting point clouds on bones; Optical tracking system, used to obtain the position information of the point cloud acquisition unit in real time; The processing unit obtains the position information of the point cloud acquisition unit obtained by the optical tracking system, and then obtains the point cloud information on the skeleton, and implements the skeleton model reconstruction method according to any one of claims 1 to 7.

9. The skeleton model reconstruction system according to claim 8, characterized in that: The processing unit obtains the point cloud information on the skeleton, including: The point cloud acquisition unit is provided with at least three non-collinear tracers, and collects point clouds by abutting the bone surface with their tips; The optical tracking system acquires the position information of the tracer in real time; The processing unit calculates the position information of the tip of the point cloud acquisition unit based on the position information of the tracer obtained by the optical tracking system, and then obtains the point cloud information on the corresponding bone.

Images