A method of registering a bone model to a world coordinate system of a surface point cloud

Abstract

The application discloses a method for registering a skeleton model to a world coordinate system of a body surface point cloud, which comprises the following steps: S1. scanning a human structure by using a CT scanning device to obtain a CT skeleton three-dimensional model and a CT human structure body surface point cloud; S2. scanning the human structure by using a structured light 3D camera to obtain a 3D structured light body surface point cloud of a 3D structured light human structure; S3. selecting an antegrade registration or a retrograde registration mode to register the body surface and the skeleton of the human structure according to the scanned human structure; the method for registering the skeleton model to the world coordinate system of the body surface point cloud: registering the torso and the pelvic bone of the human body by using the antegrade registration mode; registering the limbs of the human body by using the retrograde registration mode; constructing a virtual puncture channel, so that even if the spatial position of a puncture instrument is adjusted at will in a process of orthopedic puncture, the relative position relationship between the puncture instrument and the human structure can be acquired in real time, and the navigation puncture based on dynamic three-dimensional visualization is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and particularly relates to a method for registering a bone model to a world coordinate system of a body surface point cloud. BACKGROUND

[0002] With the development of science and technology, modern orthopedic surgery often scans the human body part to be operated by CT, MR, three-dimensional C-arm and other medical image scanning devices, realizes the visualization of the internal structure of the human body through a specific virtual coordinate system, and assists the surgeon to better complete the orthopedic surgery; however, since these devices cannot realize dynamic and continuous scanning, only a three-dimensional coordinate system and static three-dimensional visualization of a certain time cross section with static properties can be formed, and therefore visual navigation puncture based on dynamic three-dimensional visualization cannot be realized.

[0003] The existing medical image data has not realized the unification of the virtual coordinate system and the real coordinate system in the application of puncture navigation in orthopedics, so that the internal structure information of the human body cannot be interacted with the position change information of the puncture instrument in real time in the puncture process, the puncture result cannot be accurately predicted, the puncture error is caused, and the accurate positioning of the puncture position by the surgeon becomes an unavoidable problem. SUMMARY

[0004] To solve the above technical problems, the present application provides a method for registering a bone model to a world coordinate system of a body surface point cloud.

[0005] To achieve the above purposes, the following scheme is provided: a method for registering a bone model to a world coordinate system of a body surface point cloud, comprising the following steps,

[0006] S1. scanning the human body structure using a CT scanning device to obtain a CT bone three-dimensional model and a CT body structure body surface point cloud;

[0007] S2. scanning the human body structure using a structured light 3D camera to obtain a 3D structured light body surface point cloud of the human body structure;

[0008] S3. registering the body surface and the bone of the human body structure in an antegrade registration or retrograde registration mode according to the scanned human body structure.

[0009] Further, the step of antegrade registration is as follows:

[0010] A1. reconstructing the CT bone and body surface three-dimensional model using Mimics software;

[0011] A2. inputting the CT bone three-dimensional model of the registration object, the CT body structure body surface point cloud and the 3D structured light body surface point cloud data of the human body structure obtained in S2 into Cloudcompare software;

[0012] A3. Drag the CT bone three-dimensional model and the CT human body structure surface three-dimensional model to the 3D structured light surface point cloud near the human body structure by Cloudcompare software for fast coarse registration;

[0013] A4. Trim the CT human body structure surface three-dimensional model by Cloudcompare software, and the remaining part after trimming is the card position film;

[0014] A5. Perform first registration by Cloudcompare software: register the card position film in step A4 to the 3D structured light surface point cloud of the human body structure to obtain the rotation and translation matrix of the card position film;

[0015] A6. Perform second registration by Cloudcompare software: register the CT bone three-dimensional model to the 3D structured light surface point cloud of the human body structure according to the rotation and translation matrix of the card position film to form an allogeneic composite structure composed of the CT bone three-dimensional model and the 3D structured light surface point cloud of the human body structure;

[0016] A7. Detect the registration accuracy by Cloudcompare software, and calculate the average distance and standard deviation of the card position film and the 3D structured light surface point cloud.

[0017] Further, the steps of the reverse registration are as follows:

[0018] B1. Reconstruct the CT bone and surface three-dimensional model by Mimics software;

[0019] B2. Input the CT bone three-dimensional model, the CT human body structure surface point cloud, and the 3D structured light surface point cloud data of the human body structure obtained in S2 to Cloudcompare software;

[0020] B3. Drag the CT bone three-dimensional model and the CT human body structure surface three-dimensional model to the 3D structured light surface point cloud near the human body structure by Cloudcompare software for fast coarse registration;

[0021] B4. Copy the 3D structured light surface point cloud of the human body structure by Cloudcompare software, and trim the copied body to obtain a card position film;

[0022] B5. Perform first registration by Cloudcompare software: register the card position film to the CT human body structure surface three-dimensional model to obtain the rotation and translation matrix of the card position film, and copy the rotation and translation matrix of the card position film;

[0023] B6. Second registration by Cloudcompare software: Paste the rotation and translation matrix of the fiducial film in step B5 in the Apply Transformation pop-up window, select the inverse matrix Inverse matrix, and move the CT bone three-dimensional model to the 3D structured light body surface point cloud coordinate system of the human body structure;

[0024] B7. Detect registration accuracy: Calculate the average distance and standard deviation of the fiducial film and the CT human body structure surface three-dimensional model in the Cloudcompare software.

[0025] The advantages of the present application are:

[0026] The method for registering the bone model to the body surface point cloud world coordinate system is to register the torso and pelvic bones of the human body by forward registration, and to register the limbs of the human body by reverse registration. A virtual puncture channel is constructed to realize visual three-dimensional dynamic puncture navigation, which has high flexibility. When applied to the orthopedic puncture process, the position of the puncture instrument can be moved at will under the premise of visualizing the internal structure of the human body to obtain the most suitable puncture point. The present application enables the physician to observe the positional relationship between the puncture instrument and the fixed bone in need of puncture in real time, assists the physician to determine the puncture position more quickly and accurately, reduces unnecessary damage, reduces the difficulty of surgery, and improves the success rate of surgery. DETAILED DESCRIPTION

[0027] The following will be further described in detail through specific embodiments:

[0028] The method for registering the bone model to the body surface point cloud world coordinate system comprises the following steps,

[0029] S1. Scanning the human body structure using a CT scanning device to obtain a CT bone three-dimensional model and a CT human body structure surface point cloud;

[0030] S2. Scanning the human body structure using a structured light 3D camera to obtain a 3D structured light body surface point cloud of the 3D structured light human body structure;

[0031] S3. Selecting forward registration or reverse registration according to the scanned human body structure to register the body surface and the bone of the human body structure.

[0032] The hardware and software used in the method include: (1) Sizector® structured light 3D camera S028800, Shanghai Shengxiang Industrial Detection Technology Co., Ltd.; (2) application software MPSizector S SDK V2.15 provided with the structured light 3D camera, medical three-dimensional reconstruction software Mimics 19.0, and registration software Cloudcompare V2.11.3. Other hardware and software with the same functions can also be used.

[0033] The purpose of the method is to accurately register a CT bone three-dimensional model to a world coordinate system of a 3D structured light body surface point cloud, and reconstruct the positional relationship between the two. The 3D structured light body surface point cloud is a reference body (Reference) that does not move during registration, and its coordinate system is the world system. The CT bone three-dimensional model is an aligned target (Aligned) that needs to be actively moved to the Reference during registration. The intermediary between the CT bone three-dimensional model and the 3D structured light body surface point cloud of the human body structure is a CT body surface three-dimensional model or point cloud, which is essentially a scan of the same human body structure using a CT and a structured light 3D camera.

[0034] The general principles to be followed in registration are: (1) 100% overlap. Therefore, the range of Aligned must be smaller than that of Reference, and Aligned can be registered to Reference, but not vice versa; (2) The FOV of CT should not be too large. Since the resolution of CT scan FOV is only 512*512 pixels, if the FOV is too large, the number of pixels allocated to the target region of interest (ROI) will certainly be small, and the imaging quality will also decrease. Therefore, in order to ensure the quality of three-dimensional imaging, the FOV should not be blindly expanded, but the scanning of ROI should be prioritized to obtain higher scanning quality; (3) Appropriately distinguish the CT scan FOV of limbs and trunk. Since the limbs are smaller than the trunk, the CT scan FOV should include the limbs, and the trunk should be determined according to the purpose of scanning; (4) The scanning length is positively correlated with the radiation dose. Under the premise of achieving the purpose of scanning, the scanning length should not be increased, which is applicable to both limbs and trunk; (5) The scanning range of the structured light 3D camera body surface point cloud can be expanded. This is because the 3D structured light camera has no radiation, and has a point cloud resolution of millions to tens of millions, and appropriately expanding the scanning range is beneficial to completely cover the CT scan area, while ensuring a very high point cloud resolution.

[0035] In order to obtain faster registration speed, easier registration operation, higher registration success rate and more accurate registration effect, the registration can be divided into two ways, forward and reverse registration, according to whether the scanning target is limbs or trunk. The forward registration is used for trunk and pelvic part, and the reverse registration is used for limbs. Whether it is forward or reverse registration, the purpose and final result are to register the CT skeleton to the 3D structured light body surface point cloud world coordinate system, and the process includes two steps of registration and bone registration.

[0036] When the forward registration is performed on the human trunk structure, the specific steps are as follows:

[0037] A1. Reconstruct the CT skeleton and body surface three-dimensional model using Mimics software:

[0038] The two-dimensional tomographic images obtained by spiral CT scanning are input into Mimics software in Dicom format. After threshold segmentation, three-dimensional reconstruction is performed on the skeleton and body surface. In order to ensure high-quality three-dimensional reconstruction of the target area of thoracolumbar vertebrae, the FOV of the lumbar and back scanning position can be appropriately reduced during CT scanning, without the need to completely include the lumbar and back.

[0039] A2. Output the CT skeleton three-dimensional model of the registration object, CT human structure body surface point cloud and 3D structured light body surface point cloud data of the human structure to Cloudcompare software:

[0040] The CT skeleton three-dimensional model and CT human structure body surface point cloud in STL format are output from Mimics, and the 3D structured light body surface point cloud of the static human structure is output from MPSizectorS. Then the above three objects are input into the registration software Cloudcompare.

[0041] A3. Drag the CT skeleton three-dimensional model and CT human structure body surface point cloud to the 3D structured light body surface point cloud of the human structure in Cloudcompare software for fast coarse registration:

[0042] In Cloudcompare, the CT skeleton three-dimensional model and CT body surface point cloud are selected by CTRL, and the Translate / Rotate function is used to drag them to the 3D structured light body surface point cloud of the human structure, so that the CT body surface point cloud is placed outside the 3D structured light body surface point cloud of the human structure, and the two are roughly parallel. The point-point coarse registration function Align (point pairs picking) provided by Cloudcompare can also be used to select 4 pairs of registration points of Aligned and Reference to achieve fast coarse registration.

[0043] A4. Cutting the redundant part of the CT body structure surface point cloud by Cloudcompare software, and the remaining part after cutting is the card position film:

[0044] Cut the redundant part of the CT body surface point cloud with the Segment function of Cloudcompare, such as point cloud outside the body surface, inside the body, environmental debris, and noise, etc. Only the body surface part is retained, and the cut is renamed as "virtual card position film", which is called card position film. The cutting of the card position film follows the following principles: (1) It must be completely covered by the 3D structured light body surface point cloud of the human body structure; (2) Try to retain more detailed CT body surface point cloud in a larger range to facilitate accurate registration; (3) Cut directly on the CT body surface point cloud, and there is no need to retain the original piece. If necessary, the CT body surface point cloud can be loaded again; (4) Rename in time to avoid confusion.

[0045] A5. First registration by Cloudcompare software: register the card position film in step A4 to the 3D structured light body surface point cloud of the human body structure to obtain the rotation and translation matrix of the card position film:

[0046] Register the card position film to the 3D structured light body surface point cloud with the Fine Registration (ICP) function of Cloudcompare, and obtain the rotation and translation matrix of the card position film (which can be directly copied in the Console bar).

[0047] A6. Second registration by Cloudcompare software: register the CT bone three-dimensional model to the 3D structured light body surface point cloud of the human body structure according to the rotation and translation matrix of the card position film, to form a heterologous composite structure composed of the CT bone three-dimensional model and the 3D structured light body surface point cloud of the human body structure;

[0048] With the Apply Transformation function of Cloudcompare, register the CT bone three-dimensional model to the 3D structured light body surface point cloud of the human body structure according to the rotation and translation matrix of the card position film. The specific operation is to paste the copied rotation and translation matrix of the card position film in the Apply Transformation pop-up window, which can move the CT bone to the 3D structured light body surface point cloud coordinate system. After the second registration, a heterologous composite structure composed of the CT bone of the human body structure and the 3D structured light body surface point cloud is formed, that is, the data information of its body surface comes from the body surface point cloud scanned by the structured light 3D camera, and the internal bone comes from the CT scan, which is the comprehensive result of the data information of different imaging devices.

[0049] A7. Detect the registration accuracy by Cloudcompare software, calculate the average distance and standard deviation of the card position film and the 3D structured light body surface point cloud.

[0050] In Cloudcompare, the average distance and standard deviation of the 3D structured light body surface point cloud of the card position diaphragm and the human structure are calculated using the Compute cloud / cloud distance function. Note that the card position diaphragm is Compared (comparison body), and the 3D structured light body surface point cloud is Reference (reference body).

[0051] When performing retrograde registration on human limbs, the specific steps are as follows:

[0052] B1. Use Mimics software to reconstruct the CT bone and body surface three-dimensional model:

[0053] Import the two-dimensional tomographic images obtained by spiral CT scanning into Mimics software in Dicom format. After threshold segmentation, reconstruct the three-dimensional model of the bone and body surface, respectively. During CT scanning, the limb is completely included in the FOV.

[0054] B2. Output the CT bone three-dimensional model of the registration object, the CT human structure body surface three-dimensional model, and the 3D structured light body surface point cloud data of the human structure to the Cloudcompare software:

[0055] Output the STL format CT bone three-dimensional model and CT human structure body surface three-dimensional model from Mimics, and output the 3D structured light body surface point cloud of the human structure from MPSizectorS. Then input the above three objects into the registration software Cloudcompare.

[0056] B3. Through the Cloudcompare software, drag the CT bone three-dimensional model and the CT human structure body surface three-dimensional model close to the 3D structured light body surface point cloud of the human structure for fast coarse registration:

[0057] Drag the CT bone three-dimensional model and the CT human structure body surface three-dimensional model at the same time, so that the 3D structured light body surface point cloud of the human structure is between the CT bone three-dimensional model and the CT human structure body surface three-dimensional model. The same as step A3.

[0058] B4. Through the Cloudcompare software, copy the 3D structured light body surface point cloud of the human structure, and cut the copied body. The remaining part after cutting is the card position diaphragm:

[0059] In Cloudcompare, copy the 3D structured light body surface point cloud of the human structure using the Clone the selected entities function, and cut the copied body into the card position diaphragm. The specific operation is the same as step A4.

[0060] B5. First registration by Cloudcompare software: After registering the marker film to the CT body structure surface three-dimensional model, the rotation and translation matrix of the marker film can be obtained, and the rotation and translation matrix of the marker film is copied:

[0061] The marker film is registered to the CT body structure surface three-dimensional model, and the registration direction is opposite to the first registration of the antegrade registration, that is, the 3D structure point cloud of the human structure is oriented to the CT surface three-dimensional model, the rotation and translation matrix of the marker film is obtained and copied.

[0062] B6. Second registration by Cloudcompare software: paste the rotation and translation matrix of the marker film in step B5 in the Apply Transformation pop-up window, select the inverse matrix Inverse matrix, and move the CT bone three-dimensional model to the 3D structure light surface point cloud coordinate system of the human structure;

[0063] B7. Detect registration accuracy: calculate the average distance and standard deviation of the marker film and the CT body structure surface three-dimensional model in the Cloudcompare software.

[0064] The method of registering the bone model to the surface point cloud world coordinate system: (1) Sizector® S028800 is a structured light 3D camera with dynamic scanning properties, with a dynamic scanning frame rate of up to 10 FPS, a static scanning resolution of up to 2.8 million pixels, and an accuracy of up to 10 μm. Other devices with the same function can be used for replacement; (2) both Aligned and Reference have a large number of point clouds or triangular facets for registration. (3) After completing the registration, the position of the puncture instrument can be adjusted by dynamic scanning to construct a virtual puncture channel and achieve dynamic navigation; the freedom of puncture instrument position adjustment is extremely high, and almost any free adjustment can be achieved. Only by scanning a small part of the point cloud of the puncture instrument through the structured light 3D camera, the puncture channel can be reconstructed, so it has extremely high navigation and puncture flexibility. (4) Without changing the original operation habit of the surgeon; without mechanical arm, the navigation and puncture operation is completed by the surgeon throughout the operation, which is more simple and flexible, and can better adapt to complex operations, and can also shorten the operation time. (5) Cloudcompare selected is a completely free and completely open source software, which can be used for navigation special software development using its function module code, and can accelerate the development progress of navigation and puncture software.

[0065] The above is only the embodiment of the present application, and the common knowledge of the specific structure and characteristics in the scheme is not described in detail. The ordinary skilled person in the art knows all the ordinary technical knowledge in the field of the present application before the application date or the priority date, can know all the prior art in the field, and has the ability to apply conventional experimental means before that date. The ordinary skilled person in the art can perfect and implement the present scheme based on the disclosure given in the present application, and the ability of the ordinary skilled person in the art should not be an obstacle to the implementation of the present application. It should be noted that for those skilled in the art, without departing from the structure of the present application, a number of modifications and improvements can be made, which should also be considered as the protection scope of the present application, which will not affect the effect and applicability of the present application. The protection scope of the present application should be subject to the content of its claims, and the specific implementation mode and the like in the specification can be used to explain the content of the claims.

Claims

1. A method for registering a skeletal model to the world coordinate system of a body surface point cloud, characterized in that: Includes the following steps, S1. Use CT scanning equipment to scan the human body structure to obtain a 3D CT skeleton model and a CT human body structure surface point cloud; S2. Use a structured light 3D camera to scan the human body structure and obtain a 3D structured light point cloud of the human body surface. S3. Based on the scanned human body structure, use the anterograde registration method to register the human body surface and bones; The steps for the forward registration are as follows: A1. Reconstruct 3D models of the CT skeleton and body surface using Mimics software; A2. Input the CT skeleton 3D model of the registration object, the CT human body structure surface point cloud, and the 3D structured light surface point cloud data of the human body structure obtained in S2 into the Cloudcompare software; A3. Using Cloudcompare software, drag the CT skeleton 3D model and the CT human body structure surface 3D model to the 3D structured light surface point cloud close to the human body structure for quick coarse registration; A4. Use Cloudcompare software to trim the excess parts of the 3D model of the human body structure surface in CT scan. The remaining part after trimming is the positioning membrane. A5. Perform the first registration using Cloudcompare software: Register the positioning membrane from step A4 to the 3D structured light surface point cloud of the human body structure to obtain the rotation and translation matrix of the positioning membrane. A6. Second registration using Cloudcompare software: Based on the rotation and translation matrix of the positioning diaphragm, the CT skeleton 3D model is registered to the 3D structured light surface point cloud of the human body structure, forming a heterogeneous composite structure composed of the CT skeleton 3D model and the 3D structured light surface point cloud of the human body structure. A7. Use Cloudcompare software to check the registration accuracy and calculate the average distance and standard deviation between the positioning diaphragm and the point cloud on the 3D structured light surface.

2. A method for registering a skeletal model to the world coordinate system of a body surface point cloud, characterized in that: Includes the following steps, S1. Use CT scanning equipment to scan the human body structure to obtain a 3D CT skeleton model and a CT human body structure surface point cloud; S2. Use a structured light 3D camera to scan the human body structure and obtain a 3D structured light point cloud of the human body surface. S3. Based on the scanned human body structure, use retrograde registration to register the body surface and bones of the human body structure; The steps for the retrograde registration are as follows: B1. Use Mimics software to reconstruct 3D models of CT-reconstructed skeleton and body surface; B2. Input the CT skeleton 3D model of the registration object, the CT human body structure surface point cloud, and the 3D structured light surface point cloud data of the human body structure obtained in S2 into the Cloudcompare software; B3. Using Cloudcompare software, drag the CT skeleton 3D model and the CT human body structure surface 3D model to the 3D structured light surface point cloud close to the human body structure for rapid coarse registration; B4. Using Cloudcompare software, copy the 3D structured light surface point cloud of the human body structure, and then trim the copy. The remaining part after trimming is the positioning membrane. B5. First registration using Cloudcompare software: After registering the diaphragm to the 3D model of the human body structure in CT, the rotation and translation matrix of the diaphragm can be obtained and copied. B6. Perform a second registration using Cloudcompare software: In the Apply Transformation pop-up window, paste the rotation and translation matrix of the positioning membrane from step B5, select to apply the inverse matrix, and move the CT skeleton 3D model to the 3D structured light body surface point cloud coordinate system of the human body structure. B7. Detection and registration accuracy: Calculate the average distance and standard deviation between the positioning diaphragm and the 3D model of the human body structure on the CT scan using Cloudcompare software.