Joint bone point selection registration method and system based on augmented reality

Abstract

The application provides a kind of based on the joint bone point selection registration method and system of augmented reality, it is related to skeletal point selection registration technical field, including the three-dimensional model image obtained by three-dimensional reconstruction to joint CT image;Multiple registration points are selected on the surface of three-dimensional model image;Three-dimensional model image is displayed in augmented reality glasses, and make joint physical object and augmented reality glasses display image completely coincide;Select multiple points corresponding to multiple registration points on the three-dimensional model image in augmented reality glasses on joint physical object.The application based on the method of augmented reality, point selection registration is carried out through augmented reality glasses, avoids the error of directly selecting registration point on physical object, improves the registration accuracy between skeletal image coordinate system and physical object coordinate system, and further improves the system accuracy of knee joint replacement surgery robot;Make doctor can quickly and accurately find the registration point on the patient's skeleton, greatly improve the registration accuracy, shorten the point selection time.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of bone point selection registration, and particularly relates to a joint bone point selection registration method and system based on augmented reality. BACKGROUND

[0002] In recent years, a knee replacement surgery robot system has achieved rapid development due to good surgical precision and repeatability, and has become a potential technical development direction in knee replacement surgery.

[0003] The knee replacement surgery robot system uses point cloud registration technology to register a patient's knee bone image coordinate system, a real object coordinate system and an optical positioning tool coordinate system, and uses a mechanical arm to perform knee replacement surgery. In order to achieve accurate registration, usually dozens of registration points are required, and the point selection process requires the doctor to find the point on the surface of the patient's knee bone corresponding to the position of the registration point in the virtual reality three-dimensional model image by naked eye, and then select it with a probe with a positioning ball.

[0004] The current point selection method by naked eye has a high dependence on the doctor's experience. If the position deviation of the selected point and the point on the three-dimensional model image is large, a high registration error will be caused, which directly reduces the precision of the knee replacement surgery robot system. SUMMARY

[0005] In view of the above problems and assumptions, the application provides a joint bone point selection registration method and system based on augmented reality, so that the doctor user can quickly and accurately find the registration point on the patient's bone, which can greatly improve the registration accuracy and shorten the point selection time.

[0006] To achieve the above purpose, the application discloses a joint bone point selection registration method based on augmented reality, comprising the following steps:

[0007] Three-dimensional reconstruction of joint CT images to obtain a three-dimensional model image;

[0008] Selecting a plurality of registration points on the surface of the three-dimensional model image;

[0009] Displaying the three-dimensional model image in the augmented reality glasses, so that the joint real object and the augmented reality glasses display image are completely overlapped;

[0010] Selecting a plurality of points on the joint real object corresponding to the plurality of registration points on the augmented reality glasses display image.

[0011] As a further improvement of the application, the points on the joint real object are selected by a probe with a positioning ball.

[0012] As a further improvement of the present application, when selecting the point on the joint object, the needle tip of the ball probe is placed at the position of the registration point on the three-dimensional model image.

[0013] As a further improvement of the present application, four positioning balls are installed on the ball probe, and the center coordinates of the positioning balls and the geometric size from the needle tip of the ball probe to the center of the positioning balls are obtained to calculate the position coordinates of the needle tip of the ball probe.

[0014] As a further improvement of the present application, an optical positioning and tracking device is used to obtain the coordinate data of the positioning balls on the ball probe in real time to obtain the posture information of the ball probe.

[0015] The present application also provides a joint bone point selection and registration system based on augmented reality, comprising a model establishment module, a registration point selection module, an object registration module, and an object point selection module.

[0016] The model establishment module is used to:

[0017] obtain a three-dimensional model image according to three-dimensional reconstruction of a joint CT image;

[0018] The registration point selection module is used to:

[0019] select a plurality of registration points on the surface of the three-dimensional model image;

[0020] The object registration module is used to:

[0021] display the three-dimensional model image in augmented reality glasses so that the joint object and the display image of the augmented reality glasses are completely overlapped;

[0022] The object point selection module is used to:

[0023] select points on the joint object corresponding to the plurality of registration points on the display image of the augmented reality glasses.

[0024] As a further improvement of the present application, a ball probe is further included for directly selecting the points on the joint object.

[0025] As a further improvement of the present application, when the object point selection module performs the point selection operation, the needle tip of the ball probe is placed at the position of the registration point on the three-dimensional model image.

[0026] As a further improvement of the present application, four positioning balls are installed on the ball probe, and the center coordinates of the positioning balls and the geometric size from the needle tip of the ball probe to the center of the positioning balls are obtained to calculate the position coordinates of the needle tip of the ball probe.

[0027] As a further improvement of the present application, an optical positioning tracking device is further included, which obtains coordinate data of each positioning ball on the ball-equipped probe in real time to obtain posture information of the ball-equipped probe.

[0028] Compared with the prior art, the present application has the following beneficial effects:

[0029] The method based on augmented reality selects a point for registration through augmented reality glasses, avoids errors of directly selecting a registration point on a real object, improves registration accuracy between a bone image coordinate system and a real object coordinate system, and further improves system accuracy of a knee joint replacement surgery robot. BRIEF DESCRIPTION OF DRAWINGS

[0030] Figure 1 A flow chart of a joint bone point selection and registration method based on augmented reality is disclosed for an embodiment of the present application;

[0031] Figure 2 A schematic diagram of a joint bone point selection and registration system based on augmented reality is disclosed for an embodiment of the present application;

[0032] Figure 3 A schematic diagram of a hardware composition of a joint bone point selection and registration system based on augmented reality is disclosed for an embodiment of the present application.

[0033] Reference signs:

[0034] 1-optical positioning tracking device; 2-ball-equipped probe; 3-registration point; 4-three-dimensional model image; 5-joint real object; 6-bone positioning tool; 7-augmented reality glasses display image. DETAILED DESCRIPTION

[0035] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0036] The present application will be described in further detail below with reference to the drawings:

[0037] As Figure 1 shown, a joint bone point selection and registration method based on augmented reality disclosed by the present application comprises:

[0038] S1, a three-dimensional model image 4 is obtained by three-dimensional reconstruction of a joint CT image;

[0039] In the method,

[0040] First, the patient's bone CT image is taken.

[0041] S2, selecting multiple registration points 3 on the surface of the three-dimensional model image 4;

[0042] Among them,

[0043] The doctor needs to select multiple registration points 3 on the surface of the three-dimensional model by experience, and the registration points 3 are displayed on the three-dimensional model image 4 through three-dimensional visualization technology.

[0044] S3, displaying the three-dimensional model image 4 in the augmented reality glasses, so that the joint physical object 5 and the augmented reality glasses display image 7 are completely overlapped;

[0045] Among them,

[0046] Using the commonly used augmented reality registration algorithm, the image of the joint physical object 5 and the three-dimensional model image 4 are registered, so that they are overlapped in the augmented reality glasses.

[0047] S4, selecting multiple points on the joint physical object 5 corresponding to the multiple registration points 3 on the augmented reality glasses display image 7.

[0048] Among them,

[0049] The points on the joint physical object 5 are selected by the probe with positioning balls.

[0050] When selecting the points on the joint physical object 5, the needle tip of the ball probe 2 is placed at the position of the registration point 3 on the three-dimensional model image 4.

[0051] Four positioning balls are installed on the ball probe 2, and the center coordinates of each positioning ball are obtained; according to the geometric size of the needle tip of the ball probe 2 to the center of the positioning ball, the position coordinates of the needle tip of the ball probe 2 are calculated.

[0052] The optical positioning and tracking device 1 is used to obtain the coordinate data of each positioning ball on the ball probe 2 in real time, and the attitude information of the ball probe 2 is calculated.

[0053] As Figure 2 The application also provides a joint bone point selection registration system based on augmented reality, which comprises a model establishing module, a registration point 3 selecting module, a physical object registration module, and a physical object point selecting module.

[0054] The model establishing module is used to:

[0055] The three-dimensional model image 4 is obtained according to the three-dimensional reconstruction of the joint CT image;

[0056] The registration point 3 selecting module is used to:

[0057] Select multiple registration points 3 on the surface of the three-dimensional model image 4;

[0058] a real object registration module for:

[0059] displaying the three-dimensional model image 4 in the augmented reality glasses so that the joint real object 5 and the augmented reality glasses display image 7 are completely overlapped;

[0060] a real object point picking module for:

[0061] selecting points on the joint real object 5 corresponding to the plurality of registration points 3 on the augmented reality glasses display image 7.

[0062] wherein,

[0063] the real object point picking module further comprises a ball-tipped probe 2 for directly operating to select points on the joint real object 5.

[0064] when the real object point picking module performs the point picking operation, the tip of the ball-tipped probe 2 is placed at the position of the registration point 3 on the three-dimensional model image 4.

[0065] the ball-tipped probe 2 is provided with four positioning balls, and the position coordinates of the tip of the ball-tipped probe 2 are calculated by obtaining the center coordinates of the positioning balls and the geometric size of the tip of the ball-tipped probe 2 to the center of the positioning balls.

[0066] the real object point picking module further comprises an optical positioning and tracking device 1, which obtains the coordinate data of each positioning ball on the ball-tipped probe 2 in real time to calculate the attitude information of the ball-tipped probe 2.

[0067] Embodiment:

[0068] the joint bone point selection and registration process is as follows:

[0069] Step 1, taking CT images of the lower limb bones of the patient;

[0070] Step 2, obtaining a three-dimensional model image 4 of the lower limb bones of the patient by means of image segmentation and reconstruction software;

[0071] Step 3, the doctor selects a plurality of registration points 3 on the surface of the three-dimensional model image 4, and displays all the registration points 3 on the three-dimensional model image 4 through three-dimensional visualization technology;

[0072] Step 4, the doctor wears augmented reality glasses, and displays the three-dimensional model image 4 with the registration points 3 in the augmented reality glasses;

[0073] Step 5, using an augmented reality registration algorithm (such as HoloLence) to register the patient's lower limb bone real object with the three-dimensional model image 4 so that they are overlapped in the augmented reality glasses;

[0074] Step 6, using a probe to select points on the real object corresponding to the registration points 3 of the three-dimensional model image 4 in the augmented reality glasses.

[0075] The lower limb skeleton is installed with a skeleton positioning tool 6, and the skeleton positioning tool 6 is provided with a plurality of positioning balls, which are used for assisting in establishing a lower limb skeleton coordinate system. After the point selection is completed, the image coordinate system in the knee replacement surgery robot system can be registered with the coordinate system established by the lower limb skeleton in real time through a point cloud registration algorithm.

[0076] Advantages of the present application:

[0077] The method of the present application is based on augmented reality, and the point selection and registration are performed through augmented reality glasses, so that the error of directly selecting the registration points on the real object is avoided, the registration accuracy between the skeleton image coordinate system and the real object coordinate system is improved, and the system accuracy of the knee replacement surgery robot is improved.

[0078] The above is only the preferred embodiment of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A joint bone landmark registration system, comprising: The model establishing module, the registration point selecting module, the real object registration module and the real object point selecting module are included. The model establishing module is used for: reconstructing a three-dimensional model image according to a joint CT image; The registration point selecting module is used for: selecting a plurality of registration points on the surface of the three-dimensional model image; The real object registration module is used for: displaying the three-dimensional model image in the augmented reality glasses and making the joint real object and the augmented reality glasses display image completely coincide; The real object point selecting module is used for: selecting a plurality of points on the joint real object corresponding to the plurality of registration points on the augmented reality glasses display image, wherein the points on the joint real object are selected by a probe with positioning balls, when the points on the joint real object are selected, the needle tip of the probe with positioning balls is placed at the position of the registration point on the three-dimensional model image, the probe with positioning balls is installed with four positioning balls, the center coordinates of each positioning ball and the geometric size of the needle tip of the probe with positioning balls to the center of the positioning ball are acquired, and the position coordinates of the needle tip of the probe with positioning balls are calculated.

2. The system of claim 1, wherein: The optical positioning and tracking device is further included, which acquires the coordinate data of each positioning ball on the probe with positioning balls in real time to obtain the attitude information of the probe with positioning balls.

Images