Spinal endoscope augmented reality navigation method

Abstract

The invention discloses a spine endoscope augmented reality navigation method, and relates to the spine endoscope navigation field, and the spine endoscope augmented reality navigation method comprises the following steps: calculating a third matrix from a camera coordinate system to a CT coordinate system according to a first matrix, a second matrix, an endoscope tracker and a spine tracker real time attitude under an NDI coordinate system; calculating the position, focus and orientation parameters of the camera in the world coordinate system through the third matrix, and setting the calculated camera parameters according to the three-dimensional model; superposing the three-dimensional model with the set camera parameters on the endoscope image; according to the method, the global visual field can be obtained after the global spine three-dimensional model is overlaid on the basis of the local visual field image of the endoscope, and the operation convenience is improved.

Description

Technical Field

[0001] The present invention belongs to the field of spinal laparoscopic navigation, and more specifically, relates to a spinal laparoscopic augmented reality navigation method. Background Art

[0002] Doctors can currently only perform surgery through endoscopic images, without augmented reality navigation. During surgery, doctors need to determine the position of the spinal knife under the laparoscope, but the laparoscope has a narrow field of view and cannot determine the position relationship between the laparoscope and the spine. During the operation, tissue fragments cut easily block the laparoscope field of view, thus affecting the positioning of the operation and the doctor's operation of surgical instruments. The accuracy of the position is the main factor affecting the effect of the operation. Summary of the invention

[0003] The purpose of the present invention is to address the deficiencies in the prior art and provide a method for spinal laparoscopic augmented reality navigation, which can obtain a global field of view by superimposing a global spinal three-dimensional model on the local field of view image of the laparoscopy, thereby improving the convenience of the operation.

[0004] In order to achieve the above object, the present invention provides a method for spinal laparoscopic augmented reality navigation, comprising:

[0005] According to the first matrix, the second matrix, the real-time posture of the laparoscope tracker and the spine tracker in the NDI coordinate system, the third matrix from the camera coordinate system to the CT coordinate system is calculated;

[0006] The position, focus and orientation parameters of the camera in the world coordinate system are calculated by the third matrix, and the calculated camera parameters are set according to the three-dimensional model;

[0007] The three-dimensional model after setting the camera parameters is superimposed on the laparoscope image.

[0008] Optionally, the laparoscope takes pictures of the chessboard, and performs camera calibration on multiple groups of pictures to obtain an internal parameter matrix.

[0009] Optionally, an optical navigation system NDI is used to record the posture of the positioning tracker in the NDI coordinate system, and a first matrix is ​​obtained based on multiple sets of posture data and the matrix of the camera.

[0010] Optionally, positioning trackers are respectively arranged on the laparoscope and the spine.

[0011] Optionally, the first matrix is ​​a transfer matrix from the camera coordinate system to the laparoscope positioning tracker coordinate system.

[0012] Optionally, a second matrix from the CT coordinate system to the laparoscopic positioning tracker coordinate system is acquired through preoperative spinal CT image registration.

[0013] Optionally, the number of groups of photos is no less than 20.

[0014] Optionally, the three-dimensional model is a three-dimensional model of the spine.

[0015] Optionally, superimposing the 3D model on the laparoscope image is implemented under the OpenCV library.

[0016] Optionally, the laparoscope image is displayed through different views.

[0017] The present invention provides a method for augmented reality navigation of spinal laparoscopy, which has the beneficial effects of: the method can dynamically synchronize the posture of the laparoscopic spine through a virtual image, can scale the virtual image, and can also set different view displays for the augmented reality image. The image under the laparoscope and the virtual image have a high overlap accuracy at a 1:1 ratio, so that the reality effect can be enhanced by real-time tracking of the three-dimensional model and the laparoscope image through the optical positioning system NDI.

[0018] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present invention.

[0020] Figure 1 A flow chart of a spinal laparoscopic augmented reality navigation method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0021] The preferred embodiments of the present invention will be described in more detail below. Although the preferred embodiments of the present invention are described below, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0022] like Figure 1 As shown, the present invention provides a method for spinal laparoscopic augmented reality navigation, comprising:

[0023] The chessboard is photographed with a laparoscope, and the optical navigation system NDI is used to record the postures of the laparoscope tracker and spine tracker in the NDI coordinate system, with 20 sets of data being selected.

[0024] The camera is calibrated on 20 groups of photos to obtain the intrinsic parameter matrix T1.

[0025] Then, based on 20 sets of laparoscope tracker and spine tracker data and the camera internal parameter matrix, the hand-eye matrix T2 is obtained: the transfer matrix from the camera coordinate system to the laparoscope tracker coordinate system. The hand-eye matrix T2 is used as the first matrix.

[0026] The transfer matrix T3 from the CT coordinate system to the laparoscopic tracker coordinate system is obtained by preoperative spinal CT image registration. The transfer matrix is ​​used as the second matrix.

[0027] The transfer matrix T4 from the camera coordinate system to the CT coordinate system is calculated based on the real-time postures of T2, T3, laparoscope tracker and spine tracker in the NDI coordinate system. The transfer matrix is ​​used as the third matrix.

[0028] The position, focus and orientation parameters of the camera in the world coordinate system are calculated by T4, and the 3D model sets the calculated camera parameters.

[0029] The 3D model after setting the camera parameters is superimposed on the laparoscope image to achieve the augmented reality effect. The augmented image can be set with different views, and the 3D model image can be scaled.

[0030] In addition, the overlay processing of the laparoscope image and the 3D model image can be implemented in the opencv library, or they can be converted to the vtk library format for image overlay operations. During hand-eye calibration, an additional optical tracer can be placed on the calibration board where the checkerboard is located. Since the additional optical tracer and the checkerboard are fixed on the same flat plate, their positions can be kept relatively unchanged, so that the NDI can be easily moved when taking pictures. Otherwise, the NDI needs to be kept stationary during the entire shooting process.

[0031] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A method for spinal laparoscopic augmented reality navigation, characterized in that: include: According to the first matrix, the second matrix, the real-time posture of the laparoscope tracker and the spine tracker in the NDI coordinate system, the third matrix from the camera coordinate system to the CT coordinate system is calculated; The position, focus and orientation parameters of the camera in the world coordinate system are calculated by the third matrix, and the calculated camera parameters are set according to the three-dimensional model; The three-dimensional model after setting the camera parameters is superimposed on the laparoscope image.

2. The method for spinal laparoscopic augmented reality navigation according to claim 1, characterized in that: The laparoscope takes pictures of the chessboard, and the camera is calibrated on multiple sets of pictures to obtain the internal parameter matrix.

3. The method for spinal laparoscopic augmented reality navigation according to claim 1, characterized in that: The optical navigation system NDI is used to record the posture of the positioning tracker in the NDI coordinate system, and the first matrix is ​​obtained according to multiple sets of posture data and the matrix of the camera.

4. The method for spinal laparoscopic augmented reality navigation according to claim 3, characterized in that: The positioning trackers are respectively arranged on the laparoscope and the spine.

5. The method for spinal laparoscopic augmented reality navigation according to claim 4, characterized in that: The first matrix is ​​the transfer matrix from the camera coordinate system to the laparoscope positioning tracker coordinate system.

6. The method for spinal laparoscopic augmented reality navigation according to claim 4, characterized in that: The second matrix from the CT coordinate system to the laparoscopic positioning tracker coordinate system is obtained through preoperative spinal CT image registration.

7. The method for spinal laparoscopic augmented reality navigation according to claim 2, characterized in that: The number of groups of photos should not be less than 20.

8. The method for spinal laparoscopic augmented reality navigation according to claim 1, characterized in that: The three-dimensional model is a three-dimensional model of the spine.

9. The method for spinal laparoscopic augmented reality navigation according to claim 1, characterized in that: The superposition of the 3D model on the laparoscope image is implemented under the OpenCV library.

10. The method for spinal laparoscopic augmented reality navigation according to claim 1, characterized in that: Laparoscopic images are displayed in different views.

Images