X-ray equipment navigation and simulation imaging method in image navigation surgery

Abstract

The invention provides an X-ray equipment navigation and simulation imaging method in an image navigation operation, and relates to the technical field of X-ray equipment image simulation, and the method comprises the following steps: setting a tracing target, and carrying out preoperative calibration to obtain camera internal reference of X-ray equipment and a pose transformation matrix from an optical tracer coordinate system on a machine head of the X-ray equipment to a camera coordinate system; a camera internal reference of the X-ray equipment, a pose transformation matrix from an optical tracer coordinate system on a machine head of the X-ray equipment to a camera coordinate system, a three-dimensional image of a patient and an image registration matrix are loaded in the intraoperative navigation software; according to the poses, detected by the optical tracer in real time, of the patient tracer and the optical tracer on the machine head of the X-ray equipment, a pose matrix of a virtual camera in the intra-operative navigation software in a three-dimensional image coordinate system is obtained; rendering the three-dimensional image into a two-dimensional gray projection image, and displaying the two-dimensional gray projection image; whether the region of interest is in the image view of the X-ray equipment or not can be previewed, and the position of the X-ray equipment can be adjusted in an auxiliary mode in an operation.

Description

Technical Field

[0001] The present invention belongs to the technical field of X-ray equipment image simulation, and more specifically, relates to an X-ray equipment navigation and simulation imaging method in image navigation surgery. Background Art

[0002] In image-guided surgery, it is necessary to use optical tracking devices, such as NDI, and intraoperative radiological images, such as CBCT, for positioning and establishing a mapping relationship between the image coordinate system and the surgical space coordinate system. The positioning process requires the installation of tracers on the patient and the surgical instruments. After the surgical instruments are accurately calibrated before the operation, the positional relationship between them and the patient can be tracked in real time during the operation and reflected in the surgical images. However, during the operation, it is usually necessary to collect X-ray images multiple times for reference, verification, and evaluation. The adjustment process of placing the X-ray equipment in the required position is time-consuming and laborious, and often requires multiple shots to try and error to find the correct position and shooting angle.

[0003] Current image-guided surgeries lack navigation and tracking of X-ray equipment, and it is difficult for operators to predict the relative positional relationship between the patient's internal structures and the X-ray equipment. Before taking a light image, the operator needs to rely on experience to determine whether the shooting area of ​​interest is within the imaging range of the X-ray equipment based on the positional relationship between the X-ray equipment and the patient's body surface, and whether it is the required imaging angle. Some existing X-ray equipment can emit laser crosshairs and project them onto the patient's body surface for auxiliary positioning. However, the reference information provided by this solution is limited to the body surface, and it is difficult to help doctors establish the positional relationship of internal structures, such as fracture locations and lesion locations in the body, relative to the X-ray equipment. In addition, the crosshairs cannot provide a reference in the depth direction, that is, the distance of the target area from the imaging plane. Summary of the invention

[0004] The purpose of the present invention is to address the deficiencies in the prior art and to provide a method for X-ray equipment navigation and simulation imaging during image-guided surgery, which can help the operator determine whether the area of ​​interest is within the image field of view through simulated imaging preview before taking the X-ray image, assist in adjusting the position of the X-ray equipment during the operation, and reduce the operation time and the number of X-ray radiation times.

[0005] In order to achieve the above object, the present invention provides a method for navigation and simulation imaging of X-ray equipment in image-guided surgery, comprising:

[0006] By setting the tracer target, preoperative calibration is performed to obtain the camera intrinsic parameters of the X-ray device and the pose transformation matrix from the optical tracer coordinate system on the head of the X-ray device to the camera coordinate system;

[0007] Load the camera intrinsic parameters of the X-ray device, the pose transformation matrix from the coordinate system of the optical tracer on the head of the X-ray device to the camera coordinate system, the patient's three-dimensional image and the image registration matrix into the intraoperative navigation software;

[0008] According to the positions of the patient tracer and the optical tracer on the head of the X-ray device detected in real time by the optical tracker, the position matrix of the virtual camera in the intraoperative navigation software in the three-dimensional image coordinate system is obtained;

[0009] Render the 3D image into a 2D grayscale projection image and display it.

[0010] Optionally, preoperative calibration includes:

[0011] Install optical tracers, tracer targets, and optical trackers;

[0012] Obtaining X-ray images of multiple tracer targets and corresponding placement information of multiple X-ray devices;

[0013] Calculate the camera intrinsic parameters of the X-ray device and the pose transformation matrix of the tracking target of each X-ray image relative to the camera of the X-ray device;

[0014] The camera intrinsic parameters of the X-ray device and the pose transformation matrix from the optical tracer coordinate system on the head of the X-ray device to the camera coordinate system are calculated through the hand-eye calibration algorithm.

[0015] Optionally, installing the optical tracer, the tracer target and the optical tracker comprises:

[0016] Install an optical tracer on the head of the optical device;

[0017] placing a tracer target equipped with an optical tracer within the imaging area of ​​the X-ray device;

[0018] The optical tracker is set at a position where it can simultaneously track the optical tracer on the head of the X-ray device and the optical tracer on the tracer target.

[0019] Optionally, the tracing target is a steel ball array target, which includes a carrier plate and a plurality of steel balls in a checkerboard rectangular array on the carrier plate.

[0020] Optionally, the method also includes detecting the position of the steel balls in each X-ray image by using a Hough circle recognition algorithm and sorting them in order of a chessboard rectangular array.

[0021] Optionally, the positions of the steel balls in the plurality of X-ray images are used by a camera calibration algorithm to calculate the camera intrinsic parameters of the X-ray device and the pose transformation matrix of the tracking target of each X-ray image relative to the camera of the X-ray device.

[0022] Optionally, the image registration matrix is ​​an image registration matrix generated during the registration process of intraoperative navigation software.

[0023] Optionally, after obtaining the pose matrix of the virtual camera in the three-dimensional image coordinate system in the intraoperative navigation software, the method further includes:

[0024] Get the virtual camera origin, focus direction, upward viewing vector direction, and camera viewing angle.

[0025] Optionally, obtaining X-ray images of multiple tracer targets and corresponding placement information of multiple X-ray devices includes:

[0026] Images of multiple tracer targets corresponding to multiple X-ray device placement positions are collected through X-ray equipment.

[0027] The X-ray device navigation and simulation imaging method in image-guided surgery provided by the present invention has the beneficial effects that: the X-ray device navigation and simulation imaging method in image-guided surgery uses a tracer target set in the imaging area of ​​the X-ray device to perform preoperative calibration, thereby obtaining the camera intrinsic parameters of the X-ray device and the pose transformation matrix from the coordinate system of the optical tracer on the head of the X-ray device to the camera coordinate system. The camera intrinsic parameters of the X-ray device and the pose transformation matrix from the coordinate system of the optical tracer on the head of the X-ray device to the camera coordinate system can be stored and loaded into the intraoperative navigation software during the intraoperative navigation process. During the intraoperative navigation process, the intraoperative navigation software also loads the patient's three-dimensional influence and image matching matrix, and the patient's tracer and the X-ray device detected in real time by the X-ray tracker are detected by the intraoperative navigation software. The position and posture of the optical tracer on the machine head is used to obtain the position and posture matrix of the virtual camera in the three-dimensional image coordinate system in the intraoperative navigation software, and the three-dimensional effect is rendered as a two-dimensional grayscale projection image, which is displayed on the screen in real time. Through such real-time display, the operator can determine whether the area of ​​interest is within the image field of view through simulated imaging preview before taking the X-ray image. The navigation process can use optical tracking to navigate the X-ray equipment in real time, continuously simulate imaging, and assist in adjusting the position of the X-ray equipment during surgery. It only requires the installation and calibration of the optical tracer on the machine head once. When using the intraoperative navigation software during surgery, the image can be previewed without further operation, reducing the operation time and the number of X-ray radiation times, and the preview image contains information on the internal structure of the human body that is difficult to observe directly.

[0028] Other features and advantages of the present invention will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The above and other objects, features and advantages of the present invention will become more apparent through a more detailed description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, wherein like reference numerals generally represent like components throughout the exemplary embodiments of the present invention.

[0030] Figure 1 A flow chart of preoperative calibration of an X-ray device navigation and simulation imaging method in image-guided surgery according to an embodiment of the present invention is shown.

[0031] Figure 2 A flowchart of intraoperative navigation of an X-ray device navigation and a simulated imaging method in an image-guided surgery according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0032] The preferred embodiments of the present invention will be described in more detail below. Although the preferred embodiments of the present invention are described below, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0033] like Figure 1 and Figure 2 As shown, the present invention provides a method for navigation and simulation imaging of X-ray equipment in image-guided surgery, comprising:

[0034] By setting the tracer target, preoperative calibration is performed to obtain the camera intrinsic parameters of the X-ray device and the pose transformation matrix from the optical tracer coordinate system on the head of the X-ray device to the camera coordinate system;

[0035] Load the camera intrinsic parameters of the X-ray device, the pose transformation matrix from the coordinate system of the optical tracer on the head of the X-ray device to the camera coordinate system, the patient's three-dimensional image and the image registration matrix into the intraoperative navigation software;

[0036] According to the positions of the patient tracer and the optical tracer on the head of the X-ray device detected in real time by the optical tracker, the position matrix of the virtual camera in the intraoperative navigation software in the three-dimensional image coordinate system is obtained;

[0037] Render the 3D image into a 2D grayscale projection image and display it.

[0038] Specifically, in order to preview whether the region of interest is within the image field of view of the X-ray device and assist in adjusting the position of the X-ray device during surgery, the X-ray device navigation and simulation imaging method during image-navigated surgery provided by the present invention utilizes a tracer target set in the imaging area of ​​the X-ray device to perform preoperative calibration, thereby obtaining the camera intrinsic parameters of the X-ray device and the pose transformation matrix from the coordinate system of the optical tracer on the head of the X-ray device to the camera coordinate system. The camera intrinsic parameters of the X-ray device and the pose transformation matrix from the coordinate system of the optical tracer on the head of the X-ray device to the camera coordinate system can be stored and loaded into the intraoperative navigation software during the intraoperative navigation process. During the intraoperative navigation process, the intraoperative navigation software also loads the patient's three-dimensional influence and image matching matrix, and the patient's tracer detected in real time by the X-ray tracker is detected by the intraoperative tracker. and the position of the optical tracer on the head of the X-ray device, obtain the position matrix of the virtual camera in the three-dimensional image coordinate system in the intraoperative navigation software, and render the three-dimensional effect into a two-dimensional grayscale projection image, which is displayed on the screen in real time; through such real-time display, it can help the operator to determine whether the area of ​​interest is within the image field of view through simulated imaging preview before taking the X-ray image. The navigation process can use optical tracking to navigate the X-ray device in real time, continuously simulate imaging, and assist in adjusting the position of the X-ray device during surgery. It only needs to install and calibrate the optical tracer on the head once. When using the intraoperative navigation software during surgery, the image can be previewed without any more operations, reducing the operation time and the number of X-ray radiation times, and the preview image contains information on the internal structure of the human body that is difficult to directly observe.

[0039] Optionally, preoperative calibration includes:

[0040] Install optical tracers, tracer targets, and optical trackers;

[0041] Obtaining X-ray images of multiple tracer targets and corresponding placement information of multiple X-ray devices;

[0042] Calculate the camera intrinsic parameters of the X-ray device and the pose transformation matrix of the tracking target of each X-ray image relative to the camera of the X-ray device;

[0043] The camera intrinsic parameters of the X-ray device and the pose transformation matrix from the optical tracer coordinate system on the head of the X-ray device to the camera coordinate system are calculated through the hand-eye calibration algorithm.

[0044] Specifically, each X-ray image of a tracer target corresponds to a posture of an X-ray device, the camera internal parameters can be calculated using a camera calibration algorithm, and each X-ray image of a tracer target corresponds to a posture of a camera.

[0045] Optionally, installing the optical tracer, the tracer target and the optical tracker comprises:

[0046] Install an optical tracer on the head of the optical device;

[0047] placing a tracer target equipped with an optical tracer within the imaging area of ​​the X-ray device;

[0048] The optical tracker is set at a position where it can simultaneously track the optical tracer on the head of the X-ray device and the optical tracer on the tracer target.

[0049] Specifically, the optical tracer on the head of the X-ray device and the optical tracer on the tracer target are both within the tracking range of the optical tracker.

[0050] Optionally, the tracing target is a steel ball array target, which includes a carrier plate and a plurality of steel balls in a checkerboard rectangular array on the carrier plate.

[0051] In this embodiment, the tracing target is a steel ball array target, and a plurality of steel balls are arrayed on a carrier plate to form a chessboard-like rectangular array.

[0052] Optionally, the method also includes detecting the position of the steel balls in each X-ray image by using a Hough circle recognition algorithm and sorting them in order of a chessboard rectangular array.

[0053] In this embodiment, the steel ball detection is performed by using the Hough circle recognition algorithm.

[0054] Optionally, the positions of the steel balls in the plurality of X-ray images are used by a camera calibration algorithm to calculate the camera intrinsic parameters of the X-ray device and the pose transformation matrix of the tracking target of each X-ray image relative to the camera of the X-ray device.

[0055] Specifically, based on the detection result of the steel ball detection, the camera intrinsic parameters of the X-ray device and the pose transformation matrix of the tracer target of each X-ray image relative to the camera of the X-ray device are calculated through a camera calibration algorithm.

[0056] The following two formulas are matrix multiplication relations, which are two ways of writing the same content:

[0057]

[0058] In this embodiment, the image registration matrix is ​​an image registration matrix generated during the registration process of the intraoperative navigation software.

[0059] In this embodiment, after obtaining the pose matrix of the virtual camera in the three-dimensional image coordinate system in the intraoperative navigation software, the following steps are also included:

[0060] Get the virtual camera origin, focus direction, upward viewing vector direction, and camera viewing angle.

[0061] Optionally, obtaining X-ray images of multiple tracer targets and corresponding placement information of multiple X-ray devices includes:

[0062] Images of multiple tracer targets corresponding to multiple X-ray device placement positions are collected through X-ray equipment.

[0063] Specifically, each time the position of the X-ray device is changed, an X-ray image is taken, forming a one-to-one correspondence.

[0064] In summary, the X-ray equipment navigation and simulation imaging method in image-guided surgery provided by the present invention, during the preoperative calibration process:

[0065] 1. Install an optical tracer on the head of the X-ray equipment;

[0066] 2. Place the steel ball array tracer target equipped with an optical tracer in the imaging area of ​​the X-ray device, and place the optical tracker (NDI) to a position where it can track both the nose tracer and the target tracer;

[0067] 3. Make sure all the steel balls on the tracer target are within the field of view, use X-ray equipment to collect the image of the tracer target, and then use the optical tracker to record the position of the X-ray equipment and the tracer target at this time;

[0068] 4. Change the placement of the X-ray device and repeat step 3 for n times to obtain n X-ray images and corresponding X-ray device posture information;

[0069] 5. Use the Hough circle recognition algorithm to detect the position of the steel ball in each X-ray image and sort it in the order of the chessboard array; use the camera calibration algorithm to use the steel ball positions in n images to calculate the camera intrinsic parameters of the X-ray device and the position transformation matrix of the tracer target of each X-ray image relative to the camera;

[0070] 6. Using the above results, the hand-eye calibration algorithm is used to calculate the pose transformation from the optical tracer coordinate system on the head of the X-ray equipment to the camera coordinate system, that is, the hand-eye matrix.

[0071] During intraoperative navigation:

[0072] 7. Load the camera intrinsic parameters and hand-eye matrix output by the calibration process, load the patient's 3D image (CBCT or CT), and load the image registration matrix generated during the registration process;

[0073] 8. Calculate the pose matrix of the virtual camera in the three-dimensional image coordinate system according to the pose of the patient tracer and the optical tracer on the head of the X-ray device detected in real time by the optical tracer;

[0074] 9. Calculate the virtual camera origin, focus direction, upward viewing vector direction, and camera viewing angle; render the three-dimensional image into a two-dimensional grayscale projection image and display it on the screen.

[0075] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments.

Claims

1. A method for navigation and simulation imaging of X-ray equipment in image-guided surgery, characterized in that: include: By setting the tracer target, preoperative calibration is performed to obtain the camera intrinsic parameters of the X-ray device and the pose transformation matrix from the optical tracer coordinate system on the head of the X-ray device to the camera coordinate system; Load the camera intrinsic parameters of the X-ray device, the pose transformation matrix from the coordinate system of the optical tracer on the head of the X-ray device to the camera coordinate system, the patient's three-dimensional image and the image registration matrix into the intraoperative navigation software; According to the positions of the patient tracer and the optical tracer on the head of the X-ray device detected in real time by the optical tracker, the position matrix of the virtual camera in the intraoperative navigation software in the three-dimensional image coordinate system is obtained; Render the 3D image into a 2D grayscale projection image and display it.

2. The method for X-ray equipment navigation and simulation imaging in image-guided surgery according to claim 1, characterized in that: Preoperative calibration includes: Install optical tracers, tracer targets, and optical trackers; Obtaining X-ray images of multiple tracer targets and corresponding placement information of multiple X-ray devices; Calculate the camera intrinsic parameters of the X-ray device and the pose transformation matrix of the tracking target of each X-ray image relative to the camera of the X-ray device; The camera intrinsic parameters of the X-ray device and the pose transformation matrix from the optical tracer coordinate system on the head of the X-ray device to the camera coordinate system are calculated through the hand-eye calibration algorithm.

3. The method for X-ray equipment navigation and simulation imaging in image-guided surgery according to claim 2, characterized in that: Installation of optical tracers, tracer targets and optical trackers includes: Install an optical tracer on the head of the optical device; placing a tracer target equipped with an optical tracer within the imaging area of ​​the X-ray device; The optical tracker is set at a position where it can simultaneously track the optical tracer on the head of the X-ray device and the optical tracer on the tracer target.

4. The method for X-ray equipment navigation and simulation imaging in image-guided surgery according to claim 2, characterized in that: The tracing target is a steel ball array target, which includes a carrier plate and a plurality of steel balls in a checkerboard rectangular array on the carrier plate.

5. The method for X-ray equipment navigation and simulation imaging in image-guided surgery according to claim 4, characterized in that: It also includes detecting the position of the steel balls in each X-ray image by using a Hough circle recognition algorithm and sorting them in the order of a chessboard rectangular array.

6. The method for X-ray equipment navigation and simulation imaging in image-guided surgery according to claim 5, characterized in that: The camera calibration algorithm is used to calculate the camera intrinsic parameters of the X-ray device and the position transformation matrix of the tracking target of each X-ray image relative to the camera of the X-ray device using the positions of the steel balls in multiple X-ray images.

7. The method for X-ray equipment navigation and simulation imaging in image-guided surgery according to claim 1, characterized in that: The image registration matrix is ​​the image registration matrix generated during the intraoperative navigation software registration process.

8. The method for X-ray equipment navigation and simulation imaging in image-guided surgery according to claim 1, characterized in that: After obtaining the pose matrix of the virtual camera in the three-dimensional image coordinate system in the intraoperative navigation software, it also includes: Get the virtual camera origin, focus direction, upward viewing vector direction, and camera viewing angle.

9. The method for X-ray equipment navigation and simulation imaging in image-guided surgery according to claim 2, characterized in that: Obtaining X-ray images of multiple tracer targets and corresponding placement information of multiple X-ray devices includes: Images of multiple tracer targets corresponding to multiple X-ray device placement positions are collected through X-ray equipment.

Images