Image and tracking system registration

Abstract

A medical device includes a registration tool comprising a position sensor, and a position tracking system configured to acquire position coordinates of the sensor in a first reference frame defined by the position tracking system. A processing unit is configured to receive three-dimensional (3D) image data of a patient's body in a second reference frame, generate a 2D image of the patient's surface based on the 3D image data, render the 2D image onto a display screen, and overlay it onto a 2D image icon indicating the location of a corresponding marker. When the registration tool contacts a location on the patient corresponding to the icon on the display screen, the processing unit receives the position coordinates acquired by the position tracking system and registers the first and second reference frames by comparing the position coordinates with the three-dimensional image data.

Description

Technical Field

[0001] The present invention relates generally to image registration, and more specifically to images generated in different modes that can be used for image-guided surgery. Background Technology

[0002] In image-guided surgery, practitioners use instruments that are tracked in real time, allowing the position and / or orientation of the instruments to be visualized on images of the patient's anatomy during the surgical procedure. In some cases, tracking and imaging of the patient's anatomy can be achieved using a single mode, such as fluoroscopy. However, because fluoroscopy uses ionizing radiation, its use should be minimized. Therefore, in many cases, patient images are prepared in one mode (e.g., magnetic resonance imaging (MRI) or computed tomography (CT) fluoroscopy), and instrument tracking uses a different mode, such as magnetic tracking. Summary of the Invention

[0003] The embodiments of the present invention described below provide an improved method for registration of image and tracking systems, as well as systems and software for implementing such methods.

[0004] Therefore, according to one embodiment of the present invention, a medical device is provided, the medical device including a registration tool that includes a position sensor and is configured to contact a surface of a portion of a patient's body. A position tracking system is configured to acquire the position coordinates of the position sensor in a first reference frame, the first reference frame being defined by the position tracking system in the vicinity of the portion of the patient's body. The device further includes a processing unit configured to receive three-dimensional (3D) image data of at least the portion of the patient's body in a second reference frame, generate a two-dimensional (2D) image of the surface of the portion of the patient's body based on the 3D image data, and render the 2D image onto the display screen. The processing unit is further configured to overlay multiple icons onto the displayed 2D image, the multiple icons indicating the position of a corresponding mark on the surface of the part of the body; receive the position coordinates obtained by the position tracking system when the registration tool contacts the position on the surface of the part of the patient's body corresponding to the icon on the display; and register the first reference frame and the second reference frame by comparing the position coordinates with the corresponding position in the three-dimensional image data.

[0005] In the disclosed embodiments, the position tracking system includes a magnetic position tracking system.

[0006] In another embodiment, the three-dimensional image data includes data from a computed tomography (CT) system. Alternatively, the three-dimensional image data includes data from a magnetic resonance imaging (MRI) system.

[0007] In yet another embodiment, the part of the patient's body includes the patient's head, and the 2D image shows the patient's face.

[0008] In another embodiment, registering the first reference frame and the second reference frame includes applying relative scaling, rotation, and translation between the first reference frame and the second reference frame to maximize the correlation between the position coordinates and the corresponding position in the three-dimensional image data.

[0009] In yet another implementation, the processing unit receives the marked location from a user who marks the location on the 2D image.

[0010] In another embodiment, the processing unit is configured to use the registered reference frame to track and display the location of the invasive probe inside that part of the body.

[0011] According to one embodiment of the present invention, a method for registering medical images is also provided. The method includes acquiring position coordinates of a position sensor in the registration tool in a first reference frame, the first reference frame being defined by a position tracking system near the part of the patient's body, when the registration tool contacts a surface of a part of the patient's body; receiving three-dimensional (3D) image data of at least the part of the patient's body in a second reference frame; generating a two-dimensional (2D) image of the surface of the part of the patient's body based on the 3D image data; and rendering the 2D image onto a display screen. The method further includes overlaying a plurality of icons onto the displayed 2D image, the plurality of icons indicating the position of corresponding marks on the surface of the part of the body; receiving the position coordinates acquired by the position tracking system when the registration tool contacts a position on the surface of the part of the patient's body corresponding to the icon on the display screen; and registering the first reference frame and the second reference frame by comparing the position coordinates with corresponding positions in the three-dimensional image data.

[0012] The invention will be more fully understood through the following detailed description of embodiments thereof, taken in conjunction with the accompanying drawings, wherein: Attached Figure Description

[0013] Figure 1 This is a schematic diagram of a medical device according to an embodiment of the present invention;

[0014] Figure 2 This is a schematic illustration of a user interface screen according to an embodiment of the present invention; and

[0015] Figure 3 This is a flowchart illustrating the registration procedure between a 3D image and a position tracking system according to an embodiment of the present invention. Detailed Implementation

[0016] Overview

[0017] Magnetic tracking systems are used to track instruments in invasive diagnostic and therapeutic procedures, such as image-guided surgery using pre-acquired images (e.g., CT images) of a part of the patient's body undergoing the procedure. For tracking to be effective, the image and the tracking system reference frame must be registered to each other. In a typical registration procedure, for example, between a CT image and a magnetic tracking system, the coordinates of several different anatomical points (called landmarks) are marked in the CT image, and the coordinates of the same landmarks are acquired by the tracking system. Once pairs of such coordinate points are acquired, a fitting process is applied to estimate the transformations, including scaling, rotation, and translation, which optimally align (i.e., register) the two sets of points. The fit can be computed, for example, using algorithms known in the art, such as cumulative distance metrics or iterative nearest point (ICP) algorithms.

[0018] As an example, we will consider procedures that require tracking of instruments used on a patient's head, such as ear, nose, and throat (ENT) surgery. In such procedures, the patient's head is registered with a reference frame for the tracking system. An example of such a registration procedure is provided in U.S. Patent Application Publication 2019 / 0046272. In the described procedure, a practicing physician, such as a doctor, positions the distal end of a probe, including a magnetic tracking sensor, at a predetermined number of markers on the patient's skin. These markers correspond to predetermined locations in a CT image. In the cited example, four initial markers are used, including a point located below the tip of the patient's nose, the left and right sides of the patient's face excluding the eyes, and a point located between the eyes, and signals from the tracking sensor are acquired at these points.

[0019] Once the signal from the tracking sensor has been acquired, the processor calculates the corresponding position coordinates in the magnetic assembly reference frame to generate four ordered position pairs, each in the form (tracking sensor position, CT position), where each pair refers to the corresponding position. The system processor uses these four ordered pairs to generate an initial registration, which includes transformations of scaling, translation, and rotation parameters, aligning the CT coordinate system with the magnetic assembly reference frame.

[0020] The physician continues to position the distal end of the probe at other landmarks he / she has defined on the patient's skin. Each time a signal is acquired, the processor uses the coordinates determined by the signal to update the transformation, using the new location as a supplement to the source point cloud.

[0021] To define the landmark points, the current system guides the physician during registration by presenting a diagram of a "generic" head on a screen viewed by the physician, where markers on the image indicate the points to be touched. However, actual patient characteristics can differ significantly from the generic head, making it difficult for physicians to determine where to touch the patient. In extreme cases, the differences may be so large that accurate registration is impossible.

[0022] The embodiments of the present invention described herein address this problem by providing a medical device comprising a position tracking system, a registration tool with a position sensor, a display screen, and a processing unit. By generating a realistic two-dimensional (2D) image of the patient's body from 3D image data and guiding the registration process based on this 2D image, rather than simply using a generic predefined image, accurate registration between the position tracking system and the patient's 3D image is enabled. This method facilitates rapid convergence of the fitting process between the CT coordinate system and the tracking system's coordinate system, achieving more accurate registration than systems known in the art.

[0023] In the disclosed embodiment, the processing unit receives 3D image data of a part of a patient's body and generates a 2D image of the surface of that part of the patient's body based on the 3D image data. The processing unit further renders the 2D image onto a display screen and, under the guidance of a physician, overlays the 2D image onto displayed 2D image icons indicating the positions of corresponding markers on the surface of that part of the body. Guided by these icons on the 2D image, the physician uses a registration tool to touch corresponding points on the patient's body and indicates to the processing unit which point he / she touched. The processing unit receives the corresponding 3D position coordinates from the position sensors obtained by the position tracking system from these points. Finally, the processing unit registers the reference frame of the position tracking system with the reference frame of the 3D image data through relative translation and rotation between the two reference frames until the correlation between the 3D coordinates obtained by the position tracking system and the coordinates of the 3D image data corresponding to the icons is maximized.

[0024] System Description

[0025] Figure 1 This is a schematic illustration of a medical device 10 according to an embodiment of the present invention. Device 10 is used to register a magnetic position tracking system 12 with an image of a patient 14, which, by way of example, is assumed to include a computed tomography (CT) image 48 of the patient 14. The position tracking system 12, by way of example, is assumed to include a magnetic tracking system. Manufactured by Biosense Webster, Irvine, California. The system uses a tracking system similar to that described herein to track the position and orientation of the distal end of a probe inserted or brought near the patient.

[0026] The position tracking system 12 is used to track the position and orientation of one or more instruments (such as catheters or guidewires) inserted into the patient 14 during medical procedures performed on the patient. As described below, the position tracking system 12 is also capable of tracking the position and orientation of a registration probe 16 outside the patient's body. The probe 16 is fixedly attached to a handle 18, which can be held by a practicing physician 20 (typically a doctor) during use of the system 10. The combination of the probe 16 and the handle 18 forms a rigid probe assembly 22, which facilitates positioning the probe to the desired location by the physician 20.

[0027] For clarity and simplicity in the following description, it is assumed that the medical procedure mentioned above includes an invasive procedure on the sinuses of patient 14, such that it is assumed that medical device 10 and magnetic position tracking system 12 are configured to operate within and around the sinus region. However, systems 10 and 12 may alternatively be configured to operate within and around other areas of the patient, such as the chest cavity, kidneys, or abdomen, and those skilled in the art will be able to adapt the description herein for use in such other areas. Furthermore, the principles of the invention can be applied in conjunction with other types of tracking systems (not necessarily magnetic) and other types of 3D imaging modalities (such as MRI).

[0028] The tracking system 12 is operated by a system processor 24, which includes a processing unit 26 that communicates with the probe tracking module 28. The function of module 28 is described below. The system processor 24 may be mounted in a console 30, which includes operating controls 32, typically including pointing devices such as a mouse or trackball. The physician 20 uses the operating controls 32 to transmit commands to the system processor 24, which, as described below, further serves to present data and guiding images to the physician on a display screen 34.

[0029] System processor 24 typically includes a programmable processor that operates device 10 using software stored in the memory of processing unit 26. This software may be downloaded to system processor 24 electronically, for example, via a network, or alternatively, it may be provided and / or stored on a non-transitory tangible medium such as magnetic storage, optical storage, or electronic storage. System processor 24 further stores digitized 3D CT images 48 of the head 38 of patient 14, which may have been acquired at different times by separate CT systems (not shown). CT image 48 includes the 3D coordinates of each point in the image, and the radiographic density of the image at each point, which is typically given in Hounsfield units.

[0030] To track the aforementioned instruments within patient 14 and to track probe 16, processing unit 26 uses probe tracking module 28 to operate multiple magnetic field generators 36, such as coils, via cable 35. In one embodiment, this is typically suitable for situations where patient 14 is anesthetized and head 38 is supine and immobile on bed 40, such as... Figure 1 As shown, generator 36 is fixed to frame 42 placed on the bed next to the patient's head. In an alternative embodiment (not shown), suitable for cases where the patient 14 is not anesthetized, generator 36 is fixed relative to each other and to a frame attached to head 38 or to a chair in the physician's office. Triaxial reference coil 41 is fixed to head 38 and connected to processing unit 26 via cable 43.

[0031] Generator 36 radiates an alternating magnetic field into and around the head 38 of patient 14, and these fields generate signals in the magnetic detector within the instrument and in probe 16. The signals are transmitted back to processing unit 26 and probe tracking module 28 via cable 44 connecting probe 16 to console 30. The processing unit and module together analyze the signals to derive the position and orientation coordinates of the instrument and probe 16 relative to generator 36. Magnetic field generator 36 thus defines a reference coordinate system 46 for magnetic tracking system 12.

[0032] During the registration process, and as Figure 3 As further detailed in the flowchart, processing unit 26 accesses the 3D CT image 48 and renders it onto the 2D image 50 on display screen 34. Processing unit 26 further overlays icons 54 onto selected points on the 2D image 50; these points typically correspond to anatomical landmarks on the patient's face, such as... Figure 2 As further detailed below. The physician 20 sequentially contacts the probe 16 with the surface 52 of the patient 14 (i.e., with the patient's skin) at each point corresponding to icon 54, and the processing unit 26 records the 3D coordinates of the probe at each of these points. Using these recorded coordinates and the coordinates of corresponding points in the reference frame of the CT image 48, the processing unit 26 registers the reference frame of the position tracking system 12 with the reference frame of the 3D CT image 48 by calculating a transformation involving relative scaling, rotation, and translation between the two reference frames. Typically, this transformation is found through a fitting process that maximizes the correlation between the registered 3D coordinates of the probe and the 3D coordinates of the CT image 48 corresponding to icon 54. To maximize the correlation, the processing unit 26 may use algorithms such as cumulative distance metrics or iterative nearest point (ICP) algorithms, such as those used to measure the distance between the probe and the CT image 48. Figure 3 Further details are provided.

[0033] Processing unit 26 can maximize the relevance after all 3D coordinate points corresponding to all icons 54 have been collected. Alternatively, processing unit 26 can begin calculating the relevance with the previous few tracked and recorded points (as also described in cited U.S. Patent Application Publication 2019 / 0046272), and then refine the relevance with each additional recorded point. In an iterative method, processing unit 26 can also estimate whether the 3D coordinates of each next point touched by probe 16 are sufficiently close to the expected location, and indicate acceptable proximity, for example, by coloring the corresponding icon green, or unacceptable proximity by coloring the corresponding icon red.

[0034] Communication is established between physician 20 and processing unit 26 to indicate which point on surface 52 has been touched. For example, processing unit 26 may cause icon 54 to flash on display screen 34, thereby indicating to physician 20 the point he / she expects to touch. Once physician 20 has touched the point, he / she indicates that the point has been touched via control 32 (e.g., by pressing a key on the keyboard or clicking the mouse). Alternatively, each icon 54 may be numbered using a sequence of numbers, and physician 20 indicates which icon he / she has touched via control 32.

[0035] Figure 2 This is a schematic detail view of a 2D image 50 presented on display screen 34 according to an embodiment of the present invention. Image 50 is rendered from a 3D CT image 48 by processing unit 26, and is therefore a realistic and lifelike image of the patient 14's face. To render image 50 from 3D CT image 48, processing unit 26 applies an algorithm such as the Moving Cube algorithm to the CT image. This algorithm assigns a value of 1 to each point in CT image 48 where the radiometric density in Hounsfield units is not zero, and assigns a value of 0 to points where the Hounsfield value is 0 (corresponding to air). The algorithm continues to process image 48, taking eight adjacent locations at a time (thus forming an imaginary cube), and determining the polygons needed to represent the portion passing through the cube that is called an isosurface. (An isosurface is a surface that linearly approximates the boundary between a value of 0 and one of the vertices of the cube.) The corresponding polygons are then fused into a 3D surface, which is projected onto a plane corresponding to the plane of display screen 34.

[0036] Processing unit 26 overlays icons 54 onto points on the 2D image 50 corresponding to facial landmarks. Based on the rendering process, each icon 54 is automatically associated with a corresponding 3D coordinate in the CT reference frame. The physician 20 can define the position of the icons 54 on the image 50, for example, by using control 32 to move the cursor on the display screen 34 as needed to the position he / she sees, and then instruct the processing unit 26 on these positions by clicking the mouse. In this example, the physician 20 has selected these positions in facial regions that are relatively stable, i.e., not significantly compressed under slight pressure from probe 16. Such regions include, for example, the forehead, the tip of the nose, and prominent cheekbones. Alternatively or additionally, the positions of some or all of the icons 54 can be automatically selected by processing unit 26.

[0037] Figure 3 This is a flowchart 100 schematically illustrating the registration procedure between a 3D image 48 and a position tracking system 12 according to an embodiment of the present invention. The registration procedure shown in flowchart 100 refers to... Figures 1 to 2 The elements shown. Alternatively, the principles of this procedure can be applied in conjunction with other types of 3D imaging and tracking systems.

[0038] The procedure begins at start step 102. In 3D image step 104, processing unit 26 receives 3D CT image 48 of the patient 14's face. In rendering step 106, processing unit 26 renders 2D image 50 based on 3D CT image 48, as per [the relevant information]. Figure 2 As described. In overlay step 108, processing unit 26 overlays icon 54, representing a mark on the face of patient 14, onto image 50. In display step 110, processing unit 26 displays image 50 together with icon 54 on display screen 34.

[0039] In coordinate acquisition initiation step 112, physician 20 begins the 3D coordinate acquisition process by touching the face of patient 14 with probe 16. This process then enters loop 115, which includes touch step 114, recording step 117, determination step 116, and next icon step 118. In touch step 114, physician 20 touches the face of patient 14 with probe 16 at the location indicated by icon 54. Communication between physician 20 and processing unit 26 has been referenced above. Figure 1 As described above, further details will not be provided here. In recording step 117, the 3D coordinates of probe 16 sensed by position tracking system 12 are recorded by processing unit 26. After recording the 3D coordinates, processing unit 26 determines in decision step 116 whether more locations need to be touched. If the answer is yes, the next location (icon) is selected by physician 20 or by processing unit 26, as described above, and the physician touches the next location in step 114.

[0040] Once all the required positions (icons) have been determined, the process exits from decision step 116 to calculation step 120, where processing unit 26 calculates the relative scaling, rotation and translation between the two reference frames, typically to maximize the correlation between the 3D coordinates recorded in recording step 117 and the 3D coordinates of the 3D image 48 corresponding to icon 54.

[0041] U.S. Patent 7,855,723 provides an example of an algorithm for maximizing the correlation between two sets of 3D coordinates. The correlation is maximized by iteratively updating scaling, rotation, and translation coefficients to minimize a cumulative distance metric D, which is defined as...

[0042]

[0043] Where d i w is the three-dimensional Euclidean distance calculated between the i-th corresponding points in two sets of coordinates. i These are optional weights; for example, these optional weights describe the confidence level that can be assigned to each point.

[0044] Alternatively or additionally, the Iterative Closest Point (ICP) algorithm referenced in U.S. Patent Application Publication 2019 / 0046272, which is cited above, may be used. The ICP algorithm is also based on minimizing the cumulative distance metric D, with an additional option to switch the points used for 3D point pairs to further minimize the cumulative distance. The ICP algorithm applied to the described embodiment may include the following steps:

[0045] 1. Match each 3D coordinate point of the position tracking system 12 (the point recorded in recording step 117) with the nearest 3D coordinate point of the 3D CT image 48, wherein the proximity of a pair of points is determined by the 3D Euclidean distance d between them. i Decide.

[0046] 2. Estimate the combination of scaling, rotation, and translation that will minimize the cumulative distance metric D. This step may also involve weighting the points before alignment (i.e., for w). i Assigning non-uniform values) and rejecting outliers (i.e., rejecting their d values). i (Point pairs exceeding a preset threshold) Point pairs exceeding a preset threshold before alignment. For example, this step can be performed by calculating an initial estimate of the scaling, rotation, and translation of a small set of point pairs, and then iteratively refining the estimate while incorporating additional point pairs.

[0047] 3. The 3D coordinates of the position tracking system 12 are transformed by using the obtained scaling, rotation and translation transformations.

[0048] 4. Iterate back to step 1 by re-associating the 3D coordinate points of the position tracking system 12 with the 3D coordinate points of the image 48. If the re-association does not reduce the RMS distance metric, accept the last transformation as a coordinate transformation between the coordinate system of the position tracking system 12 and the coordinate system of the CT system.

[0049] Since the process of maximizing correlation can be iterative, a criterion for accepting the maximized correlation can be set, for example, by accepting the maximum correlation, when subsequent iterations increase the correlation (or decrease the cumulative distance metric D) by less than a predetermined threshold. Once the maximum correlation has been reached, the relative rotation and translation between the two reference frames are saved as registration between the frames in registration step 122. The process then ends in termination step 124.

[0050] In an alternative implementation, as described above, the correlation is calculated starting from the 3D coordinates of the first few records, with calculation step 120 moving within loop 115 (not shown here).

[0051] Once the registration process is complete, physician 20 can proceed to perform invasive procedures on patient 14 using system 10. To do this, the physician inserts a probe (not shown) with a position sensor located at or near its distal end into the patient's head, for example, into a sinus passage. Processor 24 tracks the probe's position and uses the transformation established in step 122 to register the position relative to the CT image, providing the physician with precise indication of the probe's position relative to the patient's anatomy.

[0052] It should be understood that the above embodiments are cited by way of example, and the invention is not limited to the specific content shown and described above. Rather, the scope of the invention includes combinations and sub-combinations of the various features described above, as well as variations and modifications thereof, which should be apparent to those skilled in the art upon reading the above description, and which are not disclosed in the prior art.

Claims

1. A medical device, the medical device comprising: A registration tool, the registration tool including a position sensor and configured to contact a surface of a part of the patient's body; A position tracking system configured to acquire the position coordinates of the position sensor in a first reference frame, the first reference frame being defined by the position tracking system in the vicinity of the part of the patient's body; Display screen; and Processing unit, the processing unit being configured to: Receive three-dimensional (3D) image data of at least said portions of the patient's body in a second reference frame; A two-dimensional (2D) image of the surface of the part of the patient's body is generated based on the 3D image data; Render the 2D image onto the display screen; Multiple icons are overlaid on the displayed 2D image, the multiple icons indicating the position of corresponding marks on the surface of the body part; When the registration tool contacts the position on the surface of the part of the patient's body that corresponds to the icon on the display screen, it receives the position coordinates obtained by the position tracking system; as well as The first reference frame and the second reference frame are registered by comparing the position coordinates with the corresponding positions in the three-dimensional image data.

2. The medical device according to claim 1, wherein, The position tracking system includes a magnetic position tracking system.

3. The medical device according to claim 1, wherein, The three-dimensional image data includes data from a computed tomography (CT) system.

4. The medical device according to claim 1, wherein, The three-dimensional image data includes data from a magnetic resonance imaging (MRI) system.

5. The medical device according to claim 1, wherein, The patient's body includes the patient's head, and the 2D image shows the patient's face.

6. The medical device according to claim 1, wherein, Registering the first reference frame and the second reference frame includes applying relative scaling, rotation, and translation between the first reference frame and the second reference frame to maximize the correlation between the position coordinates and the corresponding positions in the three-dimensional image data.

7. The medical device according to claim 1, wherein, The processing unit receives the marked position from the user who marks the position on the 2D image.

8. The device according to any one of claims 1 to 7, wherein, The processing unit is configured to track and display the position of the invasive probe inside the part of the body using a registered reference frame.

9. A method for registering medical images, the method comprising: When the registration tool comes into contact with the surface of a part of the patient's body, the position coordinates of the position sensor in the registration tool in a first reference frame are acquired, the first reference frame being defined by the position tracking system in the vicinity of the part of the patient's body; Receive three-dimensional (3D) image data of at least said portions of the patient's body in a second reference frame; A two-dimensional (2D) image of the surface of the part of the patient's body is generated based on the 3D image data; Render the 2D image onto the display screen; Multiple icons are overlaid on the displayed 2D image, the multiple icons indicating the position of corresponding marks on the surface of the body part; When the registration tool contacts the position on the surface of the part of the patient's body that corresponds to the icon on the display screen, it receives the position coordinates obtained by the position tracking system; as well as The first reference frame and the second reference frame are registered by comparing the position coordinates with the corresponding positions in the three-dimensional image data.

10. The method according to claim 9, wherein, The position tracking system includes a magnetic position tracking system.

11. The method according to claim 9, wherein, Receiving the 3D image data includes receiving computed tomography (CT) images.

12. The method according to claim 9, wherein, The received 3D image data includes magnetic resonance imaging (MRI) data.

13. The method according to claim 9, wherein, The patient's body part includes the patient's head, and generating the 2D image includes generating a 2D image of the patient's face.

14. The method according to any one of claims 9 to 13, wherein, Registering the first reference frame and the second reference frame includes applying relative scaling, rotation, and translation between the first reference frame and the second reference frame to maximize the correlation between the position coordinates and the corresponding positions in the three-dimensional image data.

15. The method according to any one of claims 9 to 13, wherein, The overlay of the icon includes a marker location received from a user who marks the location on the 2D image.

16. The method according to any one of claims 9 to 13, further comprising using a registered reference frame to track and display the position of the invasive probe inside the part of the body.

Images