Systems and methods for tracking surgical devices

Abstract

The invention is entitled "Systems and methods for tracking surgical devices." Systems, methods, and devices for registering a 3D image of a patient with magnetic coordinates are disclosed. A tri-axial sensor (TAS) can be added to a 3D camera. The position and orientation of the TAS sensor can be determined based on a known magnetic field applied by a magnetic field emitter. The camera coordinate system can then be converted to a magnetic coordinate system. After registration of the 3D image with a CT image and registration of the 3D image with magnetic coordinates is complete, the CT image can then be registered with magnetic coordinates.

Description

[0001] Cross-reference to related applications

[0002] This application claims the benefit of U.S. Provisional Application 63 / 059,805, which is incorporated herein by reference as fully illustrated. Technical Field

[0003] This invention relates in general to the registration of different coordinate systems, and more specifically to the registration of different coordinate systems used in surgical procedures. Background Technology

[0004] In invasive surgical procedures, including minimally invasive surgery, it is often necessary to track surgical devices such as catheters that are not directly visible to the surgeon performing the procedure. Typically, the surgeon can use computed tomography (CT) images, images from fluoroscopy, or magnetic resonance imaging (MRI).

[0005] A method for tracking surgical devices inside a patient's body is the TruDi device manufactured by Acclarent Ltd. (33 Technology Drive, Irvine, CA 92618 USA). TM An electromagnetic image-guided navigation system. In this system, an alternating magnetic field is emitted from a fixed transmitter outside the patient's body so that it passes through the patient. If surgery is being performed on the patient's head, such as ear, nose, and throat (ENT) surgery, the fixed magnetic field transmitter can be placed around the patient's head. As part of the procedure, a surgical device is placed in the center of a calibration chamber, and three orthogonal fields can be applied. Sensors (typically single-axis or multi-axis coils) are attached to the surgical device, inserted into the patient's body, and voltage measurements are taken. A processor records the current generated by the field passing through the sensor. The processor analyzes the current to determine both the position and orientation of the sensor within the electromagnetic reference frame defined by the fixed transmitter.

[0006] Current magnetically based position detection systems can also utilize flexible sensors located on surgical devices, combined with algorithms or processors, to estimate the position, shape, and size of the surgical device based on voltage measurements obtained from the sensors. Flexible sensors typically consist of a single-axis sensor (SAS) with nine transmitters and one sensor, thus obtaining a total of nine measurements at a single point. A triaxial sensor (TAS) includes three sensors with nine transmitters and two coils, thus collecting a total of 27 voltage measurements at a single point. Additional points can be collected in a short time as the surgical device is navigated or advanced.

[0007] Registration of CT images with 3D camera images is TruDi TMPreparatory steps in navigation procedures. Registration involves positioning a surgical device relative to the registered CT image. In other words, by registering or matching the CT image with the 3D image and magnetic coordinates, the position of the surgical device tracked by the electromagnetic tracking system can be accurately determined and displayed on the 3D camera image, CT image, or a combination thereof. In some cases, it may be beneficial to provide the operator with a 3D view of the surfaces of anatomical structures in the patient's head. After completing the registration of the 3D image with the CT image and the 3D image with magnetic coordinates, the CT image can then be registered with the magnetic coordinates. Current systems for registering CT images with 3D camera images require attaching a device (such as a patient tracker) to the patient's head, and certain measures must be taken into account for the patient tracker during registration. A streamlined system and method for registering CT images with 3D camera images is desired. Summary of the Invention

[0008] This invention discloses a system, method, and apparatus for registering 3D images of a patient with magnetic coordinates. A triaxial sensor (TAS) can be added to a 3D camera. The position and orientation of the TAS sensor can be determined based on a known magnetic field applied by a magnetic field emitter. The camera coordinate system can then be transformed to a magnetic coordinate system. After registering the 3D image with a CT image and with magnetic coordinates, the CT image can then be registered with magnetic coordinates. By registering the CT image with magnetic coordinates, the position of a catheter tracked by an electromagnetic tracking system can be accurately displayed on a display of CT images, optical images, or a combination thereof. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of a registration system based on an implementation plan;

[0010] Figure 2A and Figure 2B This illustrates an implementation scheme. Figure 1 A schematic diagram of the patient tracker used in the system;

[0011] Figure 3 This is a flowchart of a method for performing a registration algorithm according to an implementation plan;

[0012] Figure 4 This is a flowchart of a method for performing a registration algorithm according to another implementation scheme;

[0013] Figure 5 It is a schematic diagram of a 3D scatter plot corresponding to an optical image of a patient positioned in a registration system, according to one embodiment; and

[0014] Figures 6A to 6C This is a schematic diagram showing the mapping from a 3D scatter plot corresponding to an optical image to a CT image. Detailed Implementation

[0015] Figure 1 This is a schematic diagram of a registration system 20 according to an embodiment of the present invention.

[0016] Suppose the medical procedure the patient undergoes involves tracking a surgical device, such as a catheter, inserted into the patient's body by a medical professional 25. Tracking is provided by an electromagnetic tracking system 24, which is described in more detail below.

[0017] The electromagnetic tracking system includes a magnetic radiator assembly 26 positioned around the patient's head. Assembly 26 includes magnetic field emitters 28 fixed in place and emitting an alternating sinusoidal magnetic field into the region 30 containing the patient's head. By way of example, the magnetic field emitters 28 of assembly 24 are arranged in a generally horseshoe shape around the patient's head. However, alternative configurations of the radiators of assembly 26 will be apparent to those skilled in the art, and it is assumed that all such configurations are included within the scope of this invention.

[0018] In this paper, a magnetic sensor, assuming the form of a coil, is attached to a tracked surgical device within the patient 22. The attached coil generates electrical signals in response to an alternating magnetic field passing through it, and these signals are transmitted to a system processor 40. The processor 40 is configured to process the signals to derive the sensor's position and orientation values. Other components of the system 20, including the magnetic transmitter 28, are controlled by the system processor 40.

[0019] TruDi mentioned above TM The system uses a tracking system similar to that described herein to locate the position and orientation of the coil in the region irradiated by the magnetic field.

[0020] Processor 40 uses software stored in memory 42 to operate system 20. For example, the software may be downloaded to processor 40 electronically via a network, or additionally or alternatively, the software may be provided and / or stored on a non-transitory tangible medium such as magnetic, optical, or electronic memory. Processor 40 uses the software to analyze signals received from the magnetic sensor. Software executed by processor 40 for implementing registration algorithm 60 of registration system 20 is also stored in memory 42. Registration algorithm 60 is described in more detail below.

[0021] Processor 40 may be mounted in console 50, which includes operating controls 58, typically including a keypad and / or pointing devices such as a mouse or trackball. Console 50 is connected to the radiator via cable 92 and / or wirelessly. Medical professional 25 may use operating controls 58 to interact with the processor while performing the medical procedure described above. During the procedure, the processor may present the results of the procedure on screen 56. The display of the surgical results on screen 56 allows medical professional 25 using the system to visualize the precise location of surgical devices such as catheters relative to the patient's CT images.

[0022] As described above, the electromagnetic tracking system 24 is capable of tracking the position and orientation of the magnetic sensor in region 30 by means of the magnetic field emitted from the magnetic emitter 28 into the region. It should be understood that the position and orientation derived for system 24 are with reference to a frame of reference (FOR) of the magnetic system, such as that defined by the position of the magnetic emitter 28. For the tracking sensor to be useful, the magnetic system FOR needs to be registered with the FOR of an image of the patient 22 stored in memory 42. Subsets 66 and 68 of images 64, further described below, are also stored in memory 42.

[0023] Although CT images can typically include magnetic resonance imaging (MRI) images or fluoroscopic images, in the description herein, by way of example, images are assumed to include fluoroscopic CT images.

[0024] Medical professional 25 uses a three-dimensional (3D) camera 70 to capture a 3D optical image of the patient 22's face. In some embodiments, camera 70 is a RealSense 3D camera manufactured by Intel Corporation (Santa Clara, California). 3D camera 70 may include at least one optical sensor. In some embodiments, 3D camera 70 may include two separate optical sensors. The 3D optical image includes a set of optical voxels, each voxel having three Cartesian coordinates and color values, typically red, green, and blue (RGB) values. This set of optical voxels is also referred to herein as a 3D scatter plot 74, and the optical voxels of scatter plot 74 are stored in memory 42.

[0025] For registration to be performed by system 20, patient tracker 78 is positioned on patient 22. See below for reference. Figure 2A and Figure 2B Describe the patient tracker 78.

[0026] Figure 2A and Figure 2B This is a schematic diagram illustrating a patient tracker 78 according to one embodiment. The patient tracker 78 is formed as a substantially planar sheet, and Figure 2AThe view of the tracker, as seen by camera 70 (i.e., after the tracker has been positioned on patient 22), is shown. Figure 2B This is an exploded view of the tracker.

[0027] In some implementations, the patient tracker 78 is composed of five layered sheets 80A, 80B, 80C, 80D, and 80E, all of which have substantially the same shape and are joined together. Sheet 80A is the upper sheet, also... Figure 2A As shown, multiple optically identifiable markers 82 are incorporated into the sheet. By way of example, sheet 80A includes three optical markers 82. However, other embodiments may include other numbers of markers.

[0028] Sheet 80C is an intermediate layer sheet typically formed of a flexible insulating material, on which planar conductive coils 84, usually in the form of conductive spirals, are formed by printing. The coils 84 act as electromagnetic sensors. The same number of coils 84 as the markers 82 are present, and each coil is positioned on sheet 80C such that it is in a known spatial relationship with the corresponding marker 82. By way of example, each coil 84 is positioned to be directly aligned with the corresponding marker 82 when the tracker sheets are joined together. However, other embodiments may be configured to have different known spatial relationships between the coils and markers. For example, the coils and markers may be offset by a known spatial amount.

[0029] Cable 90 ( Figure 1 (As shown) Connect coil 84 to processor 40. For simplicity, the connection between coil 84 and the cable is not shown. Figure 2A and 2B As shown in the image.

[0030] Sheet 80E is a lower layered sheet formed by a biocompatible adhesive, and it is this sheet that comes into contact with patient 22 during operation of system 20.

[0031] Sheets 80B and 80D are intermediate layered sheets formed of a conductive material to act as electrical shielding for coil 84. Within sheet 80B is a non-conductive region 86 aligned with coil 84. The presence of the non-conductive region 86 enables the coil to function properly. In some embodiments, the non-conductive region 86 is an opening.

[0032] Figure 3 This is a flowchart of a method 300 for performing registration algorithm 60 according to one implementation scheme. (The remaining text appears to be incomplete and possibly contains errors.) Figure 1 The registration system 20 shown executes method 300.

[0033] At point 301, the electromagnetic tracking system 24 is activated, and the patient 22's head is positioned within the system's area 30. The patient tracker 78 is attached to the patient's forehead using a biocompatible adhesive sheet 80E, ensuring the optical landmark 82 is at the top and visible. A cable 90 connects the patient tracker and the processor 40, which can be activated to receive signals transmitted via the cable from the coil 84. The processor analyzes the signals to calculate the coil's position within the FOR defined by the magnetic emitter 28. If the calculated position is found to be within the expected portion of area 30, the processor 40 can provide an indication to the medical professional 25 that the electromagnetic tracking system 24 is functioning correctly. An exemplary indication of the electromagnetic tracking system 24's proper functioning is a notification sent by the processor and displayed on screen 56.

[0034] At 302, processor 40 can analyze CT images of the patient's head stored in memory 42. In some embodiments, processor 40 can analyze the images to identify a subset of CT voxels in the stored images corresponding to surface features of the patient's head, and the subset can be stored as surface subset 66.

[0035] At point 303, the medical professional 25 activates the 3D camera 70 to acquire a 3D optical image of the patient 22's face and stores the acquired image as a scatter plot 74 in memory 42. It should be understood that the image acquired by the 3D camera 70 includes an image of the patient tracker 78 on the patient's face.

[0036] Any suitable algorithm can be used to find the optimal transformation that maps the surface subset of CT voxels 66 to the optical voxels of the 3D scatter plot 74. For example, any point cloud matching algorithm, such as robust point matching and kernel correlation, can be used. In some implementations, the Iterative Closest Point (ICP) algorithm can be used. However, up to 303, there is a known difference between the two voxel sets because the patient tracker's image exists in the scatter plot 74 but not in the CT voxel subset 66.

[0037] At 304, the lack of patient tracker images in CT voxel subset 66 is compensated by adding images of the patient tracker to the CT voxel subset. This addition is achieved by presenting images of the CT voxel subset to a medical professional 25 on screen 56, allowing the medical professional to overlay the patient tracker images onto the presented images, and storing the combined image as the adjusted CT voxel subset 68.

[0038] Alternatively, at 304, a professional 25 selects a portion of the subset 66 that does not include the patient tracker image to derive an adjusted subset 68 from the CT voxel subset 66. The medical professional 25 can perform the selection on the image of the subset 66 presented on the screen 56, and the selected portion is stored as the adjusted CT voxel subset 68.

[0039] At 305, processor 40 maps the adjusted CT voxel subset 68 to voxels in scatter plot 74. If the adjusted CT subset includes images of the patient tracker at 304, mapping can be performed for all voxels in both sets. Alternatively, if 304 is implemented by selecting the portion of subset 66 that does not include images of the patient tracker, processor 40 makes corresponding selections in voxels in scatter plot 74 and performs mapping between the selected voxel sets.

[0040] The mapping provides a registration between the FOR of the CT image of patient 22 and the FOR of the optical image of the patient. The processor 40 can quantize the registration into a first transformation matrix M [CT-OPT], which can be used to transform entities in one reference frame to another FOR.

[0041] At 306, the processor 40 uses the known spatial relationship between the optical landmark 82 and the coil 84 to perform a mapping between the position of the landmark in the optical 3D scattering map 74 and the position of the coil in the FOR of the electromagnetic tracking system 24, as seen at 301. This mapping provides registration between the FOR of the electromagnetic tracking system and the FOR of the optical image, and this can be quantized as a second transformation matrix M[MAGN-OPT].

[0042] At 307, processor 40 combines the two registrations generated at 305 and 306 to produce a third registration between the FOR of electromagnetic tracking system 24 and the FOR of the CT image. The resulting registration can be quantized into a third transformation matrix M[CT-MAGN], and it should be understood that matrix M[CT-MAGN] can be generated from matrices M[MAGN-OPT] and M[CT-OPT].

[0043] As described above, it is possible to include patient tracker 78 Figure 1 Registration system 20 execution Figure 3 The method involves, in some implementations, registration system 20 not including patient tracker 78. Instead, additional sensors may be added to 3D camera 70 to register a 3D image of patient 22's head with the magnetic coordinates of electromagnetic tracking system 24. The additional sensors determine the position and orientation of 3D camera 70. The position and orientation of 3D camera 70 can be used to find the registration between the electromagnetic system and the CT image, as described in more detail below.

[0044] In some implementations, the additional sensor is a triaxial sensor (TAS). The TAS comprises three sensors with nine transmitters and two coils, thereby collecting a total of 27 voltage measurements (3 × 9 = 27) at a single point. The three sensors of the TAS can provide simultaneous measurements in three orthogonal directions. A 3D camera can be tracked and navigated to match the true location (magnetic position and orientation) of the TAS. The position and orientation of the TAS sensor can be determined based on a known magnetic field applied by the magnetic field transmitters. The TAS sensor can be used to track and navigate the 3D camera 70, reading the field configuration to search for the position and orientation of the 3D camera 70. Adding a TAS sensor to the 3D camera 70 provides the ability to register 3D camera images with magnetic coordinates, enhancing the accuracy of CT image registration with magnetic coordinates.

[0045] In other implementations, the additional sensor is a single-axis sensor (SAS) or a dual-axis sensor (DAS). The SAS collects a total of 9 voltage measurements (1×9=9). The DAS collects 18 voltage measurements (2×9=18).

[0046] Figure 4 This is a flowchart of a method 400 for performing a registration algorithm according to one implementation scheme. (The remaining text appears to be incomplete and possibly contains errors.) Figure 1 The registration system 20 shown performs method 400; however, method 400 does not require the use of a patient tracker 78. The 3D camera 70 does not use a patient tracker 78; in addition to at least one optical sensor, the 3D camera 70 includes at least one magnetic sensor. This allows the position and orientation of the 3D camera to be determined and used for registering the 3D image with the CT image. In some embodiments, the magnetic sensor is a TAS, as described above. The TAS can provide simultaneous measurements in three orthogonal directions. In other embodiments, a SAS or DAS can be used instead of a TAS.

[0047] As described above, the 3D camera 70 also includes at least one optical sensor. In some embodiments, the 3D camera 70 may include two separate optical sensors. The 3D optical image includes an optical voxel set, each voxel having three Cartesian coordinates and color values, typically red, green, and blue (RGB) values. This optical voxel set is also referred to herein as a 3D scatter plot 74, and the optical voxels of the scatter plot 74 are stored in memory 42.

[0048] At point 401, the electromagnetic tracking system 24 is activated, and the patient 22's head is placed within area 30 of the system. While the electromagnetic tracking system 24 is activated, magnetic field transmitters 28 are placed at the location of the patient 22's head; these transmitters are fixed in place and emit alternating sinusoidal magnetic fields into area 30.

[0049] At 402, medical professional 25 activates camera 70 to acquire a 3D optical image of patient 22's face. Processor 40 stores the acquired 3D image as a 3D scatter plot 74 in memory 42.

[0050] At 403, processor 40 can be activated to analyze the voltage measurement values ​​of the TAS. Processor 40 is configured to process signals to derive the position and orientation values ​​of the sensor. Other components of system 20 (including magnetic transmitter 28) are controlled by system processor 40. Processor 40 is configured to process signals to derive the position and orientation values ​​of the sensor, and thereby derive the position and orientation values ​​of the camera. Other components of system 20 (including magnetic transmitter 28) are controlled by system processor 40.

[0051] In some implementations, 402 and 403 are performed simultaneously. In other implementations, timestamps can be obtained at 402 and 403 and referenced to each other to ensure they match (i.e., to confirm that the time of acquiring the patient's 3D image at 402 is the same as the time of determining the camera position and orientation at 403).

[0052] At 404, processor 40 transmits the optical 3D scatter plot 74 to the magnetic coordinate system of the electromagnetic tracking system. The position and orientation of the 3D camera 70, determined at 403, can be used to map the 3D scatter plot 74 to magnetic coordinates. Any suitable positioning algorithm can be used to map the 3D scatter plot 74 to magnetic coordinates. For example, point cloud matching algorithms such as ICP or robust point matching can be used. The mapping provides registration between the optical image and the magnetic coordinates.

[0053] At 405, the processor can register the 3D image with the CT image of patient 22. The CT image of patient 22's head can be retrieved from memory 42. Processor 40 can analyze the image to identify CT voxels in the stored image corresponding to surface features of patient 22's head. CT voxels can be mapped to optical voxels of 3D scattering map 74. Any suitable algorithm can be used to find the transformation of the surface that best maps the optical voxels of 3D scattering map 74 to the CT voxels. In some embodiments, the ICP algorithm can be used. This mapping provides registration between the optical image and the CT image.

[0054] In some implementations, after registering the 3D image to the CT image and the 3D image to magnetic coordinates are completed, the CT image can then be registered to magnetic coordinates. For example, processor 40 can combine the registration between the optical image and magnetic coordinates, and the registration between the optical image and the CT image, to produce another registration between the magnetic coordinates and the CT image. Registering the 3D image to magnetic coordinates first can improve the accuracy of the CT image to magnetic coordinates registration.

[0055] Figure 5 This is a schematic diagram of a 3D scatter plot 510 of a 3D optical image of a patient 22 located within region 30 of the registration system, according to one embodiment. The registration system may include... Figure 1 The registration system 20 shown includes, optionally, a patient tracker 78. Figure 5 In the illustration, a 3D scatter plot 510 of the acquired 3D image is superimposed on the face of patient 22.

[0056] Figures 6A to 6C This is a schematic diagram illustrating, according to one embodiment, the mapping of a 3D scatter plot 510 of an acquired 3D image to a CT coordinate system 520 of an electromagnetic tracking system. This mapping can occur relative to... Figure 4 In step 405 of the method described. Figure 6A The 3D scatter plot 510 and CT voxel 520 are shown (at which time they are separated). Figure 6B and Figure 6C This is a schematic diagram of the 3D scatter plot 510 mapped to the CT voxel 520. In some implementations, the algorithm for mapping the 3D scatter plot to magnetic coordinates uses the position and orientation of the 3D camera 70 when acquiring the 3D image, as described above.

[0057] It should be understood that the above embodiments are cited by way of example, and this disclosure is not limited to the content specifically shown and described above. Rather, the scope of this disclosure includes combinations and sub-combinations of the various features described above, as well as variations and modifications thereof, which should be apparent to those skilled in the art upon reading the above description, and which are not disclosed in the prior art.

Claims

1. A method for tracking, comprising: Activate an electromagnetic tracking system on the patient's head, the electromagnetic tracking system including one or more transmitters configured to apply a magnetic field; The system receives a 3D image of the patient's head and stores the 3D image in a scatter plot in a memory, wherein the 3D image is acquired using a 3D camera, the 3D camera including at least one optical sensor and at least one magnetic sensor, wherein the at least one magnetic sensor is a triaxial sensor. The position and orientation of the 3D camera when acquiring the 3D image are determined by the at least one magnetic sensor based on a known magnetic field applied by the one or more transmitters of the electromagnetic tracking system; The scatter plot is registered with the magnetic coordinates of the electromagnetic tracking system using the determined position and orientation of the 3D camera to achieve registration between the optical image and the magnetic coordinates, wherein the registered scatter plot is stored in the memory. The scatter plot is registered with the CT image of the patient's head to achieve registration between the optical image and the CT image; as well as The registration between the optical image and the magnetic coordinates, as well as the registration between the optical image and the CT image, are combined to generate another registration between the magnetic coordinates and the CT image.

2. The method of claim 1, wherein the scatter plot is a reference for two independent optical sensors fixed to the 3D camera.

3. The method of claim 1, wherein the CT image is registered with the magnetic coordinates using an iterative nearest-point algorithm.

4. A system for tracking, comprising: An electromagnetic tracking system, the electromagnetic tracking system including one or more transmitters configured to apply a magnetic field; A 3D camera, the 3D camera including at least one optical sensor and at least one magnetic sensor, wherein the at least one magnetic sensor is a triaxial sensor; A memory configured to store 3D images of the patient’s head acquired by the 3D camera as scatter plots; as well as Processor, the processor being configured to: Start the electromagnetic tracking system; Retrieve the scatter plot from the memory; The position and orientation of the 3D camera when acquiring the 3D image are determined by the at least one magnetic sensor based on a known magnetic field applied by the one or more transmitters of the electromagnetic tracking system; The scatter plot is registered with the magnetic coordinates of the electromagnetic tracking system to achieve registration between the optical image and the magnetic coordinates, wherein the memory is further configured to store the registered scatter plot; The scatter plot is registered with the CT image of the patient's head to achieve registration between the optical image and the CT image; as well as The registration between the optical image and the magnetic coordinates, as well as the registration between the optical image and the CT image, are combined to generate another registration between the magnetic coordinates and the CT image.

5. The system of claim 4, wherein the scatter plot is a reference to the at least one optical sensor fixed to the 3D camera.

6. The system of claim 4, wherein the CT image is registered with the magnetic coordinates using an iterative nearest-point algorithm.

7. A non-transitory computer-readable storage medium, wherein program instructions are stored in the non-transitory computer-readable storage medium, the program instructions, when read by a processor, cause the processor to: Activate an electromagnetic tracking system, the electromagnetic tracking system including one or more transmitters configured to apply a magnetic field; A scatter plot of a 3D image of a patient’s head is retrieved, wherein the 3D image is acquired by a 3D camera, the 3D camera including at least one optical sensor and at least one magnetic sensor, wherein the at least one magnetic sensor is a triaxial sensor; The position and orientation of the 3D camera when acquiring the 3D image are determined by the at least one magnetic sensor based on a known magnetic field applied by the one or more transmitters of the electromagnetic tracking system; The scatter plot is registered with the magnetic coordinates of the electromagnetic tracking system to achieve registration between the optical image and the magnetic coordinates; The scatter plot is registered with the CT image of the patient's head to achieve registration between the optical image and the CT image; as well as The registration between the optical image and the magnetic coordinates, as well as the registration between the optical image and the CT image, are combined to generate another registration between the magnetic coordinates and the CT image.

8. The non-transitory computer-readable storage medium of claim 7, wherein the scatter plot is a reference fixed to the at least one optical sensor.

Images