Electromagnetic field distortion correction using inside-out tracking

Abstract

The magnetic tracking device is configured to track an object in an environment by receiving a measurement of a non-magnetic signal and a corresponding measurement of a magnetic signal for a location of the magnetic tracking device in the environment. The magnetic tracking device estimates a non-magnetic attitude of the magnetic tracking device in the environment for the location based on the measurement of the non-magnetic signal. The device estimates a magnetic attitude of the magnetic tracking device in the environment for the location based on the measurement of the magnetic signal. The device determines a difference between the magnetic attitude estimate and the non-magnetic attitude estimate for the location. The device determines a magnetic distortion correction value for the location based on the difference. The magnetic tracking device generates a distortion correction model including the distortion value and outputs a representation of the distortion correction model.

Description

Technical Field

[0001] The present disclosure relates to tracking one or more objects in a magnetic field, and in particular to a system for tracking a surgical instrument using electromagnetic (EM) signals. Background Art

[0002] Electromagnetic tracking (EMT) systems are used to assist in understanding the location of instruments and anatomy during medical procedures.Such systems can determine the location of a receiver based on the measured field lines of an emitted magnetic field. Summary of the invention

[0003] Electromagnetic tracking (EMT) systems (also known as magnetic tracking systems) can be used to track devices for multiple applications (e.g., for medical applications used during endoscopic surgery or other types of surgery). EMT systems (also known as magnetic tracking systems) include at least one transmitter and at least one receiver. For example, the transmitter transmits a magnetic signal, and the receiver receives the magnetic signal and measures the magnetic signal. The measured magnetic signal provides information that the magnetic tracking system uses to determine the relative position of the transmitter with respect to the receiver (or vice versa). If the transmitter or receiver is attached to another device (e.g., a tracked device), the magnetic tracking system can determine the relative position of the tracked device in the environment of the magnetic tracking system. In some implementations, the magnetic tracking system can detect distortions in the magnetic signal due to metal objects in the environment. Many additional applications for tracking objects are known.

[0004] The magnetic tracking system described in the document includes a magnetic tracking device, which includes a magnetic signal receiver and a non-magnetic signal receiver. The non-magnetic tracking device may include a visual tracking device, such as a camera. The non-magnetic tracking device is configured to measure additional data in addition to the magnetic signals measured by the magnetic tracking device. The magnetic tracking system is configured to use the additional data (e.g., visual data) to determine how the magnetic signals are distorted in the environment of the magnetic tracking system. More specifically, the magnetic tracking system is configured to use the additional data at these locations as ground truth signals to map these distortions to locations in the environment.

[0005] Typically, a magnetic tracking system is configured to determine the position of one or more tracked objects relative to a transmitter assembly, for example by detecting magnetic signals emitted by the transmitter assembly. Typically, objects in the environment of the magnetic tracking system cause the magnetic field emitted by the transmitter to be distorted in one or more areas of the environment. These distortions can cause errors in estimating the location of the tracked object. It is useful to determine what these distortions are at as many locations as possible in the environment of the magnetic tracking system and to compensate for the distortions during tracking of the tracked object.

[0006] To determine the distortion, the magnetic tracking system performs a calibration process. The magnetic tracking system maps the magnetic signals measured by the receiver at different locations in the environment and compares the measured magnetic signals with the expected undistorted signals at these locations. The exact location of the measured magnetic signal is determined by one or more non-magnetic techniques (e.g., by visual techniques, ultrasonic signals, radio signals, etc.). Once the locations where the receivers measure the magnetic signals are known, the magnetic tracking system can determine the distortion of the magnetic signal at each location.

[0007] Typically, a magnetic tracking system can use visual icons placed around an environment to determine the location of a receiver at various locations. In some implementations, a transmitter can be coupled to a camera and the receiver location is fixed. Visual icons can be placed at known locations in the environment, and a camera coupled to a magnetic receiver is configured to view the icons as the receiver moves around the environment. The visual tracking system provides a position estimate for the receiver. The magnetic tracking system determines the difference between the magnetic signal measured at the estimated location and the expected magnetic signal at that location (as if the magnetic signal of the transmitter was not distorted). For any location of the receiver in the environment, this undistorted expected magnetic signal can be known. In some implementations, the expected magnetic signal can be determined based on an input that receives an indication of the location of the transmitter of the magnetic signal in the environment.

[0008] The technology described herein includes one or more of the following advantages. A magnetic tracking system is configured to more accurately determine the location of a tracked device by compensating for distortion in magnetic signals. Placing visual markers around the environment of the magnetic tracking system to establish a visual reference system is a relatively low-cost solution to compensate for magnetic distortion.

[0009] In a general aspect, a magnetic tracking system is configured to determine an object pose of a tracked object in an environment of the magnetic tracking system. The magnetic tracking system includes a non-magnetic signal detector configured to measure non-magnetic signals in the environment. The magnetic tracking system includes a magnetic signal detector configured to measure magnetic signals in the environment, the magnetic signal detector coupled to the non-magnetic signal detector. The magnetic tracking system includes a computing device configured to perform the following operations. The computing device of the magnetic tracking system receives a measurement of a non-magnetic signal and a corresponding measurement of a magnetic signal for a position of a magnetic tracking device in the environment. The computing device estimates a non-magnetic pose of the magnetic tracking device in the environment for the position based on the measurement of the non-magnetic signal. The computing device estimates a magnetic pose of the magnetic tracking device in the environment for the position based on the measurement of the magnetic signal. The computing device determines a difference between the magnetic pose estimate and the non-magnetic pose estimate for the position. The computing device determines a magnetic distortion correction value for the position based on the difference. The computing device generates a distortion correction model including a distortion value. The computing device outputs a representation of the distortion correction model.

[0010] In some implementations, the non-magnetic signal is generated using a visual marker. In some implementations, the non-magnetic signal detector includes a camera. In some implementations, the marker includes a shape of a transmitter assembly that is different from other shapes of other markers in a plurality of markers in the environment. In some implementations, the marker includes an adhesive configured to removably attach the marker to another surface in the environment. In some implementations, the marker includes an infrared retroreflector, and wherein the non-magnetic signal detector includes an infrared source.

[0011] In some implementations, the non-magnetic signal includes one of an ArUco pattern, a ChArUco pattern, a light source, an ultrasonic source, a radio signal source, or an outer shape of a transmitter assembly. In some implementations, the magnetic signal is generated by a transmitter assembly, which includes: a memory configured to store calibration data related to the transmitter assembly; a processing device configured to control the transmission of the magnetic signal from the transmitter assembly; a communication interface for sending data and receiving data from a computing device; and a power supply configured to provide electrical power to the memory, the processing device, and the communication interface.

[0012] In some implementations, a magnetic tracking system is configured to track a tracked object by performing operations including: receiving measurements of magnetic signals at a specific location in an environment; retrieving a distortion correction model from a memory in communication with a computing device; determining a specific pose estimate for the tracked object; correcting the specific pose estimate based on a value of the distortion correction model corresponding to the specific location; and outputting a representation of the corrected specific pose estimate.

[0013] In some implementations, the tracked object includes one of a catheter, an endoscope, or a surgical instrument.

[0014] In some implementations, the magnetic tracking system is configured to: receive multiple measurements of a magnetic signal corresponding to a known planar surface of an environment, the planar surface associated with a bracket positioned at a known location in the environment; for each of the multiple measurements of the magnetic signal, retrieve a predetermined correction value associated with the location of the measurement in the environment; determine a distortion correction value for each measurement; and include the distortion correction value for each measurement in a distortion correction model.

[0015] In a general aspect, a process for determining an object pose of a tracked object in an environment of a magnetic tracking system includes measuring a non-magnetic signal in the environment by a non-magnetic signal detector. The process includes measuring a magnetic signal in the environment by a magnetic signal detector, wherein the magnetic signal detector is coupled to the non-magnetic signal detector. The process includes receiving a measurement of a non-magnetic signal for a location of a magnetic tracking device in the environment and a corresponding measurement of a magnetic signal. The process includes estimating a non-magnetic pose of the magnetic tracking device in the environment for the location based on the measurement of the non-magnetic signal. The process includes estimating a magnetic pose of the magnetic tracking device in the environment for the location based on the measurement of the magnetic signal. The process includes determining a difference between a magnetic pose estimate and a non-magnetic pose estimate for the location. The process includes determining a magnetic distortion correction value for the location based on the difference. The process includes generating a distortion correction model including distortion values. The process includes outputting a representation of the distortion correction, such as a distortion map.

[0016] In some implementations, the non-magnetic signal is generated using a visual marker. In some implementations, the non-magnetic signal detector includes a camera. In some implementations, the marker includes a shape of a transmitter assembly that is different from other shapes of other markers in a plurality of markers in the environment. In some implementations, the marker includes an adhesive configured to removably attach the marker to another surface in the environment. In some implementations, the marker includes an infrared retroreflector, and wherein the non-magnetic signal detector includes an infrared source.

[0017] In some implementations, the non-magnetic signal includes one of an ArUco pattern, a ChArUco pattern, a light source, an ultrasonic source, a radio signal source, or an outer shape of a transmitter assembly. In some implementations, the magnetic signal is generated by a transmitter assembly, which includes: a memory configured to store calibration data related to the transmitter assembly; a processing device configured to control the transmission of the magnetic signal from the transmitter assembly; a communication interface for sending data and receiving data from a computing device; and a power supply configured to provide electrical power to the memory, the processing device, and the communication interface.

[0018] In some implementations, a magnetic tracking system is configured to track a tracked object by performing operations including: receiving measurements of magnetic signals at a specific location in an environment; retrieving a distortion correction model (e.g., calculation of distortion compensation) from a memory in communication with a computing device; determining a specific pose estimate for the tracked object; correcting the specific pose estimate based on a value of the distortion correction model corresponding to the specific location; and outputting a representation of the corrected specific pose estimate.

[0019] In some implementations, the tracked object includes one of a catheter, an endoscope, or a surgical instrument.

[0020] In some implementations, the magnetic tracking system is configured to: receive multiple measurements of a magnetic signal corresponding to a known planar surface of an environment, the planar surface associated with a bracket positioned at a known location in the environment; for each of the multiple measurements of the magnetic signal, retrieve a predetermined correction value associated with the location of the measurement in the environment; determine a distortion correction value for each measurement; and include the distortion correction value for each measurement in a distortion correction model.

[0021] The details of one or more embodiments of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the subject matter will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1A A diagram showing an environment including an example of a magnetic tracking system for calibrating the magnetic tracking system is shown.

[0023] Figure 1B A diagram showing an environment including an example of a magnetic tracking system for tracking a tracked object.

[0024] Figure 2 A block diagram of the example magnetic tracking system of FIG. 1 is shown.

[0025] Figure 3 A diagram showing an environment including an example of a magnetic tracking system is shown.

[0026] Figure 4A , Figure 4B and Figure 5 An example bracket for calibrating the magnetic tracking system of FIGS. 1-3 is shown.

[0027] Figures 6A-6C is a flow chart of a process for performing magnetic tracking using the magnetic tracking system of FIGS. 1-5 .

[0028] Figure 7 is a block diagram of an example computer system.

[0029] Like reference numbers in the various drawings indicate like elements. DETAILED DESCRIPTION

[0030] Electromagnetic tracking (EMT) systems (also referred to as magnetic tracking systems) can be used in various environments (e.g., medical environments) to track objects (e.g., tracked objects). For example, in a surgical environment, an EMT system can be used to track medical devices (e.g., surgical tools) for one or more purposes (e.g., endoscopic surgery), thereby allowing a medical professional (e.g., a surgeon) to understand the three-dimensional location (e.g., position) and orientation of an object during a medical procedure. Typically, a magnetic tracking system is configured to track an object inside the body to assist a medical professional in performing an operation performed by the medical professional.

[0031] Figure 1A A diagram of an environment including an example of a magnetic tracking system 100 for calibrating the magnetic tracking system using a calibration device 140 is shown. Figure 1B A diagram of an environment including an example of a magnetic tracking system 100 for tracking a tracked object 142 is shown. The magnetic tracking system 100 includes a magnetic signal detector 130 (also referred to as a receiver 130) coupled to a non-magnetic signal detector 110 (e.g., a camera) in a calibration device 140. The magnetic tracking system 100 is configured to determine Figure 1B 1. The position of one or more tracked objects 142 (also referred to as tracked objects) is shown in FIG. The position is relative to a transmitter assembly 112 that is configured to transmit a magnetic signal 132 in the environment of a magnetic tracking system.

[0032] In general, the calibration device 140 is configured to detect the magnetic signal 132 emitted by the transmitter assembly 112 and determine the position and orientation (referred to as the pose) of the calibration device 140 in the environment. Based on the position and orientation of the calibration device 140 in the environment, the distortion in the magnetic signal can be mapped to the environment for subsequent tracking of the tracked object 142. The calibration device 140 uses the non-magnetic signal to determine a location in the environment of the calibration device and determines the distortion of the magnetic signal generated by the transmitter 112 for that location. A distortion map is generated based on the distortion measured at a set of locations in the environment.

[0033] Typically, the magnetic signal in the environment may be distorted by objects in the environment other than the tracked object (referred to as non-tracked objects 120). The non-tracked objects 120 may include objects such as operating tables, chairs, shelves, or any other metallic or magnetically responsive objects in the environment. Typically, the magnetic tracking system 100 is calibrated to update the determined pose of the calibration device 140 based on the known distortion of the magnetic signal 132 caused by the non-tracked objects 120. The calibration may be performed prior to operation of the magnetic tracking system 100 to track the tracked object 142. The calibration generates a distortion correction model that indicates the distortion of the magnetic signal 132 caused by the non-tracked objects 120 for any location in the environment.

[0034] In order to correct the distortion in the magnetic signal 132 caused by the non-tracked object 120, the magnetic tracking system 100 uses a non-magnetic signal that is part of the calibration device 140. The magnetic tracking system 100 uses the non-magnetic signal (e.g., a visual signal) to determine the location of the calibration device 140 in the environment. The magnetic tracking system 100 uses a distortion correction model generated during calibration of the magnetic tracking system 100 to determine the distortion in the magnetic signal 132 caused by the non-tracked object 120 at the location of the calibration device 140. The magnetic tracking system 100 performs distortion correction to remove the distortion caused by the non-tracked object 120 to generate data representing a corrected magnetic signal. The magnetic tracking system 100 uses the corrected magnetic signal to determine the location of the tracked object 142.

[0035] The magnetic tracking system 100 uses a non-magnetic device to determine the pose of the calibration device 140 in the environment. The non-magnetic device (e.g., a non-magnetic signal detector 110) is used to calibrate the magnetic tracking system 100 by generating a distortion correction model as previously described. The non-magnetic signal detector 110 receives a non-magnetic signal (e.g., a visual signal). The calibration device 140 receives a magnetic signal 132 for the same position in the environment as the visual signal from the transmitter 112. The computing device 102 estimates the pose of the calibration device 140 based on the visual signal (e.g., including a representation of the marker array 108) for the position and the corresponding magnetic signal 132. The magnetic tracking system 100 generates a final pose estimate for the calibration device 140 using a first pose estimate from the non-magnetic signal (e.g., a visual signal detectable by the camera 110) and a second pose estimate from the magnetic signal 132. When these operations are performed in real time, the final pose estimate can be referred to as a fused pose estimate because the magnetic tracking system 100 combines pose estimates from magnetic signals and non-magnetic signals. These operations will typically be done using a Kalman filter-based simultaneous localization and mapping (SLAM) process to perform fusion. Examples of these operations can be found in U.S. Patent Application Publication No. 2019 / 0242952, filed on February 8, 2019, and the U.S. Patent Application is incorporated by reference in its entirety into this document.

[0036] Another approach is to collect: 1) a pose estimate from a non-magnetic signal (e.g., a visual signal from a camera 110) and a second pose estimate from a received magnetic signal 132; or 2) a pose estimate from a non-magnetic signal (e.g., from a camera 110) and distortion in the magnetic signal 132 caused by a non-tracked object 120. When poses for both non-magnetic and magnetic signals are collected, a compensation method can be performed to correct the distorted magnetic pose to non-magnetic pose information. The correction is calculated offline but applied to the magnetic pose estimate in real time to perform improved EM tracking. When non-magnetic poses and magnetic signals are collected, the non-magnetic poses are converted to virtual magnetic signals, and the differences between these virtual magnetic signals and the magnetic signals are used to map the distorted magnetic signals to virtual non-magnetic signals. The mapping is calculated offline but applied to the magnetic signals in real time to perform improved EM tracking.

[0037] In order to acquire non-magnetic signals, the non-magnetic signal detector 110 is moved to various positions around the magnetic tracking system 200. Figure 1A In FIG. 1 , an example of a non-magnetic signal detector 110 is a camera. However, other non-magnetic signals may be used to determine the ground truth position of the calibration device 140 in the environment. Figure 2 The camera (and other components of the magnetic tracking system 100 ) are described in more detail. Other examples of the non-magnetic signal detector 110 may include an ultrasonic device, a radio device, an infrared device, an acoustic device, etc. configured to receive the signal and locate the non-magnetic signal detector 110 using the received non-magnetic signal.

[0038] The non-magnetic signal detector 110 may include a camera. The camera may be configured to view markers 106a-e (collectively referred to as markers 106) and marker arrays 108 positioned in the environment to determine the posture of the calibration device 140 (which includes the camera) in the environment. The markers 106 and marker arrays 108 may be optical markers. As the magnetic tracking system 100 (and the camera) moves around the environment, the field of view 144 of the camera passes through the markers 106a-e and / or marker arrays 108. The orientation of the markers relative to each other and relative to the orientation of the camera provide information about the position of the camera in the environment. For example, the position of each of the markers 106a, 106b, 106c and 106d, 106e and the marker array 108 in the environment is known. When the camera field of view 144 records one of the markers 106a-e or the marker array 108 in an image or video, the positioning and orientation of the marker relative to the camera can be discerned from the image or video captured by the camera. The absolute pose of the camera may be discerned using data regarding the location of each of the markers 106a-e or marker array 108 in the environment and the relative pose of the camera with respect to the markers.

[0039] In some implementations, the marker array 108 is placed at or near the location where the tracked object will be located during operation of the magnetic tracking system 100. For example, the marker array can be placed on an operating table, an operating chair, the floor below the tracked object 142, the ceiling above the tracked object 142, and so on. In some implementations, the markers in the marker array 108 can be placed in a Figure 1A In some implementations, the markers in the marker array 108 can be arranged in an irregular pattern that can indicate where in the array the camera is looking when only a portion of the array is visible. In some implementations, the markers in the marker array 108 are each unique in one or more aspects (e.g., pattern, shape, size), or unique in another aspect.

[0040] The magnetic tracking system 100 can use various localization processes to determine the pose of the camera relative to the markers 106a-e and / or the marker array 108. For example, the magnetic tracking system 100 can use a simultaneous localization and mapping (SLAM) process. The SLAM process can use the markers 106a-e and the marker array 108 as key points. In some implementations, each of the markers 106a-e and the marker array 108 is unique, so that each marker can be distinguished from the other markers.

[0041] Once the magnetic tracking system 100 has a pose estimate using the non-magnetic signals, the magnetic signal receiver 130 can be used to measure the magnetic signal 132 at the same pose of the calibration device 140 for which the pose estimate was generated. The magnetic tracking system 100 generates another pose estimate using the magnetic signal 132. The pose estimate generated from the magnetic signal 132 is corrected using the pose estimate from the non-magnetic signals. This process can be repeated as the calibration device 140 moves around the environment.

[0042] like Figure 1A and Figure 1B , the markers 106a-e and the marker array 108 can be placed in a pattern near the location of the tracked object 142 in the environment to be tracked. The calibration device 140 can view the markers 106a-e and / or the marker array 108. In some implementations, the markers 106a-d can be placed on and around non-tracked objects 120 in the environment. In some implementations, one or more markers (e.g., marker 106e) can be located on the transmitter 112. The calibration device 140 can determine the relative pose with respect to the transmitter 112 by viewing the marker(s) 106e on the transmitter assembly 112.

[0043] In some implementations, calibration device 140 may include a transmitter (e.g., transmitter 112), and receiver 130 may be fixed in position in the environment as calibration device 140 moves around the environment. This also enables calibration device 140 to determine its position and orientation in the environment through magnetic units.

[0044] In some implementations, magnetic signal 132 may be directly transformed into pose data based on which pose compensation (eg, distortion compensation) is performed. The pose data from magnetic signal 134 is compared to the optically determined pose of calibration device 140 to generate a distortion-corrected map of the environment.

[0045] For simplicity, throughout this disclosure, an example non-magnetic signal detector 110 including a camera is described in detail as an example embodiment. However, other non-magnetic signal detector 110 configurations are also possible. For example, the non-magnetic signal detector 110 may include a radio signal receiver. Radio beacons (or other radio signal transmitters) may be placed around the environment to transmit signals that can be used to locate the non-magnetic signal detector 110 in the environment. Other frequencies in the electromagnetic spectrum are also possible.

[0046] The processing of the magnetic signal 132 and the non-magnetic signal may be performed by the computing device 102. The computing device 102 may include one or more processors, as described below with respect to Figure 7 For example, the computing device 102 is configured to receive data representing the magnetic signal 132 and the non-magnetic signal and perform a distortion correction process to determine the pose of the calibration device 140 and the tracked object 142 .

[0047] The user interface 104 is configured to communicate with the computing device 102. The user interface 104 is configured to present representations of the poses of the calibration device 140, the tracked object 142, etc. The user interface 104 can be configured to display a three-dimensional representation of the calibration device 140 and the tracked object 142 in the environment. Other such representations of the calibration device 140 and the tracked object 142 and other components of the magnetic tracking system 100 are possible. For example, the user interface 104 can display a textual representation of the pose of the calibration device 140 or other objects in the environment according to a coordinate system (e.g., a Cartesian coordinate system, a polar coordinate system, etc.).

[0048] The user interface 104 may be configured to present a representation of the distortion correction model determined for the environment by the computing device 102. For example, a heat map overlay may show which areas of the environment have greater distortion (eg, have greater heat map values).

[0049] The user interface 104 may include a display for reporting the position and orientation of the tracked object to a user of the magnetic tracking system 100. The position and orientation reported to the user may be used to assist the user in one or more applications, e.g., performing a medical procedure. For example, the user interface may report the position and orientation relative to a portion of the magnetic tracking system 100 as a visual representation of the tracked object, report the coordinates of the tracked object, superimpose the tracked object in an image captured by the camera 110, and the like.

[0050] The user interface 104 may be configured to control the operation of the transmitter assembly 112. The user interface 104 may include one or more controls (software controls, hardware controls, etc.). The controls may be configured to enable a user to turn the transmitter assembly 112 off or on, change the operating frequency of one or more of the transmitter assemblies, cause the transmitter assembly to upload calibration data, and the like.

[0051] In some implementations, the computing device 102 is configured to send or transmit the distortion correction data to one or more other devices. For example, in a medical environment, the computing device 102 can be configured to transmit the distortion correction data to one or more other surgical instruments, magnetic tracking devices, visualization generation devices, etc.

[0052] Go to Figure 1B , shows the magnetic tracking system 100 during tracking of a tracked object 142. The tracked object 142 moves in the environment of the magnetic tracking system 100. The computing device 102 maps distortions based on data measured by the calibration device 140. The position of the tracked object 142 relative to the transmitter 112 is determined by removing distortion effects from non-tracked objects 120 to improve the accuracy of determining the position of the tracked object 142 in the environment. The tracked object 142 includes items such as surgical tools or tool tips that are tracked in the environment. Typically, the calibration device 140 does not need to be present during tracking of the tracked object 142. Although the marker array 108 can still be present in the environment, the distortion has been mapped by the calibration device 140. If the marker interferes with one or more tasks performed by the magnetic tracking system 100 during tracking, the marker can be removed from the environment if necessary, for example, away from the operating table or operating area.

[0053] Go to Figure 2 , shows an example of a magnetic tracking system 200. In some implementations, the magnetic tracking system 200 can be similar to Figure 1A and Figure 1B As previously described, the magnetic tracking system 200 includes a computing device 202 (which may be similar to Figure 1A and Figure 1B1 ), a computing device 202 (similar to user interface 104), a user interface 204 (similar to user interface 104), a camera device 210 (similar to camera device 110), and a transmitter assembly 212 (similar to transmitter 112). Magnetic tracking system 200 also includes amplifiers 232a-b and a magnetic receiver 230 (also referred to as receiver assembly 230 or receiver 230). Various embodiments of components of magnetic tracking system 200 are now described.

[0054] The magnetic tracking system 200 is configured to track the position(s) and orientation(s) of one or more tracked objects (not shown) in the environment of the magnetic tracking system 200. In a medical context, the tracked objects typically include medical devices or parts of medical devices. For example, the magnetic tracking system 200 can be used to track items such as surgical instruments, probes, endoscopes, catheters, etc. (when they are inside the human body). In some implementations, the calibration device 240 (similar to Figure 1A The calibration device 240 of the magnetic tracking system 200 is the tracked object. In some implementations, the tracked object is another object, and the calibration device 240 is used only for distortion correction. The calibration device 240 includes a receiver 130 that senses magnetic signals 132 from the transmitter assembly 212. The magnetic tracking system 200 can determine the location of the receiver 230 of the tracked object based on these signals, as described in more detail later.

[0055] The magnetic tracking system 200 is configured to transmit a magnetic signal (e.g., a magnetic field) from a transmitter assembly 212. The receiver 230 is configured to measure the magnetic field and send the measured signal to the computing device 102. In some implementations, an amplifier 232 is included to amplify the signal measured by the receiver 230. The amplifier 232 typically has a positive gain and is configured to amplify any analog signal received from the receiver 230 so that the computing device 202 can receive the amplified signal as an input. The receiver 230 may include one or more elements for measuring the magnetic signal transmitted by the transmitter assembly 212, such as a magnetometer, a coil, etc. In order to perform tracking of the tracked object, the location and orientation of the receiver 230 are calculated by the computing device 202 using the measurement of the field from the transmitter assembly 212. The location and orientation of the transmitter assembly 212 is known by observing the transmitter assembly 212 using the camera 210a-b or an equivalent non-magnetic sensor.

[0056] Transmitter assembly 212 includes a transmitting element, such as a transmitter coil, configured to generate a magnetic signal. In some implementations, multiple different transmitter assemblies may be used. The transmitter assembly may be slightly different (or even unique) from one or more of the other transmitter assemblies. For example, transmitter assembly 212 may be configured to transmit magnetic signals using different modulation frequencies. In some implementations, each coil has slightly different magnetic properties. Computing device 202 may store information characterizing the magnetic properties of transmitter assembly 212 for calibration purposes. In some implementations, transmitter assembly 212 includes a marker 226 (similar to marker 206), which can be used to optically determine the position of the transmitter assembly relative to camera 210 or calibration device 240.

[0057] During operation of magnetic tracking system 200 , transmitter assembly 212 is configured to transmit a magnetic signal that may be measured by receiver 230 .

[0058] In some implementations, the transmitter assembly 212 is configured to be identifiable by a non-magnetic unit to determine the position and orientation of the transmitter assembly relative to the calibration device 240. The non-magnetic unit may include one or more of an optical unit, an ultrasonic unit, a radio unit, an acoustic unit, an infrared unit, and the like. Typically, one or more of markers 206a-b (which may be similar to markers 106a-e and / or marker array 108) such as optical markers or fiducial markers may be included on or near the transmitter assembly 212. The computing device 202 is configured to recognize the marker, distinguish the marker from other markers, and determine the position and orientation of the marker (and therefore the position and orientation of the transmitter assembly) based on an image of the marker. Although markers 206a and 206b are not necessarily identical in appearance, they may be used to identify the marker. Figure 2 206a and 206b are presented as a group of tags, but each tag 206a and 206b can be an individual tag.

[0059] Although the markers 206a-b can be placed on the transmitter itself, the markers can also be placed throughout the environment. One or more of the markers 206a-b can include a unique pattern so that each marker is different from other markers in the environment. Typically, the markers 206a-b are positioned so that the field of view of the camera 210a-b includes at least one marker at any given posture for distortion correction. The magnetic tracking system 200 is configured to optically determine the posture of the calibration device 240 (which includes the camera 210, the computing device 202, and the receiver 230). As previously described, the optically determined posture is used to determine the distortion present in the magnetic signal 132 received by the receiver. For example, in a medical context, the camera 210 is placed above the patient and the markers 206a-b (e.g., including 106a-e and / or the marker array 108) are placed around the location where the patient surgery is to be performed (e.g., the area of ​​distortion being mapped). The camera 110 is configured to capture images of the markers 206a-b (e.g., including the markers 106a-e and / or the marker array 108) to measure the corresponding magnetic signal 132 for each posture of the calibration device 240. The measured magnetic signal 132 can be compared with the ideal magnetic signal that would exist without distortion. The difference in the magnetic signal can be used to generate a distortion value for each posture in the region. The distortion value for each posture in the region is determined, and its resolution is acceptable for the application being performed (e.g., surgery). For example, the calibration device 240 can move one inch, two inches, five inches, etc. between measurements. For example, the calibration device 240 can rotate 1 degree, 2 degrees, 5 degrees, etc. between measurements as needed.

[0060] In some implementations, the camera 210 is removed (or not included) to perform magnetic tracking during operation of the calibration device 240. For example, during surgery, the camera 210 may be disconnected from the calibration device 240.

[0061] In some implementations, the camera 210 can be a stereo camera that includes two cameras that are displaced from each other, for example, camera 210a and camera 210b. The cameras 210a, 210b are configured to capture images from different angles relative to the markers 206a-b. The images from the stereo cameras 210a, 210b can be used to determine not only the planar "x, y" positioning data and yaw of the calibration device 240, but also the depth "z" positioning of the calibration device 240 and the roll and pitch of the magnetic tracking device relative to the markers 206a-b. In some implementations, a single camera is used to determine depth data using multiple markers 206 on the same object (e.g., multiple markers spaced a known distance from each other).

[0062] The computing device 202 uses the data representing the relative position and orientation of the calibration device 240 to determine how to interpret the magnetic signals received from the receiver 230. The magnetic signals indicate the position and orientation of the tracked device relative to the transmitter assembly 212. In order to determine the absolute position and orientation of the tracked device, the position and orientation of the markers are determined. Typically, each of the markers has a known position and orientation relative to the transmitter assembly 212. The coordinate system established by these markers is a global coordinate system (e.g., Figure 1A and Figure 1B Cartesian coordinate axes shown in ).

[0063] Receiver 230 is configured to measure the magnetic signal emitted by transmitter assembly 212 to determine the position and orientation of the tracked object relative to the transmitter assembly. The position of the tracked object can be measured relative to any global reference point (e.g., transmitter assembly 212). Computing device 202 is configured to convert the measured magnetic signal into positioning and orientation data. In some implementations, the positioning data can be represented as a positioning vector of positioning coordinates (e.g., x, y, z coordinates). In this example, receiver 230 uses a Cartesian coordinate system (with x, y, and z coordinates) to represent a position in space; however, other types of coordinate systems (e.g., cylindrical, spherical, etc.) can also be used.

[0064] The orientation of a tracked object refers to the direction that the tracked device is facing relative to a global reference point (e.g., transmitter assembly 212), and can be similarly represented using a coordinate system, and represented, for example, as a vector of orientation coordinates (e.g., azimuth (ψ), altitude (θ), and roll (φ)). The magnetic tracking system 200 operates as a measurement system with up to six degrees of freedom (6DoF), which is configured to allow measurement of positioning and orientation information related to front / back positioning, up / down positioning, left / right positioning, azimuth, altitude, and roll. For example, if the receiver 230 includes a single receiving coil, a group of at least five transmitter assemblies 212 can provide five degrees of freedom (e.g., no roll). In one example, if the receiver 230 includes at least two receiving coils, a group of at least six transmitter assemblies 212 can provide sufficient data to determine all six degrees of freedom. Additional transmitter assemblies or receiving coils can be added to increase tracking accuracy or allow for larger tracking volumes.

[0065] Computing device 202 includes one or more processors and is configured to receive data representing magnetic signals 132 and non-magnetic signals to estimate distortion of magnetic signals in the environment. Computing device 202 receives magnetic signals from receiver 230 and converts the magnetic signals into positioning data and orientation data for calibration device 240. Computing device 202 may include input ports and output ports for sending and receiving both analog data and digital data. Computing device 202 may include a waveform generator (not shown) for driving transmitter assembly 212. Figure 7 Aspects and examples of computing device 102 are further described.

[0066] The computing device 202 may include circuitry for driving the transmitter assembly 212 and controlling the operation of the transmitter assembly 212. For example, the computing device 202 may include a controller configured to control the transmitter assembly 212. The transmitter assembly 212 may be configured to transmit magnetic signals at different times or frequencies during the measurement cycle. For example, the computing device 202 may be configured to control the transmitter assembly 212 to transmit magnetic signals at specific times, transmit magnetic signals in a specific sequence, and so on during the measurement cycle. The receiver 230 measures each of the magnetic signals. If a timing mechanism is used, the computing device 202 may associate the received magnetic signal with a specific transmitter assembly based on the time at which the computing device 202 receives the magnetic signal from the receiver 230. The controller may control the transmitter assembly 212 using time slicing multiplexing, frequency multiplexing, and the like.

[0067] The transmitter assemblies 212 may be calibrated individually. In some implementations, the transmitter assemblies 212 have one or more different physical properties. For example, the transmitter assemblies 212 may be calibrated using modeling parameters, mapping of magnetic fields from magnetic signals, spherical harmonics, closed form solutions, and the like. The calibration data may be stored by the computing device 202. The calibration data for the transmitter assemblies may be stored locally on a local storage device along with the corresponding transmitter assemblies. In one aspect, the calibration data may be sent (e.g., wired or wirelessly) to the computing device 202 to help the computing device 202 determine the location and orientation of the transmitter assemblies. As previously stated, the transmitter assemblies 212 may be configured to operate at frequencies different from one another, to be turned on at different times, and the like.

[0068] The transmitter coil (not shown) is configured to generate a magnetic signal received by the receiver 230. The coil can transmit a signal (e.g., a magnetic field) that is unique to a particular transmitter assembly 212. For example, the transmitter assembly 212 can modulate the magnetic signal with a specific frequency. The coil can be a single-turn coil or a multi-turn coil. The coil can include any geometric shape that can generate a magnetic field when an electric current is provided. The coil can be part of the circuitry of the transmitter assembly 212 (e.g., a printed circuit board (PCB)), or the coil can be separately attached to the transmitter assembly.

[0069] The transmitter assembly 212 may not include active circuitry. The coil may be driven from a remote source (e.g., a waveform generator of the computing device 202). The transmitter assembly 212 may be configured to be plugged into the computing device 102 (or another device) to drive a magnetic signal. In some implementations, the transmitter assembly 212 may be configured to be connected to one or more other transmitter assemblies, for example, in a daisy chain. In another example, the transmitter assembly 212 may be connected in parallel with one or more other transmitter assemblies.

[0070] Computing device 202 may be configured to determine the position and orientation of the tracked object in a variety of ways. For example, a least squares method may be used to determine the position and orientation of receiver 230 relative to transmitter assembly 212. In another example, a Kalman filter or one or more other numerical methods may be used to determine the position and orientation of receiver 230 relative to transmitter assembly 212.

[0071] The computing device 202, the transmitter assembly 212, the camera 210, and the user interface 204 can communicate with each other via a wired or wireless connection. For example, the transmitter assembly 212 can be wired to a port of the computing device 202. In such a configuration, the computing device 202 can provide a power signal to drive the transmitter assembly 212, and the transmitter assemblies can each include passive electronic devices. In another example, the transmitter assembly 212 can be equipped with a data transceiver that is configured to wirelessly transmit data (e.g., calibration information) to the computing device and receive data (e.g., control signals) from the computing device 202.

[0072] The markers 206a-b may be rearranged during distortion correction for the magnetic tracking system 200, and the camera 210 data may be used by the computing device 202 to determine a new position and orientation of each of the markers. Figure 1AThe marker array 106a-e and / or marker array 108) can be moved around during calibration to ensure that one or more of the markers are within a few inches of where the tracked object (e.g., catheter) will be as it moves around the environment (e.g., within a patient). Once the markers 206a-b are in position in the environment, the calibration device 140 is moved around the environment to calibrate the distortion correction model for that area.

[0073] In some implementations, markers 206a-b each include an adhesive patch that can be attached to one or more surfaces of the environment of magnetic tracking system 200. For example, markers 206a-b can be attached to a subject (eg, a patient) of a medical procedure.

[0074] The combination of receiver 230 and transmitter assembly 212 can be a distortion indicator or distortion correction system for the environment. The magnetic tracking system 200 can use the determined difference between the optical determination of the optical attitude of each assembly in the assembly and the EM determination of the attitude as an indication of the distortion present in the environment. The magnetic tracking system 200 can therefore perform distortion compensation. For example, data collected by the receiver assembly 230 (including magnetic field measurements and attitude, distorted data, optical data (real) and gradient data (estimated by the difference in the field as the sensor moves)) can be used to correct for distortion. Distortion can be modeled using physical models. These models can include curve fitting (e.g., for magnetic signal and attitude solutions), splines, triangulation, radial basis functions, and using machine learning methods.

[0075] Go to Figure 3 , an example of a magnetic tracking system 200 for distortion correction of magnetic signals 132 is shown. Markers 306a-g (collectively, markers 306) are placed throughout the magnetic tracking system 200. Markers 306 may be arranged or rearranged depending on the location in the magnetic tracking system 200 where distortion correction is being performed, similar to markers 106 and 206 described previously. Generally, markers 306 are placed such that at least one marker is visible to a camera 310 (which may be similar to calibration devices 140 and / or 240) of a calibration device 340 (similar to calibration devices 140 and / or 240) in any posture of the magnetic tracking device. Figure 1A The camera 110 and / or Figure 2 The camera 210) is visible.

[0076] In one aspect, one or more markers 306 may include any visual mechanism for distinguishing markers from one another. Markers 306 are each distinct from one another so that computing device 302 (which may be similar to computing devices 102 and 202 described previously) can distinguish the markers when visually determining the pose of calibration device 340. The positioning of markers 306 relative to one another indicates to computing device 302 how transmitter assembly 312 is positioned and oriented relative to camera 310, receiver 330 (e.g., similar to receiver 130 and / or receiver 230 described previously), or other portions of magnetic tracking system 300. For example, if marker 306b appears in a positive direction along the x-axis relative to icon 306c, the computing system may determine that transmitter assembly 312 is rotated (e.g., rotated relative to the plane of camera 310) by a particular yaw value. Alternatively, computer vision or machine learning methods may be used to track transmitter assembly 312, as is known in the art. Each of the markers 306 can be selected from an icon library that the computing device 302 is configured to recognize and assign to different locations in the magnetic tracking system 200. The computing device 302 associates the received magnetic signal with a location determined based on which marker 306 is seen and what its orientation is. This helps with location and orientation calculations. In other words, the computing device 302 uses the markers 306 to know which magnetic gesture is associated with which visual gesture. The computing device 302 can then determine what the ideal magnetic signal is at the ground truth location and determine the magnetic distortion value.

[0077] In an aspect, camera device 310 may include a stereo camera that provides distance information to computing device 302. Computing device 302 may use the distance information (whether determined at camera device 310 or calculated at computing device 302) to determine a three-dimensional position of transmitter assembly 312. Camera device 310 may include a single camera that may determine distance based on the use of multiple markers 306a-h and their known geometry.

[0078] As previously stated, the markers 306a-h are generally configured to be recognized for computer vision recognition (similar to the markers 106 and 206 and the marker array 108). The markers can include any image, such as a barcode, a QR code, a symbol, an icon, etc. In the example of the magnetic tracking system 300, the markers 306a-h are pixelated symbols. Although for each marker 306, less than 8 symbols are included in this example, additional symbols can be added to determine the posture. In addition to being a symbol, the marker can also include a retroreflector configured for infrared excitation or another type of key point that can be detected by a non-magnetic unit.

[0079] In some implementations, markers 306a-h may be varied, as long as the configuration provides sufficient information for the magnetic tracking system 300 to generate an estimate of the visual pose. For example, the marker may include an ArUco or ChArUco pattern. For example, the marker may include a passive retroreflector that responds to IR excitation from an infrared (IR) source near the camera apertures 110a and 110b. The markers 306a-f may include active beacons that transmit light signals, radio frequency data, ultrasonic signals, or other pose information. In some implementations, the shape of the marker 306a itself may be used for pose determination.

[0080] On the one hand, each mark 306a-f can include an adhesive layer applied to one side. The adhesive layer is configured to adhere the mark 306 to another surface, such as a wall, ceiling, non-tracking object, etc. The surface is not required to be flat or regular. The adhesive layer can include an adhesive gel, glue, suction cup or other adhesive surface. The adhesive layer is usually configured to be removable and repeatedly reapplied to the surface. The adhesive layer will not obscure the side of the mark 306 that includes the symbol.

[0081] Go to Figures 4A-4B , a calibration environment with a stand 400 is shown. The stand allows a calibration device 440 (e.g., similar to calibration devices 140, 240, and / or 340) to move in a known area of ​​the environment relative to a marker 406 (similar to markers 106, 206, and 306, as previously described). For example, the stand 400 allows planar movement of the calibration device 440 relative to the marker 406. A plurality of planes P1, P2, P3, P4, P5 (and so on) may be provided in the stand 400. At each plane, the calibration device 440 may be positioned and the magnetic signal 132 measured. Due to the fixed nature of the stand 400, the pose of the calibration device 440 is known with high accuracy. Shelves (or other support devices) may be added or removed to allow the calibration device 440 to move relative to the marker 406. Various measurements of the magnetic signal 132 may be made at each of the planes. A distortion correction model may then be generated as previously described. For example, in Figure 4A In FIG. 4 , the calibration device 440 is located at position P1. Figure 4B In FIG. 4 , the calibration device 440 is located at position P2. The planes can be located close together to increase the resolution of the distortion correction model as needed. The calibration device 440 can be moved around the planes to capture a large number of magnetic signal measurements at the planes.

[0082] Go to Figure 5, a calibration environment with a stand 500 is shown. The stand includes stackable layers 504 that can be quickly built or disassembled, for example, on a patient on an operating table. The stand 500 is portable and allows a magnetic tracking device 502 (e.g., similar to magnetic tracking systems 200, 300) to move in a known area of ​​the environment relative to a transmitter 112. For example, the stand 500 allows planar motion of the magnetic tracking device 502 in the environment. In addition to including markings, the stand can be formed with precise dimensions so that the absolute positioning of the magnetic tracking device 502 along the longitudinal direction is known. An encoder (e.g., a laser-based encoder) can be used to track the position of the magnetic tracking device 502 as it moves around the plane. For example, a computer mouse coupled to a receiver (similar to receivers 130, 230 and / or 330) can be used to measure a magnetic signal 132 at each position on the plane. For each plane, the magnetic tracking device 502 can be iteratively positioned at various points on the plane and the magnetic signal 132 is measured at each point. Due to the fixed nature of the stand 500, the pose of the magnetic tracking device 502 is known with high accuracy. Shelves (or other support devices) can be added or removed to allow the magnetic tracking device 502 to move in additional dimensions. The distortion correction model can then be generated as previously described. The magnetic tracking device 502 can be moved around a plane to capture a large number of magnetic signal measurements at the plane.

[0083] Go to Fig. 6A , a flow chart of a process 600 for performing magnetic tracking using the magnetic tracking system and transmitter assembly of Figures 1-5 is shown. Process 600 represents how a magnetic tracking system (e.g., the magnetic tracking system of Figures 1-5) is configured to determine an object pose of a tracked object in an environment of the magnetic tracking system.

[0084] The computing system is configured to receive (602) a measurement of a non-magnetic signal and a corresponding measurement of a magnetic signal from a magnetic tracking device. The non-magnetic signal may include a visual signal, such as an image or video. The computing system estimates (604) a non-magnetic attitude of the magnetic tracking device in an environment for the location based on the measurement of the non-magnetic signal. The computing system is configured to estimate (606) a magnetic attitude of the magnetic tracking device in an environment for the same location in the environment based on the measurement of the magnetic signal. In some implementations, the magnetic attitude and the visual attitude are estimated in parallel. In some implementations, the estimated attitude may be saved to a buffer to pipeline the generation of a distortion correction model. For example, the computing system determines (608) a difference between the magnetic attitude estimate and the non-magnetic attitude estimate to generate an error value. When the difference is determined, the computing system may be configured to measure the next iteration of the magnetic signal and the non-magnetic signal to estimate a second magnetic attitude and a second non-magnetic attitude.

[0085] The computing system is configured to determine (610) a magnetic distortion correction value for associating with a particular location in the environment based on the determined difference in attitude values. Once the distortion values ​​are calculated for multiple locations, the computing system can establish or generate (612) a distortion correction model and output a representation of the distortion correction model, which can be formed by a distortion model, spherical harmonics, mathematical functions, etc., and can be performed using pre-computed corrections or when evaluating the model in real time, the distortion correction model including the distortion values. The distortion correction model can be stored in a memory so that the distortion correction can be retrieved to correct the magnetic attitude determined by the magnetic tracking system as the magnetic tracking system moves around the environment. In other words, because the distortion value for a given location in the environment is known, once object tracking is performed, the magnetic tracking system can improve the magnetic attitude estimate of the tracked object. During tracking, the computing device is configured to output a representation of the tracked object attitude, for example to a user interface.

[0086] In some implementations, the marking includes one of an ArUco pattern, a ChArUco pattern, an infrared retroreflector, a light source, an ultrasonic source, a radio signal source, and an outer shape of a transmitter assembly.

[0087] Figure 6B A process 620 of using inside-out tracking to correct for distortion of the electromagnetic field is shown. Process 620 shows a function fitting process where the non-magnetic signal is a function of the magnetic signal. For example, the x-position of the magnetic signal can be compensated by a polynomial composed of powers of the magnetic signal so that it is equal to the non-magnetic signal. These functions take a variety of forms, such as polynomials, radial basis functions, spherical harmonics, kriging, neural networks, splines, etc. The function fitting process is typically performed offline after all signal data is captured. The magnetic signal can be in the measurement domain (magnetic field) and / or the tracking (solution or attitude) domain.

[0088] A computing device (e.g., computing device 102, 202, or 302 of FIGS. 1-3) is configured to receive (622) a measurement of a non-magnetic signal and a corresponding measurement of a magnetic signal. The computing device is configured to generate (624) a mapping function that relates the magnetic signal to the corresponding non-magnetic signal. The computing device is configured to store (626) the mapping function for use during magnetic tracking.

[0089] Figure 6C A process 630 is shown for distortion correction of electromagnetic fields using inside-out tracking. Process 630 shows a process for using the function generated by process 620 during EM-only tracking. A magnetic signal is received and used in a function to generate a corrected magnetic signal. The corrected magnetic signal can be in the measurement domain (magnetic field) and / or the tracking (e.g., solution or attitude) domain. If the corrected magnetic signal is in the measurement domain, the corrected magnetic signal is run through a standard tracking algorithm to generate attitude data.

[0090] The computing device (eg, computing device 102, 202, or 302 of FIGS. 1-3) is configured to receive (632) measurements of magnetic signals during the tracking process. The computing device is configured to receive from Figure 6B The mapping function generated by the calibration process is applied (634) to the received signal. The computing device is configured to output (636) a corrected magnetic signal indicating the true position and orientation of the tracked device.

[0091] Figure 7 is a block diagram of an example computer system 700 . Figure 1A-1B The computing device 102 of the present invention can be an example of a computer system 700 described herein. The system 700 may include a processor 710, a memory 720, a storage device 730, and an input / output device 740. Each of the components 710, 720, 730, and 740 may be interconnected, for example, using a system bus 750. The processor 710 is capable of processing instructions for execution within the system 700. The processor 710 may be a single-threaded processor, a multi-threaded processor, or a quantum computer. The processor 710 is capable of processing instructions stored on the memory 720 or the storage device 730. The processor 710 may perform operations such as causing the EMT system 100 to determine the location and / or orientation of the tracked device 102.

[0092] The memory 720 stores information within the system 700. In some implementations, the memory 720 is a computer-readable medium. The memory 720 can be, for example, a volatile memory unit or a non-volatile memory unit.

[0093] The storage device 730 can provide mass storage for the system 700. In one aspect, the storage device 730 is a non-transitory computer-readable medium. The storage device 730 may include, for example, a hard disk device, an optical disk device, a solid-state drive, a flash drive, a tape, or some other mass storage device. The storage device 730 may alternatively be a cloud storage device, for example, a logical storage device including multiple physical storage devices distributed on a network and accessed using the network. In some implementations, the information stored on the memory 720 may also or alternatively be stored on the storage device 730.

[0094] The input / output device 740 provides input / output operations for the system 700. In some examples, the input / output device 740 includes one or more of a network interface device (e.g., an Ethernet card), a serial communication device (e.g., an RS-232 10 port), and / or a wireless interface device (e.g., a short-range wireless communication device, a 602.11 card, a 3G wireless modem, or a 4G wireless modem). Typically, the input / output device 740 includes a driver device configured to receive input data and send output data to other input / output devices (e.g., a keyboard, a printer, and a display device). In some implementations, mobile computing devices, mobile communication devices, and other devices are used.

[0095] System 700 may include a microcontroller. A microcontroller is a device that contains multiple elements of a computer system in a single electronic package. For example, a single electronic package may contain a processor 710, a memory 720, a storage device 730, and an input / output device 740.

[0096] Despite Figure 7 An example computer system has been described in the specification, but the subject matter and implementation of the functional operations described above may be implemented in other types of digital electronic circuits, or in computer software, firmware, or hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of them. The implementation of the subject matter described in this specification may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible program carrier (e.g., a computer-readable medium) to be executed by a processing system or to control the operation of a processing system. The computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that effects a machine-readable propagated signal, or a combination of one or more of them.

[0097] The term "computer system" may encompass all devices, apparatuses, and machines for processing data, including, by way of example, a programmable processor, a computer, or multiple processors or computers. In addition to hardware, a processing system may also include code that creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.

[0098] A computer program (also referred to as a program, software, software application, script, executable logic, or code) may be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages, and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store portions of one or more modules, subroutines, or code). A computer program may be deployed to execute on one computer, or on multiple computers located at one site or distributed across multiple sites and interconnected by a communications network.

[0099] Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile or volatile memory, media, and memory devices, including, by way of example, semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks or tapes; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated in special purpose logic circuitry. The components of the system may be interconnected by any form of medium for digital data communication, such as a communication network. Examples of communication networks include local area networks ("LANs") and wide area networks ("WANs"), such as the Internet.

[0100] A number of embodiments have been described. However, it should be understood that various modifications can be made without departing from the spirit and scope of the subject matter described herein. Other such embodiments are within the scope of the following claims.

Claims

1. A magnetic tracking system for determining an object pose of a tracked object in an environment of the magnetic tracking system, the magnetic tracking system comprising: A magnetic tracking device comprising: a non-magnetic signal detector configured to measure a non-magnetic signal in the environment; and a magnetic signal detector configured to measure a magnetic signal in the environment, the magnetic signal detector being coupled to the non-magnetic signal detector; A computing device configured to perform operations including: receiving measurements of the non-magnetic signals and corresponding measurements of the magnetic signals for a position of the magnetic tracking device in the environment; estimating a non-magnetic pose of the magnetic tracking device in the environment for the location based on the measurement of the non-magnetic signal, estimating a magnetic attitude of the magnetic tracking device in the environment for the location based on the measurements of the magnetic signals, determining a difference between a magnetic attitude estimate and a non-magnetic attitude estimate for the location; determining a magnetic distortion correction value for the location based on the difference; generating a distortion correction model including the magnetic distortion correction value; and A representation of the distortion correction model is output.

2. The magnetic tracking system according to claim 1, wherein: The non-magnetic signal is generated using a marker, and wherein the non-magnetic signal detector comprises a camera.

3. The magnetic tracking system according to claim 2, wherein: The marker includes an appearance that is different from other appearances of other markers in the plurality of markers in the environment.

4. The magnetic tracking system according to claim 2, wherein: The marker includes an adhesive configured to removably attach the marker to another surface in the environment.

5. The magnetic tracking system according to claim 2, wherein: The marker comprises an infrared retroreflector, and wherein the non-magnetic signal detector comprises an infrared source.

6. The magnetic tracking system of claim 1, wherein: The non-magnetic signal includes one of an ArUco pattern, a ChArUco pattern, a light source, an ultrasonic source, a radio signal source, or an outer shape of a transmitter assembly.

7. The magnetic tracking system of claim 1, wherein: The magnetic signal is generated by a transmitter assembly, the transmitter assembly comprising: a memory configured to store calibration data associated with the transmitter assembly; a processing device configured to control transmission of the magnetic signal from the transmitter assembly; a communication interface for sending data to and receiving data from the computing device; and A power supply is configured to provide electrical power to the memory, the processing device, and the communication interface.

8. The magnetic tracking system of claim 1, wherein: The operations also include: Track the tracked object by performing actions including the following: receiving a measurement of the magnetic signal at a particular location in the environment; retrieving the distortion correction model from a memory in communication with the computing device; determining a specific pose estimate for the tracked object; correcting the particular pose estimate based on a value of the distortion correction model corresponding to the particular position; and Output a representation of the corrected particular pose estimate.

9. The magnetic tracking system according to claim 8, wherein: The tracked object includes one of a catheter, an endoscope, or a surgical instrument.

10. The magnetic tracking system of claim 1, wherein: The operations also include: receiving a plurality of measurements of the magnetic signal corresponding to a planar surface of the environment, the planar surface associated with a stent positioned at a known location in the environment; for each of the plurality of measurements of the magnetic signal, retrieving a predetermined correction value associated with a position of the measurement in the environment; determining a distortion correction value for each measurement; and The distortion correction value for each measurement is included in the distortion correction model.

11. An apparatus for determining an object pose of a tracked object in an environment of a magnetic tracking system, the apparatus comprising: a unit for measuring non-magnetic signals in said environment by means of a non-magnetic signal detector; means for measuring magnetic signals in the environment by means of a magnetic signal detector, wherein the magnetic signal detector is coupled to the non-magnetic signal detector; means for receiving measurements of the non-magnetic signal and corresponding measurements of the magnetic signal for a position of a magnetic tracking device in the environment; means for estimating a non-magnetic pose of the magnetic tracking device in the environment for the location based on the measurement of the non-magnetic signal; means for estimating a magnetic pose of the magnetic tracking device in the environment for the location based on the measurements of the magnetic signals; means for determining a difference between a magnetic attitude estimate and a non-magnetic attitude estimate for the location; means for determining a magnetic distortion correction value for the location based on the difference; means for generating a distortion correction model including said magnetic distortion correction value; and Means for outputting a representation of the distortion correction model.

12. The device according to claim 11, wherein The non-magnetic signal is generated using a visual marker, and wherein the non-magnetic signal detector comprises a camera.

13. The device according to claim 12, wherein: The marker includes an outer shape of a transmitter assembly that is different from other outer shapes of other markers in the plurality of markers in the environment.

14. The device according to claim 12, wherein: The marker includes an adhesive configured to removably attach the marker to another surface in the environment.

15. The device according to claim 12, wherein: The marker comprises an infrared retroreflector, and wherein the non-magnetic signal detector comprises an infrared source.

16. The device according to claim 11, wherein The non-magnetic signal includes one of an ArUco pattern, a ChArUco pattern, a light source, an ultrasonic source, a radio signal source, or an outer shape of a transmitter assembly.

17. The device according to claim 11, wherein: The magnetic signal is generated by the transmitter assembly by: storing calibration data associated with the transmitter assembly; and Transmission of the magnetic signal from the transmitter assembly is controlled based on the calibration data.

18. The apparatus according to claim 11, further comprising: A unit used to track a tracked object by performing operations including: receiving a measurement of the magnetic signal at a particular location in the environment; retrieving the distortion correction model from a memory device; determining a specific pose estimate for the tracked object; correcting the particular pose estimate based on a value of the distortion correction model corresponding to the particular position; and Output a representation of the corrected particular pose estimate.

19. The device according to claim 18, wherein: The tracked object includes one of a catheter, an endoscope, or a surgical instrument.

20. The apparatus according to claim 11, further comprising: means for receiving a plurality of measurements of the magnetic signal corresponding to a planar surface of the environment, the planar surface being associated with a support positioned at a known location in the environment; means for retrieving, for each of said plurality of measurements of said magnetic signal, a predetermined correction value associated with a position of the measurement in said environment; means for determining a distortion correction value for each measurement; as well as Means for including the distortion correction value for each measurement in the distortion correction model.

Images