Robotically controllable electromagnetic field generators

Abstract

To provide systems with robotically controllable electromagnetic field generators, and to provide applications thereof.SOLUTION: One application of a robotically controllable electromagnetic field generator uses a robotic arm to move the electromagnetic field generator coupled thereto to facilitate electromagnetic field generator setup and instrument tracking. Another application of such a robotically controllable electromagnetic field generator uses the robotic arm to move the electromagnetic field generator coupled thereto to facilitate alignment of insertable instruments with a target detected within a volume of the electromagnetic field generator. Another application of such a robotically controllable electromagnetic field generator uses the robotic arm to move the electromagnetic field generator coupled thereto to detect distortion occurring within a medical system. Another application of such a robotically controllable electromagnetic field generator uses the robotic arm to facilitate multi-modal sensor fusion.SELECTED DRAWING: Figure 29

Description

[Technical field]

[0001] The systems and methods disclosed herein are directed to electromagnetic field generators for robotic medical systems, and more particularly to robotically positionable and / or controllable electromagnetic field generators for detecting distortions in medical systems and related devices, systems, and methods. [Background technology]

[0002] Medical procedures, such as laparoscopy or endoscopy, can involve accessing and visualizing an interior region of a patient. In a laparoscopic procedure, a medical instrument can be inserted into the interior region through a laparoscopic access port. In an endoscopic procedure, a thin, flexible, tubular medical instrument can be inserted into the interior region through a natural orifice of the patient. The medical instrument can include an end effector configured to perform a function during the procedure.

[0003] In certain procedures, a robotic-enabled medical system can be used to control the insertion and / or manipulation of medical instruments and end effectors. A robotic-enabled medical system can include a robotic arm or other instrument positioning device having a manipulator assembly that is used to control the positioning of the instrument during the procedure.

[0004] The robot-enabled medical system may be configured to determine the position of the medical instrument based on the output of one or more position sensors that may be positioned on the medical instrument. Summary of the Invention [Means for solving the problem]

[0005] A robotic medical system may include an electromagnetic (EM) field generator configured to be coupled (or otherwise integrated) with a robotic arm of the system. Such an EM field generator may be considered robotically controllable or positionable because it may be controlled or repositioned using the robotic arm. The EM field generator may generate a magnetic field that may determine the position of one or more EM sensors. Because the EM field generator is coupled to the robotic arm, the kinematics of the robotic arm may be used to establish alignment between the EM coordinate frame of the EM field generator and the robot coordinate frame or global coordinate frame of the system.

[0006] This configuration can reduce or eliminate the need for more complicated alignment steps that may require user input to establish a relationship between the EM coordinate frame and the robot or global coordinate frame. This configuration can also improve the accuracy with which the EM sensor position is determined and facilitate sensor fusion with additional modalities, for example, by improving the settings of the electromagnetic field generators corresponding to the robot arm, tracking medical instruments, and detecting distortions that may affect the system's sensors.

[0007] These and other features and advantages of the robotically controllable or positionable EM field generator are described in more detail below. The systems, methods and devices of the present disclosure each have multiple innovative aspects, no one of which is solely responsible for the desirable attributes disclosed herein. [Brief description of the drawings]

[0008] The disclosed aspects are hereinafter described in conjunction with the accompanying drawings, in which like designations refer to like elements, and in which: [Figure 1] FIG. 1 illustrates an embodiment of a cart-based robotic system positioned for diagnostic and / or therapeutic bronchoscopy, according to an exemplary embodiment. [Diagram 2]2 illustrates further aspects of the robotic system of FIG. 1 in accordance with an exemplary embodiment. [Diagram 3] FIG. 2 illustrates an embodiment of the robotic system of FIG. 1 positioned for ureteroscopy, according to an exemplary embodiment. [Figure 4] FIG. 2 illustrates an embodiment of the robotic system of FIG. 1 positioned for a vascular procedure, according to an exemplary embodiment. [Diagram 5] FIG. 1 illustrates an embodiment of a table-based robotic system positioned for a bronchoscopic procedure, in accordance with an exemplary embodiment. [Figure 6] FIG. 6 is an alternative view of the robotic system of FIG. 5 in accordance with an exemplary embodiment. [Figure 7] FIG. 1 illustrates an exemplary system configured to accommodate a robotic arm, according to an exemplary embodiment. [Figure 8] FIG. 1 illustrates an embodiment of a table-based robotic system configured for a ureteroscopic procedure, in accordance with an exemplary embodiment. [Figure 9] FIG. 1 illustrates one embodiment of a table-based robotic system configured for a laparoscopic procedure, in accordance with an exemplary embodiment. [Figure 10] FIG. 10 illustrates an embodiment of the table-based robotic system of FIGS. 5-9 with pitch or tilt adjustment, in accordance with an exemplary embodiment. [Figure 11] FIG. 11 is a detailed diagram of the interface between the table and column of the table-based robotic system of FIGS. 5-10 in accordance with an exemplary embodiment. [Figure 12] FIG. 1 illustrates an alternative embodiment of a table-based robotic system in accordance with an illustrative embodiment. [Figure 13] FIG. 13 is an end view of the table-based robotic system of FIG. 12 in accordance with an exemplary embodiment. [Figure 14] FIG. 1 illustrates an end view of a table-based robotic system with an attached robotic arm in accordance with an illustrative embodiment. [Figure 15]FIG. 1 illustrates an exemplary instrument driver in accordance with an exemplary embodiment. [Figure 16] 1 illustrates an exemplary medical instrument having paired instrument drivers, according to an exemplary embodiment. [Figure 17] 13A-13C show alternative designs for the instrument driver and instrument, in which the axis of the drive unit is parallel to the axis of the elongated shaft of the instrument, in accordance with an exemplary embodiment. [Figure 18] FIG. 1 illustrates an instrument having an instrument-based insertion architecture, according to an exemplary embodiment. [Figure 19] FIG. 2 illustrates an example controller according to an example embodiment. [Figure 20] FIG. 19 is a block diagram illustrating a localization system that estimates the location of one or more elements of the robotic system of FIGS. 1-10, such as the location of the instrument of FIGS. 16-18, in accordance with an exemplary embodiment. [Figure 21] 1 illustrates an exemplary EM field generator generating a magnetic field having a working volume, along with an exemplary EM sensor positioned within the working volume of the EM field generator, in accordance with an exemplary embodiment. [Figure 22A] FIG. 22 is a perspective view illustrating an embodiment of a robotic medical system including the EM field generator of FIG. 21 in accordance with an exemplary embodiment. [Figure 22B] FIG. 22B illustrates an exemplary alignment step that may be used to align a robot coordinate frame associated with the robotic medical system of FIG. 22A with an EM coordinate system associated with an EM field generator, according to an exemplary embodiment. [Figure 22C] FIG. 22B illustrates an exemplary alignment step that may be used to align a robot coordinate frame associated with the robotic medical system of FIG. 22A with an EM coordinate system associated with an EM field generator, according to an exemplary embodiment. [Diagram 23] FIG. 1 is a perspective view illustrating an embodiment of a robotic medical system including an embodiment of a robotically controllable electromagnetic field generator coupled to a robotic arm of the system, according to an exemplary embodiment. [Figure 24]FIG. 1 is a perspective view of an embodiment of a robotically controllable electromagnetic field generator configured to couple to a robotic arm in accordance with an illustrative embodiment. [Diagram 25] FIG. 25 illustrates that the robot-controllable electromagnetic field generator of FIG. 24 can be configured to couple to an instrument drive mechanism of a robotic arm, according to an exemplary embodiment. [Figure 26] 1 is a flowchart illustrating one embodiment of a method for performing a robotic medical procedure using a robotically controllable electromagnetic field generator configured to couple to a robotic arm, in accordance with an illustrative embodiment. [Figure 27A] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator configured for automatic instrument tracking and mapping of anatomical features, according to an exemplary embodiment. [Figure 27B] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator configured for automatic instrument tracking and mapping of anatomical features, according to an exemplary embodiment. [Figure 28A] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator configured to facilitate setup and placement of the electromagnetic field generator and to extend its working volume, according to an exemplary embodiment. [Figure 28B] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator configured to facilitate setup and placement of the electromagnetic field generator and to extend its working volume, according to an exemplary embodiment. [Figure 29] FIG. 1 is a block diagram representing one embodiment of a robotic medical system including a robotically controllable electromagnetic field generator coupled to a robotic arm of the system, in accordance with an illustrative embodiment. [Figure 30A] 1 is a perspective view illustrating an example of using a robotic arm to move an EM field generator so that an EM sensor is positioned at a predetermined location within the EM field in accordance with an illustrative embodiment. FIG. [Figure 30B]1 is a perspective view illustrating an example of using a robotic arm to move an EM field generator so that an EM sensor is positioned at a predetermined location within the EM field in accordance with an illustrative embodiment. FIG. [Figure 31A] 1 is a perspective view illustrating an example of using a robotic arm to move an EM field generator so that an EM sensor is positioned within a predetermined region in the EM field in accordance with an illustrative embodiment. FIG. [Figure 31B] 1 is a perspective view illustrating an example of using a robotic arm to move an EM field generator so that an EM sensor is positioned within a predetermined region in the EM field in accordance with an illustrative embodiment. FIG. [Figure 32A] FIG. 13 is a perspective view illustrating an example of using a robotic arm to move an EM field generator along a path that tracks the path of movement of an EM sensor in accordance with an illustrative embodiment. [Figure 32B] FIG. 13 is a perspective view illustrating an example of using a robotic arm to move an EM field generator along a path that tracks the path of movement of an EM sensor in accordance with an illustrative embodiment. [Figure 33A] 1 is a perspective view illustrating an example of using a robotic arm to move an EM field generator such that an EM sensor moving along a path remains positioned within a predetermined region of the EM field in accordance with an illustrative embodiment. [Figure 33B] 1 is a perspective view illustrating an example of using a robotic arm to move an EM field generator such that an EM sensor moving along a path remains positioned within a predetermined region of the EM field in accordance with an illustrative embodiment. [Figure 33C] 1 is a perspective view illustrating an example of using a robotic arm to move an EM field generator such that an EM sensor moving along a path remains positioned within a predetermined region of the EM field in accordance with an illustrative embodiment. [Figure 33D] 1 is a perspective view illustrating an example of using a robotic arm to move an EM field generator such that an EM sensor moving along a path remains positioned within a predetermined region of the EM field in accordance with an illustrative embodiment. [Diagram 34]1 is a flowchart providing an example method for moving an EM field generator coupled to a robotic arm based on a determined position of an EM sensor within an EM field, in accordance with an example embodiment. [Figure 35A] 1 is a perspective view illustrating an example of using a robotic arm to move an EM field generator to an electromagnetic field generator position based on determined positions of multiple EM sensors within an EM field in accordance with an illustrative embodiment. [Figure 35B] 1 is a perspective view illustrating an example of using a robotic arm to move an EM field generator to an electromagnetic field generator position based on determined positions of multiple EM sensors within an EM field in accordance with an illustrative embodiment. [Figure 36A] FIG. 13 is a perspective view illustrating an example of realigning an electromagnetic field generator position of an EM field generator using a robotic arm based on determined positions of multiple EM sensors within the EM field, where at least one of the multiple EM sensors is moving, according to an illustrative embodiment. [Figure 36B] FIG. 13 is a perspective view illustrating an example of realigning an electromagnetic field generator position of an EM field generator using a robotic arm based on determined positions of multiple EM sensors within the EM field, where at least one of the multiple EM sensors is moving, according to an illustrative embodiment. [Figure 36C] FIG. 13 is a perspective view illustrating an example of realigning an electromagnetic field generator position of an EM field generator using a robotic arm based on determined positions of multiple EM sensors within the EM field, where at least one of the multiple EM sensors is moving, according to an illustrative embodiment. [Figure 37] 11 is a flowchart illustrating an exemplary method for determining an electromagnetic field generator position of an EM field generator mounted on a robotic arm based on determined positions of a plurality of EM sensors within an EM field in accordance with an exemplary embodiment. [Figure 38A] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator that can be moved with a robotic arm to expand the working volume of the electromagnetic field generator, according to an illustrative embodiment. [Figure 38B]FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator that can be moved with a robotic arm to expand the working volume of the electromagnetic field generator, according to an illustrative embodiment. [Figure 38C] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator that can be moved with a robotic arm to expand the working volume of the electromagnetic field generator, according to an illustrative embodiment. [Figure 39A] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator that can be moved with a robotic arm to detect the position of an EM sensor and expand the working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 39B] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator that can be moved with a robotic arm to detect the position of an EM sensor and expand the working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 39C] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator that can be moved with a robotic arm to detect the position of an EM sensor and expand the working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 39D] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator that can be moved with a robotic arm to detect the position of an EM sensor and expand the working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 40A] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator that can be moved with a robotic arm to track the position of an EM sensor within the extended working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 40B] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator that can be moved with a robotic arm to track the position of an EM sensor within the extended working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 40C] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator that can be moved with a robotic arm to track the position of an EM sensor within the extended working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 40D] FIG. 1 illustrates an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator that can be moved with a robotic arm to track the position of an EM sensor within the extended working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 41A] 1 is a flowchart illustrating an exemplary method for expanding a working volume of a robot-controllable electromagnetic field generator, in accordance with an illustrative embodiment. [Figure 41B] 11 is a flowchart illustrating another exemplary method for expanding a working volume of a robot-controllable electromagnetic field generator, in accordance with an exemplary embodiment. [Figure 42A] FIG. 1 illustrates an embodiment of a robotic medical system having a robotically controllable electromagnetic field generator configured to facilitate alignment of a percutaneously insertable instrument with an EM target, according to an exemplary embodiment. [Figure 42B] FIG. 1 illustrates an embodiment of a robotic medical system having a robotically controllable electromagnetic field generator configured to facilitate alignment of a percutaneously insertable instrument with an EM target, according to an exemplary embodiment. [Figure 42C] FIG. 1 illustrates an embodiment of a robotic medical system having a robotically controllable electromagnetic field generator configured to facilitate alignment of a percutaneously insertable instrument with an EM target, according to an exemplary embodiment. [Figure 43A] FIG. 1 illustrates an embodiment of a robotic medical system having a robotically controllable electromagnetic field generator configured to facilitate alignment of an instrument guide mounted on the electromagnetic field generator with an EM target, according to an exemplary embodiment. [Figure 43B]FIG. 1 illustrates an embodiment of a robotic medical system having a robotically controllable electromagnetic field generator configured to facilitate alignment of an instrument guide mounted on the electromagnetic field generator with an EM target, according to an exemplary embodiment. [Figure 44A] FIG. 1 illustrates an embodiment of a robotic medical system having a robotically controllable electromagnetic field generator configured to facilitate alignment of an instrument guide with an EM target, according to an exemplary embodiment. [Figure 44B] FIG. 1 illustrates an embodiment of a robotic medical system having a robotically controllable electromagnetic field generator configured to facilitate alignment of an instrument guide with an EM target, according to an exemplary embodiment. [Figure 45A] 1 is a flowchart illustrating a method for aligning a percutaneously insertable instrument with an EM target using a robotically controllable electromagnetic field generator, in accordance with an illustrative embodiment. [Figure 45B] 1 is a flow chart illustrating a method for aligning an instrument guide for a percutaneously insertable instrument with an EM target using a robotically controllable electromagnetic field generator, in accordance with an illustrative embodiment. [Figure 46A] FIG. 1 illustrates an embodiment of a robotic medical system that moves an EM field generator relative to a fixed EM position sensor to detect EM distortion, according to an exemplary embodiment. [Figure 46B] FIG. 46B illustrates a comparison of the robot trajectory and the EM sensor trajectory during the movement of the EM field generator of FIG. 46A in accordance with an illustrative embodiment. [Figure 46C] FIG. 1 illustrates an embodiment of a robotic medical system that moves an EM field generator relative to a fixed EM position sensor to detect EM distortions that can account for changes in position and orientation, according to an exemplary embodiment. [Figure 47] 4 is a flowchart illustrating an example method for EM distortion detection, in accordance with an example embodiment. [Figure 48] 5 is a flowchart illustrating another example method for EM distortion detection, in accordance with an example embodiment. [Figure 49A] FIG. 1 illustrates a robotic medical system having an ultrasound probe attached thereto during a procedure for calibrating the imaging plane of the ultrasound probe, according to an exemplary embodiment. [Figure 49B] 1 illustrates a robotic medical system including an EM field generator and an ultrasound probe, the imaging plane of which is calibrated to a robot coordinate frame and an EM coordinate frame such that the positions of the robotic tool and the EM sensor can be overlaid on the imaging plane, according to an exemplary embodiment. [Figure 50] 1 is a flowchart illustrating an exemplary method for calibrating an imaging plane of an ultrasound probe for use with a robotic medical system, in accordance with an exemplary embodiment. [Figure 51A] FIG. 1 illustrates an exemplary heat and point map generated using a depth sensor attached to a robotic arm of a robotic medical system, in accordance with an exemplary embodiment. [Figure 51B] FIG. 1 illustrates an exemplary heat and point map generated using a depth sensor attached to a robotic arm of a robotic medical system, in accordance with an exemplary embodiment. [Figure 51C] FIG. 2 illustrates an embodiment of an exemplary depth sensor, in accordance with an exemplary embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] 1. Overview. Aspects of the present disclosure may be integrated into a robot-enabled medical system capable of performing a variety of medical procedures, including both minimally invasive procedures such as laparoscopy, and non-invasive procedures such as endoscopy. Among endoscopic procedures, the system may perform bronchoscopy, ureteroscopy, gastroscopy, etc.

[0010] In addition to performing a wide range of procedures, the system can provide additional benefits such as enhanced imaging and guidance to assist the physician. Additionally, the system can provide the physician with the ability to perform procedures from an ergonomic position without requiring awkward arm movements and postures. Still further, the system can provide the physician with the ability to perform procedures with improved ease of use, such that one or more of the instruments of the system can be controlled by a single user.

[0011] Various embodiments are described below in conjunction with the drawings for purposes of explanation. It should be understood that many other implementations of the disclosed concepts are possible, and that various advantages may be achieved in the disclosed implementations. Headings are included herein for reference and to aid in locating the various sections. These headings are not intended to limit the scope of the concepts described with respect thereto. Such concepts may have applicability throughout this specification.

[0012] A. Robotic System - Cart. Robot-enabled medical systems can be configured in a variety of ways depending on the particular procedure. FIG. 1 illustrates one embodiment of a cart-based robot-enabled system 10 deployed for diagnostic and / or therapeutic bronchoscopy. During bronchoscopy, the system 10 can include a cart 11 having one or more robotic arms 12 for delivering medical instruments, such as a steerable endoscope 13, which can be a procedure-specific bronchoscope for bronchoscopy, to a natural orifice access point (i.e., the mouth of a patient positioned on a table in this example) for delivering diagnostic and / or therapeutic tools. As shown, the cart 11 can be positioned adjacent to the patient's upper torso to provide access to the access point. Similarly, the robotic arms 12 can be actuated to position the bronchoscope relative to the access point. The configuration of FIG. 1 can also be utilized when performing gastro-intestinal (GI) procedures with a gastroscope, an endoscope specialized for GI procedures. FIG. 2 illustrates an exemplary embodiment of the cart in more detail.

[0013] With continued reference to FIG. 1, once the cart 11 is properly positioned, the robotic arm 12 can insert the steerable endoscope 13 into the patient robotically, manually, or a combination thereof. As shown, the steerable endoscope 13 may include at least two nested parts, such as an inner leader portion and an outer sheath portion, each portion coupled to a separate instrument driver of a set of instrument drivers 28, each instrument driver coupled to the distal end of an individual robotic arm. This linear configuration of the instrument driver 28, which facilitates coaxial alignment of the leader portion with the sheath portion, forms a "virtual rail" 29 that can be repositioned in space by manipulating one or more robotic arms 12 to different angles and / or positions. The virtual rails described herein are illustrated in the figures using dashed lines, which therefore do not illustrate the physical structure of the system. Translation of the instrument driver 28 along the virtual rail 29 nests the inner leader portion relative to the outer sheath portion or advances or retracts the endoscope 13 from the patient. The angle of the virtual rail 29 may be adjusted, translated, and pivoted based on clinical application or physician preference. For example, in bronchoscopy, the angle and position of the virtual rail 29 as shown represents a compromise that provides the physician access to the endoscope 13 while minimizing friction from bending the endoscope 13 into the patient's mouth.

[0014] The endoscope 13 may be directed downstream of the patient's trachea and lungs after insertion using precise commands from the robotic system until the target destination or surgical site is reached. To facilitate navigation through the patient's pulmonary network and / or to reach the desired target, the endoscope 13 may be steered to telescope the inner leader portion out from the outer sheath portion for enhanced articulation and a larger bend radius. The use of a separate instrument driver 28 also allows the leader portion and sheath portion to be driven independently of one another.

[0015] For example, the endoscope 13 may be oriented to deliver a biopsy needle to a target, such as a lesion or nodule in the patient's lung. The needle may be deployed down a working channel that spans the length of the endoscope to obtain a tissue sample that is analyzed by a pathologist. Depending on the results of the pathology, additional tools may be deployed down the working channel of the endoscope for additional biopsies. After identifying the nodule as malignant, the endoscope 13 may endoscopically deliver a tool to ablate the potential cancerous tissue. In some cases, the diagnostic and therapeutic procedures may be delivered during separate procedures. In these situations, the endoscope 13 may also be used to deliver fiducials to "mark" the location of the target nodule. In other examples, the diagnostic and therapeutic procedures may be delivered during the same procedure.

[0016] The system 10 may also include a movable tower 30 that may be connected to the cart 11 via a support cable to provide support for control, electronics, fluidics, optics, sensors, and / or power to the cart 11. Locating such functionality within the tower 30 allows for a smaller form factor cart 11 that can be more easily adjusted and / or repositioned by the operating physician and his / her staff. Additionally, the division of functionality between the cart / table and the support tower 30 reduces clutter in the operating room and facilitates improved clinical workflow. The cart 11 may be positioned in close proximity to the patient, while the tower 30 may be housed in a remote location so as not to get in the way during the procedure.

[0017] To support the robotic system described above, the tower 30 may include computer-based control system components that store computer program instructions in a non-transitory computer-readable storage medium, such as, for example, a persistent magnetic storage drive, a solid-state drive, or the like. Execution of these instructions, whether execution occurs in the tower 30 or in the cart 11, may control the entire system or subsystems thereof. For example, when executed by a processor in the computer system, the instructions may cause components of the robotic system to actuate associated carriages and arm mounts, operate a robotic arm, and control a medical instrument. For example, in response to receiving control signals, motors in the joints of the robotic arm may position the arm in a particular pose.

[0018] The tower 30 may also include pumps, flow meters, valve controls, and / or fluid access to provide controlled irrigation and aspiration functions to a system that may be deployed through the endoscope 13. These components may also be controlled using the computer system of the tower 30. In some embodiments, irrigation and aspiration capabilities may be delivered directly to the endoscope 13 via separate cables.

[0019] The tower 30 may include voltage and surge protection designed to provide filtered and protected power to the cart 11, thereby avoiding the need to mount power transformers and other auxiliary power components within the cart 11, resulting in the cart 11 being smaller and more mobile.

[0020] The tower 30 may also include support equipment for sensors located throughout the robotic system 10. For example, the tower 30 may include optoelectronic equipment for detecting, receiving, and processing data received from optical sensors or cameras anywhere in the robotic system 10. In combination with a control system, such optoelectronic equipment may be used to generate real-time images for display in any number of consoles located throughout the system, including in the tower 30. Similarly, the tower 30 may also include electronic subsystems for receiving signals from and processing received electromagnetic (EM) sensors located therein. The tower 30 may also be used to house and position EM field generators for detection by EM sensors in or on the medical instrument.

[0021] The tower 30 may also include a console 31 in addition to other consoles available to the rest of the system, for example a console mounted on top of a cart. The console 31 may include a user interface for the operator physician and a display screen, such as a touch screen. The consoles in the system 10 are generally designed to provide both pre-operative and real-time information of the procedure, such as robotic control and navigation and localization information for the endoscope 13. If the console 31 is not the only console available to the physician, the console 31 may be used by a second operator, such as a nurse, to provide procedure-specific data, such as navigation and localization information, as well as monitor the patient's health or vitals and the operation of the system 10. In other embodiments, the console 31 is housed in a body separate from the tower 30.

[0022] The tower 30 may be coupled to the cart 11 and the endoscope 13 via one or more cables or connections (not shown). In some embodiments, support functions from the tower 30 may be provided to the cart 11 through a single cable, simplifying and decluttering the operating room. In other embodiments, certain functions may be coupled with separate wiring and connections. For example, power may be provided to the cart 11 through a single power cable, while support for control, optics, fluidics, and / or navigation may be provided through separate cables.

[0023] FIG. 2 provides a detailed view of one embodiment of a cart 11 from the cart-based robot-enabled system shown in FIG. 1. The cart 11 generally includes an elongated support structure 14 (often referred to as a "column"), a cart base 15, and a console 16 at the top of the column 14. The column 14 may include one or more carriages, such as a carriage 17 (alternatively an "arm support") for supporting the deployment of one or more robotic arms 12 (three are shown in FIG. 2). The carriage 17 may include an individually configurable arm mount that rotates along a vertical axis to adjust the base of the robotic arm 12 for better positioning relative to the patient. The carriage 17 also includes a carriage interface 19 that allows the carriage 17 to translate vertically along the column 14.

[0024] The carriage interface 19 is connected to the column 14 through slots, such as slots 20 positioned on either side of the column 14 to guide the vertical translation of the carriage 17. The slots 20 include vertical translation interfaces for positioning and holding the carriage 17 at various vertical heights relative to the cart base 15. The vertical translation of the carriage 17 allows the cart 11 to adjust the reach of the robotic arm 12 to meet various table heights, patient sizes, and physician preferences. Similarly, an individually configurable arm mount on the carriage 17 allows the robotic arm base 21 of the robotic arm 12 to be angled in various configurations.

[0025] In some embodiments, a slot cover may be added to the slot 20 that is flush and parallel with the slot surface to prevent dirt and fluids from entering the interior chamber of the column 14 and the vertical translation interface as the carriage 17 translates vertically. The slot cover may be deployed through a pair of spring spools that are positioned near the vertical top and bottom of the slot 20. The cover is coiled within the spools until it is deployed to extend and retract from the coiled state as the carriage 17 translates vertically up and down. The spring loading of the spools provides the force to retract the cover into the spool as the carriage 17 translates towards the spool, while also maintaining a seal as the carriage 17 translates away from the spool. The cover may be connected to the carriage 17 using, for example, a bracket within the carriage interface 19 to ensure that the cover extends and retracts properly as the carriage 17 translates.

[0026] Column 14 may include mechanisms therein, such as gears and motors, designed to use a vertically aligned leadscrew to mechanically translate carriage 17 in response to control signals generated in response to user input, such as input from console 16.

[0027] The robotic arms 12 may generally include a robotic arm base 21 and an end effector 22 separated by a series of links 23 connected by a series of joints 24, each joint including an independent actuator, each actuator including an independently controllable motor. Each independently controllable joint represents an independent degree of freedom available to the robotic arms 12. Each of the robotic arms 12 may have seven joints, thus providing seven degrees of freedom. A large number of joints results in a large number of degrees of freedom, allowing for "redundant" degrees of freedom. Having redundant degrees of freedom allows the robotic arms 12 to position their respective end effectors 22 at specific positions, orientations, and trajectories in space using different link positions and joint angles. This allows the system to position and orient the medical instrument from a desired point in space, while allowing the physician to move the arm joints to a clinically advantageous position away from the patient to create better access while avoiding arm collisions.

[0028] The cart base 15 balances the weight of the column 14, carriage 17, and robotic arm 12 on the floor. Thus, the cart base 15 houses the heavier parts such as electronics, motors, power supplies, and components that allow either movement and / or immobilization of the cart 11. For example, the cart base 15 includes casters 25 in the form of rollable wheels that allow the cart 11 to be easily moved around the room before a procedure. After reaching the proper position, the casters 25 may be immobilized using wheel locks to hold the cart 11 in place during a procedure.

[0029] The console 16 positioned at the vertical end of the column 14 allows for both a user interface and a display screen (or dual-purpose device, such as, for example, a touch screen 26) to receive user input to provide both pre-operative and intra-operative data to the physician user. Potential pre-operative data on the touch screen 26 may include pre-operative planning, navigation and mapping data derived from a pre-operative computed tomography (CT scan), and / or notes from a pre-operative patient interview. Intra-operative data on the display may include optical information provided by tools, sensor and coordinate information from sensors, and vital patient statistics such as respiration, heart rate, and / or pulse. The console 16 may be positioned and angled to allow the physician to access the console 16 from the opposite side of the column 14 from the carriage 17. From this position, the physician can view the console 16, the robotic arm 12, and the patient while operating the console 16 from behind the cart 11. As shown, the console 16 also includes a handle 27 to aid in maneuvering and stabilizing the cart 11.

[0030] FIG. 3 illustrates one embodiment of the robot-enabled system 10 positioned for ureteroscopy. In a ureteroscopy procedure, the cart 11 may be positioned to deliver a ureteroscope 32, a procedure-specific endoscope designed to traverse the patient's urethra and ureters, to the patient's lower abdominal region. In a ureteroscopy, it may be desirable for the ureteroscope 32 to be aligned directly with the patient's urethra to reduce friction and forces on sensitive anatomical structures in the area. As shown, the cart 11 may be aligned to the legs of a table to allow the robotic arm 12 to position the ureteroscope 32 to directly access the patient's urethra in a straight line. From the legs of the table, the robotic arm 12 may insert the ureteroscope 32 along a virtual rail 33 directly through the urethra and into the patient's lower abdomen.

[0031] After insertion into the urethra using control techniques similar to those in bronchoscopy, the ureteroscope 32 may be navigated to the bladder, ureters, and / or kidneys for diagnostic and / or therapeutic applications. For example, the ureteroscope 32 may be aimed at the ureters and kidneys to break up formed kidney stones using a laser or ultrasonic lithotripsy device deployed down the working channel of the ureteroscope 32. After lithotripsy is completed, the resulting stone fragments may be removed using a basket deployed down the ureteroscope 32.

[0032] FIG. 4 shows an embodiment of a robot-enabled system 10 similarly positioned for a vascular procedure. In a vascular procedure, the system 10 may be configured such that the cart 11 can deliver a medical instrument 34, such as a steerable catheter, to an access point in the femoral artery in the patient's leg. The femoral artery presents both a larger diameter for navigation and a less circuitous and tortuous path to the patient's heart, simplifying navigation. As in a ureteroscopy procedure, the cart 11 may be positioned toward the patient's leg and lower abdomen, allowing the robotic arm 12 to provide a virtual rail 35 with direct linear access to the femoral artery access point in the patient's thigh / hip region. After insertion into the artery, the medical instrument 34 may be oriented and inserted by translating the instrument driver 28. Alternatively, the cart may be positioned around the patient's upper abdomen to reach alternative vascular access points, such as the carotid and brachial arteries near the shoulder and wrist.

[0033] B. Robot system-table. An embodiment of a robot-enabled medical system may also incorporate a patient table. The incorporation of a table reduces the amount of capital equipment in the operating room by eliminating carts, thereby allowing better access to the patient. FIG. 5 shows one embodiment of such a robot-enabled system deployed for a bronchoscopy procedure. The system 36 includes a support structure or column 37 for supporting a platform 38 (shown as a "table" or "bed") above the floor. As with cart-based systems, the end effector of the robotic arm 39 of the system 36 includes an instrument driver 42 designed to manipulate an elongated medical instrument, such as the bronchoscope 40 of FIG. 5, through or along a virtual rail 41 formed from a linear array of instrument drivers 42. In practice, a C-arm for providing fluoroscopic imaging may be positioned above the upper abdominal region of the patient by placing emitters and detectors around the table 38.

[0034] FIG. 6 provides an alternative view of the system 36 without the patient and medical instruments for illustrative purposes. As shown, the column 37 may include one or more carriages 43, illustrated as a ring shape in the system 36, which may be the base for one or more robotic arms 39. The carriages 43 may translate along a vertical column interface 44 that spans the length of the column 37 to provide different vantage points where the robotic arms 39 may be positioned to reach the patient. The carriages 43 may rotate around the column 37 using mechanical motors positioned within the column 37 to allow the robotic arms 39 to have access to multiple sides of the table 38, such as both sides of the patient. In embodiments with multiple carriages, the carriages may be individually positioned on the column and may translate and / or rotate independently of the other carriages. The carriages 43 do not have to surround the column 37 or even be circular, but the ring shape as illustrated facilitates the rotation of the carriages 43 around the column 37 while maintaining structural balance. Rotation and translation of carriage 43 allows system 36 to align medical instruments such as endoscopes and laparoscopes to different access points on the patient. In other embodiments (not shown), system 36 may include a patient table or bed with adjustable arm supports in the form of parallel extending bars or rails. One or more robotic arms 39 may be mounted on adjustable arm supports that can be adjusted vertically (e.g., via a shoulder with an elbow joint). By providing vertical adjustment, robotic arm 39 can advantageously be stored compactly under a patient table or bed and then raised during a procedure.

[0035] The robotic arm 39 may be mounted to the carriage 43 via a set of arm mounts 45 that include a series of joints that may be individually rotated and / or telescopically extended to provide additional configurability to the robotic arm 39. Additionally, the arm mounts 45 may be positioned on the carriage 43 such that, when the carriage 43 is appropriately rotated, the arm mounts 45 may be positioned either on the same side of the table 38 (as shown in FIG. 6), on opposite sides of the table 38 (as shown in FIG. 9), or on adjacent sides of the table 38 (not shown).

[0036] The column 37 structurally provides support for the table 38 and a path for the vertical translation of the carriage 43. Internally, the column 37 may include a lead screw for guiding the vertical translation of the carriage, and a motor for mechanizing the translation of the carriage 43 based on the lead screw. The column 37 may also transmit power and control signals to the carriage 43 and a robotic arm 39 mounted thereon.

[0037] The table base 46 serves a similar function as the cart base 15 of the cart 11 shown in FIG. 2, housing the heavier components to counterbalance the table / bed 38, column 37, carriage 43, and robotic arm 39. The table base 46 may also incorporate rigid casters to provide stability during a procedure. The casters that deploy from the bottom of the table base 46 extend in opposite directions on either side of the base 46 and may be retracted when the system 36 needs to be moved.

[0038] Continuing to refer to FIG. 6, the system 36 may also include a tower (not shown) that divides the functionality of the system 36 between the table and the tower to reduce the form factor and bulk of the table. As with the previously disclosed embodiments, the tower may provide various support functions to the table, such as processing, computing, and control capabilities, power, fluidics, and / or optical and sensor processing. The tower may also be movable to be positioned away from the patient to improve physician access and reduce clutter in the operating room. Additionally, locating components in the tower allows for more storage space in the table base 46 for potential accommodation of the robotic arm 39. The tower may also include a master controller or console that provides both a user interface for user input, such as a keyboard and / or pendant, and a display screen (or touch screen) for pre-operative and intra-operative information, such as real-time imaging, navigation, and tracking information. In some embodiments, the tower may also include a holder for a gas tank used for insufflation.

[0039] In some embodiments, the table base may house and store the robot arm when not in use. FIG. 7 shows a system 47 for housing the robot arm in one embodiment of a table-based system. In the system 47, the carriage 48 may be translated vertically into the base 49 to house the robot arm 50, arm mount 51, and carriage 48 therein. The base cover 52 may be translated and retracted to open to deploy the carriage 48, arm mount 51, and robot arm 50 about the column 53, and closed to house and protect them when not in use. The base cover 52 may be sealed with a membrane 54 along the edge of its opening to prevent dirt and fluid ingress when closed.

[0040] FIG. 8 illustrates one embodiment of a robot-enabled table-based system configured for a ureteroscopy procedure. For ureteroscopy, the table 38 may include a swivel portion 55 for positioning the patient off-angle from the column 37 and table base 46. The swivel portion 55 may rotate or pivot about a pivot point (e.g., located below the patient's head) to position the bottom of the swivel portion 55 away from the column 37. For example, pivoting the swivel portion 55 allows a C-arm (not shown) to be positioned above the patient's lower abdomen without competing for space with the column (not shown) below the table 38. By rotating the carriage (not shown) about the column 37, the robotic arm 39 may insert the ureteroscope 56 along the virtual rail 57 directly into the patient's groin area to reach the urethra. For ureteroscopy, stirrups 58 may also be secured to the swivel portion 55 of the table 38 to support the position of the patient's legs during the procedure and allow clear access to the patient's groin area.

[0041] In a laparoscopic procedure, minimally invasive instruments may be inserted into a patient's anatomy through small incisions in the patient's abdominal wall. In some embodiments, the minimally invasive instruments include an elongated rigid member, such as a shaft, that is used to access anatomy within the patient. After distension of the patient's abdominal cavity, the instruments may be directed to perform a surgical or medical task, such as grasping, cutting, ablation, suturing, etc. In some embodiments, the instruments may include a scope, such as a laparoscope. FIG. 9 illustrates one embodiment of a robot-enabled table-based system configured for laparoscopic procedures. As illustrated in FIG. 9, the carriage 43 of the system 36 may be rotated and vertically adjusted to position a pair of robotic arms 39 on either side of the table 38 such that the instruments 59 may be positioned using arm mounts 45 to pass through minimal incisions on either side of the patient to reach the patient's abdominal cavity.

[0042] To accommodate laparoscopic procedures, the robotic table system may also tilt the platform to a desired angle. FIG. 10 illustrates an embodiment of a robotic medical system with pitch or tilt adjustment. As shown in FIG. 10, the system 36 can accommodate the tilt of the table 38 to position one portion of the table further away from the floor than the other portion. Additionally, the arm mount 45 may rotate to match the tilt so that the robotic arm 39 maintains the same planar relationship with the table 38. To accommodate steeper angles, the column 37 may also include a telescoping portion 60 that allows the column 37 to extend vertically to prevent the table 38 from contacting the floor or colliding with the table base 46.

[0043] FIG. 11 provides a detailed view of the interface between the table 38 and the column 37. The pitch rotation mechanism 61 may be configured to change the pitch angle of the table 38 relative to the column 37 in multiple degrees of freedom. The pitch rotation mechanism 61 may be enabled by the positioning of orthogonal axes 1, 2 at the column-table interface, each actuated by a separate motor 3, 4 in response to an electrical pitch angle command. Rotation along one screw 5 allows tilt adjustment in one axis 1, and rotation along the other screw 6 allows tilt adjustment along the other axis 2. In some embodiments, a ball joint may be used to change the pitch angle of the table 38 relative to the column 37 in multiple degrees of freedom.

[0044] For example, pitch adjustment is particularly useful when attempting to position the table in the Trendelenburg position, i.e., positioning the patient's lower abdomen higher off the floor than the patient's upper abdomen for lower abdominal surgery, which allows gravity to slide the patient's internal organs toward the patient's upper abdomen, emptying the abdominal cavity and allowing minimally invasive tools to enter and perform lower abdominal surgical or medical procedures, such as laparoscopic prostatectomy.

[0045] 12 and 13 show isometric and end views of an alternative embodiment of a table-based surgical robotic system 100. The surgical robotic system 100 includes one or more adjustable arm supports 105 (see, e.g., FIG. 14 ) that can be configured to support one or more robotic arms relative to the table 101. In the illustrated embodiment, a single adjustable arm support 105 is shown, but additional arm supports 105 can be provided on the opposite side of the table 101. The adjustable arm support 105 can be configured to move relative to the table 101 to adjust and / or change the position of the adjustable arm support 105 and / or any robotic arm mounted thereto relative to the table 101. For example, the adjustable arm support 105 can be adjusted with one or more degrees of freedom relative to the table 101. The adjustable arm support 105 provides the system 100 with great versatility, including the ability to easily accommodate one or more adjustable arm supports 105 and any robotic arms attached thereto under the table 101. The adjustable arm support 105 can be elevated from a stowed position to a position below the top surface of the table 101. In other embodiments, the adjustable arm support 105 can be elevated from a stowed position to a position above the top surface of the table 101.

[0046] The adjustable arm support 105 can provide multiple degrees of freedom, including lift, lateral translation, tilt, etc. In the illustrated embodiment of FIGS. 12 and 13, the arm support 105 is configured with four degrees of freedom, which are indicated by arrows in FIG. 12. The first degree of freedom allows for adjustment of the adjustable arm support 105 in the z-direction ("Z-lift"). For example, the adjustable arm support 105 can include a carriage 109 configured to move up and down along or relative to the column 102 that supports the table 101. The second degree of freedom allows the adjustable arm support 105 to tilt. For example, the adjustable arm support 105 can include a rotational joint, which can allow the adjustable arm support 105 to align with a bed in the Trendelenburg position. The third degree of freedom can allow the adjustable arm support 105 to "pivot up", which can be used to adjust the distance between the side of the table 101 and the adjustable arm support 105. The fourth degree of freedom allows the adjustable arm support 105 to translate along the longitudinal length of the table.

[0047] The surgical robotic system 100 of Figures 12 and 13 can include a table supported by a column 102 mounted to a base 103. The base 103 and the column 102 support the table 101 against a support surface. A bed axis 131 and a support axis 133 are shown in Figure 13.

[0048] The adjustable arm support 105 can be mounted to the column 102. In other embodiments, the arm support 105 can be mounted to the table 101 or the base 103. The adjustable arm support 105 can include a carriage 109, a bar or rail connector 111, and a bar or rail 107. In some embodiments, one or more robot arms mounted to the rail 107 can translate and move relative to each other.

[0049] The carriage 109 may be attached to the column 102 by a first joint 113, which allows the carriage 109 to move relative to the column 102 (e.g., up and down on a first or vertical axis 123). The first joint 113 may provide a first degree of freedom ("Z lift") for the adjustable arm support 105. The adjustable arm support 105 may include a second joint 115 that provides a second degree of freedom (tilt) for the adjustable arm support 105. The adjustable arm support 105 may include a third joint 117 that may provide a third degree of freedom ("upward pivot") for the adjustable arm support 105. An additional joint 119 (shown in FIG. 13) may be provided that mechanically constrains the third joint 117 to maintain the orientation of the rail 107 when the rail connector 111 is rotated about a third axis 127. The adjustable arm support 105 may include a fourth joint 121 that may provide a fourth degree of freedom (translation) to the adjustable arm support 105 along a fourth axis 129 .

[0050] 14 shows an end view of a surgical robotic system 140A having two adjustable arm supports 105A, 105B mounted on either side of a table 101. A first robotic arm 142A is attached to a bar or rail 107A of the first adjustable arm support 105B. The first robotic arm 142A includes a base 144A attached to the rail 107A. The distal end of the first robotic arm 142A includes an instrument drive mechanism 146A that can be coupled to one or more robotic medical instruments or tools. Similarly, a second robotic arm 142B includes a base 144B attached to the rail 107B. The distal end of the second robotic arm 142B includes an instrument drive mechanism 146B. The instrument drive mechanism 146B can be configured to couple to one or more robotic medical instruments or tools.

[0051] In some embodiments, one or more of the robotic arms 142A, 142B include arms with seven or more degrees of freedom. In some embodiments, one or more of the robotic arms 142A, 142B may include eight degrees of freedom, including an insertion axis (one degree of freedom including insertion), wrist (three degrees of freedom including wrist pitch, yaw and roll), elbow (one degree of freedom including elbow pitch), shoulder (two degrees of freedom including shoulder pitch and yaw), and base 144A, 144B (one degree of freedom including translation). In some embodiments, the insertion degree of freedom may be provided by the robotic arms 142A, 142B, while in other embodiments, the instrument itself provides the insertion via an instrument-based insertion architecture.

[0052] C. Instrument drivers and interfaces. The end effector of the robotic arm of the system may include (i) an instrument driver (alternatively called an "instrument drive mechanism" or "instrument device manipulator") that incorporates electromechanical means for actuating the medical instrument, and (ii) a removable or detachable medical instrument, which may lack any electromechanical components such as a motor. This dichotomy may be caused by the need to sterilize the medical instruments used in medical procedures and the inability to adequately sterilize expensive capital equipment due to the complex mechanical assembly of the medical instruments and the sensitivity of the electronics. Thus, medical instruments may be designed to be detached, removed, and replaced from the instrument driver (and thus the system) for individual sterilization or disposal by the physician or physician's staff. In contrast, instrument drivers do not need to be replaced or sterilized and can be draped for protection.

[0053] FIG. 15 illustrates an exemplary instrument driver. The instrument driver 62, positioned at the distal end of the robotic arm, includes one or more drive units 63 arranged with parallel axes to provide a controlled torque to the medical instrument via a drive shaft 64. Each drive unit 63 includes an individual drive shaft 64 for interacting with the instrument, a gear head 65 for converting the rotation of the motor shaft to a desired torque, a motor 66 for generating the drive torque, an encoder 67 for measuring the speed of the motor shaft and providing feedback to the control circuit, and a control circuit 68 for receiving control signals and operating the drive unit. Each drive unit 63 is independently controlled and motorized, and the instrument driver 62 can provide multiple (e.g., four as shown in FIG. 15) independent drive outputs to the medical instrument. In operation, the control circuit 68 receives the control signals, sends motor signals to the motors 66, compares the resulting motor speed measured by the encoders 67 to a desired speed, and modulates the motor signals to generate the desired torque.

[0054] For procedures requiring a sterile environment, the robotic system may incorporate a drive interface, such as a sterile adapter connected to a sterile drape, located between the instrument driver and the medical instrument. The primary purpose of the sterile adapter is to transmit angular motion from the drive shaft of the instrument driver to the drive input of the instrument while maintaining physical separation, and therefore sterility, between the drive shaft and the drive input. Thus, an exemplary sterile adapter may include a set of rotational inputs and outputs intended to mate with the drive shaft of the instrument driver and a drive input to the instrument. The sterile drape, which is connected to the sterile adapter, is composed of a thin flexible material, such as a transparent or translucent plastic, and is designed to cover the instrument driver, the robotic arm, and capital equipment, such as a cart (in a cart-based system) or a table (in a table-based system). The use of the drape allows the capital equipment to be positioned in close proximity to the patient while still being located in an area that does not require sterility (i.e., the non-sterile field). On the other side of the sterile drape, the medical instrument may interface with the patient in an area that requires sterility (i.e., the sterile field).

[0055] D. Medical equipment. FIG. 16 illustrates an exemplary medical instrument with a paired instrument driver. Similar to other instruments designed for use with a robotic system, the medical instrument 70 includes an elongated shaft 71 (or elongated body) and an instrument base 72. The instrument base 72, also referred to as an "instrument handle" due to its design intended for manual interaction by a physician, may generally include a rotatable drive input 73, e.g., a receptacle, pulley, or spool, designed to mate with a drive output 74 extending through a drive interface on an instrument driver 75 at the distal end of a robotic arm 76. When physically connected, latched, and / or coupled, the mated drive input 73 of the instrument base 72 may share an axis of rotation with the drive output 74 in the instrument driver 75 to enable the transfer of torque from the drive output 74 to the drive input 73. In some embodiments, the drive output 74 may include a spline designed to mate with a receptacle on the drive input 73.

[0056] The elongated shaft 71 is designed to be delivered through either an anatomical opening or lumen, as in, for example, an endoscope, or a minimally invasive incision, as in, for example, a laparoscopy. The elongated shaft 71 may be either flexible (e.g., having similar properties to an endoscope) or rigid (e.g., having similar properties to a laparoscope), or may include a customized combination of both flexible and rigid portions. When designed for laparoscopy, the distal end of the rigid elongated shaft may be connected to an end effector extending from a jointed wrist formed from a clevis having at least one degree of freedom, and a surgical tool or medical instrument, such as, for example, a grasper or scissors, that may be actuated based on forces from a tendon as the drive input rotates in response to torque received from the drive output 74 of the instrument driver 75. When designed for endoscopy, the distal end of the flexible elongated shaft may include a steerable or controllable bend that may articulate and bend based on torque received from the drive output 74 of the instrument driver 75.

[0057] Torque from the instrument driver 75 is transferred downstream of the elongated shaft 71 using tendons along the elongated shaft 71. These individual tendons, such as pull wires, may be individually secured to individual drive inputs 73 in the instrument handle 72. From the instrument handle 72, the tendons are directed down one or more pull lumens along the elongated shaft 71 and secured to a distal portion of the elongated shaft 71, or to a wrist at the distal portion of the elongated shaft. During a surgical procedure, such as a laparoscopic, endoscopic, or hybrid procedure, these tendons may be coupled to a distally attached end effector, such as a wrist, grasper, or scissors. Under such a configuration, torque exerted on the drive input 73 transfers tension to the tendons, thereby actuating the end effector in some way. In some embodiments, during a surgical procedure, the tendons can rotate the joint about an axis, thereby moving the end effector in one direction or another. Alternatively, the tendon may be connected to one or more jaws of a grasper at the distal end of the elongate shaft 71, with tension from the tendon causing the grasper to close.

[0058] In endoscopy, the tendons may be coupled to a flex or articulating section positioned along the elongate shaft 71 (e.g., at the distal end) via adhesive, control rings, or other mechanical fixation. If fixedly attached to the distal end of the flex, torque exerted on the drive input 73 is transferred downstream of the tendons, causing the softer flex (sometimes called the articulatable section or articulatable region) to flex or articulate. Along the non-flexing portions, it may be advantageous to spiral or convolute the individual pull lumens that direct the individual tendons along (or inwardly) the wall of the endoscope shaft to counterbalance the radial forces resulting from tension in the pull wires. The angle of the helix and / or spacing between them can be altered or designed for a particular purpose, with tighter helices exhibiting less shaft compression under load forces, while lesser amounts of helices result in greater shaft compression under load forces but limited bending. At the other end of the spectrum, the pull lumen may be oriented parallel to the longitudinal axis of the elongate shaft 71 to allow for controlled articulation at a desired bend or articulatable section.

[0059] In endoscopy, the elongated shaft 71 houses multiple components that aid in robotic procedures. The shaft 71 may include a working channel for deploying surgical tools (or medical instruments), irrigating, and / or aspirating the surgical field at the distal end of the shaft 71. The shaft 71 may also house wires and / or optical fibers for transmitting signals to and from an optical assembly at the distal tip, which may include an optical camera. The shaft 71 may also house optical fibers for carrying light from a proximally located light source, such as a light emitting diode, to the distal end of the shaft 71.

[0060] At the distal end of the instrument 70, the distal tip may include a working channel opening for delivering tools to the surgical site for diagnosis and / or treatment, irrigation, and aspiration. The distal tip may also include a port for a camera, such as a fiberscope or digital camera, to capture images of the internal anatomical space. Relatedly, the distal tip may also include a port for a light source to illuminate the anatomical space when a camera is used.

[0061] 16, the drive shaft axis, i.e., the drive input axis, is orthogonal to the axis of the elongated shaft 71. However, this configuration complicates the ability of the elongated shaft 71 to roll. Rolling the elongated shaft 71 along its axis while holding the drive input 73 stationary can result in undesirable entanglement of tendons as they extend from the drive input 73 and enter the pull lumen within the elongated shaft 71. Such resulting entanglement of tendons can interfere with control algorithms intended to predict movement of the flexible elongated shaft 71 during an endoscopic procedure.

[0062] FIG. 17 shows an alternative design of the instrument driver and instrument, where the axis of the drive unit is parallel to the axis of the elongated shaft of the instrument. As shown, a circular instrument driver 80 includes four drive units with parallel aligned drive outputs 81 at the end of a robotic arm 82. The drive units and their respective drive outputs 81 are housed in a rotating assembly 83 of the instrument driver 80, which is driven by one of the drive units in the assembly 83. In response to torque provided by the rotating drive units, the rotating assembly 83 rotates along a circular bearing connecting the rotating assembly 83 to a non-rotating portion 84 of the instrument driver 80. Power and control signals may be transferred from the non-rotating portion 84 of the instrument driver 80 to the rotating assembly 83 via electrical contacts that may be maintained throughout the rotation by a brushed slip ring connection (not shown). In other embodiments, the rotating assembly 83 may be responsive to a separate drive unit that is integrated into the non-rotatable portion 84 and thus not parallel to the other drive units. The rotation mechanism 83 enables the instrument driver 80 to rotate the drive units and their respective drive outputs 81 as a single unit about the instrument driver axis 85 .

[0063] Similar to the previously disclosed embodiments, the instrument 86 may include an elongated shaft portion 88 and an instrument base 87 (shown with a clear exterior skin for purposes of illustration) that includes a number of drive inputs 89 (such as receptacles, pulleys, and spools) configured to receive the drive outputs 81 in the instrument driver 80. Unlike the previously disclosed embodiments, the instrument shaft 88 extends from the center of the instrument base 87 with an axis that is substantially parallel to the axis of the drive inputs 89, rather than orthogonal as in the design of FIG.

[0064] When coupled to the rotation assembly 83 of the instrument driver 80, the medical instrument 86, including the instrument base 87 and the instrument shaft 88, rotates with the rotation assembly 83 about the instrument driver axis 85. Because the instrument shaft 88 is positioned in the center of the instrument base 87, the instrument shaft 88 is coaxial with the instrument driver axis 85 when attached. Thus, rotation of the rotation assembly 83 causes the instrument shaft 88 to rotate about its own longitudinal axis. Furthermore, as the instrument base 87 rotates with the instrument shaft 88, any tendons connected to the drive input 89 in the instrument base 87 do not become entangled during rotation. Thus, the parallelism of the axes of the drive output 81, drive input 89, and instrument shaft 88 allows for rotation of the shaft without entangling any of the control tendons.

[0065] FIG. 18 illustrates an instrument 150 having an instrument-based insertion architecture, according to some embodiments. The instrument 150 may be coupled to any of the instrument drivers described above. The instrument 150 includes an elongated shaft 152, an end effector 162 connected to the shaft 152, and a handle 170 coupled to the shaft 152. The elongated shaft 152 includes a tubular member having a proximal portion 154 and a distal portion 156. The elongated shaft 152 includes one or more channels or grooves 158 along its outer surface. The grooves 158 are configured to receive one or more wires or cables 180 therethrough. Thus, the one or more cables 180 extend along the outer surface of the elongated shaft 152. In other embodiments, the cables 180 may extend through the elongated shaft 152. Manipulation of one or more of the cables 180 (e.g., via an instrument driver) results in actuation of the end effector 162.

[0066] The instrument handle 170, which may also be referred to as an instrument base, may generally include a mounting interface 172 having one or more mechanical inputs 174, such as receptacles, pulleys or spools, designed to reciprocally mate with one or more torque couplers on a mounting surface of the instrument driver.

[0067] In some embodiments, instrument 150 includes a series of pulleys or cables that allow elongate shaft 152 to translate relative to handle 170. In other words, instrument 150 itself includes an instrument-based insertion architecture that accommodates the insertion of the instrument, thereby minimizing reliance on a robotic arm to provide insertion of instrument 150. In other embodiments, the robotic arm can be significantly involved in the insertion of the instrument.

[0068] E. Controller. Any of the robotic systems described herein can include an input device or controller for manipulating an instrument attached to the robotic arm. In some embodiments, the controller can be coupled (e.g., communicatively, electronically, electrically, wirelessly, and / or mechanically) to the instrument such that manipulation of the controller causes a corresponding manipulation of the instrument, e.g., via master-slave control.

[0069] 19 is a perspective view of one embodiment of controller 182. In this embodiment, controller 182 includes a hybrid controller that can have both impedance control and admittance control. In other embodiments, controller 182 can utilize only impedance or passive control. In other embodiments, controller 182 can utilize only admittance control. By being a hybrid controller, controller 182 can advantageously have a lower perceived inertia during use.

[0070] In the illustrated embodiment, the controller 182 is configured to enable operation of two medical instruments and includes two handles 184. Each of the handles 184 is connected to a gimbal 186. Each gimbal 186 is connected to a positioning platform 188.

[0071] 19, each positioning platform 188 includes a selective compliance assembly robot arm (SCARA) 198 coupled to a column 194 by a prismatic joint 196. The prismatic joint 196 is configured to translate along the column 194 (e.g., along rails 197) to allow each of the handles 184 to be translated in the z-direction, providing a first degree of freedom. The SCARA 198 is configured to enable movement of the handles 184 in the xy plane, providing two additional degrees of freedom.

[0072] In some embodiments, one or more load cells are positioned within the controller. For example, in some embodiments, a load cell (not shown) is positioned within the body of each of the gimbals 186. By providing load cells, a portion of the controller 182 can operate under admittance control, thereby advantageously reducing the perceived inertia of the controller during use. In some embodiments, the positioning platform 188 is configured for admittance control and the gimbal 186 is configured for impedance control. In other embodiments, the gimbal 186 is configured for admittance control and the positioning platform 188 is configured for impedance control. Thus, in some embodiments, the translational or positional degree of freedom of the positioning platform 188 can rely on admittance control, while the rotational degree of freedom of the gimbal 186 relies on impedance control.

[0073] F. Navigation and Control. Traditional endoscopy may involve the use of fluoroscopy (e.g., as may be delivered through a C-arm) and other forms of radiation-based imaging modalities to provide intraluminal guidance to the operator-physician. In contrast, the robotic system contemplated by the present disclosure may provide non-radiation-based navigation and localization means to reduce the physician's exposure to radiation and reduce the amount of equipment in the operating room. As used herein, the term "localization" may refer to determining and / or monitoring the position of an object in a reference coordinate system. Techniques such as pre-operative mapping, computer vision, real-time EM tracking, and robot command data may be used individually or in combination to achieve a radiation-free operating environment. In other cases where radiation-based imaging modalities are still used, pre-operative mapping, computer vision, real-time EM tracking, and robot command data may be used individually or in combination to improve the information obtained solely by radiation-based imaging modalities.

[0074] FIG. 20 is a block diagram illustrating a localization system 90 for estimating a location of one or more elements of a robotic system, such as a location of an instrument, according to an exemplary embodiment. The localization system 90 may be a set of one or more computing devices configured to execute one or more instructions. The computing devices may be embodied by a processor (or processors) and computer readable memory in one or more of the components discussed above. By way of example, and not by way of limitation, the computing devices may be in the tower 30 shown in FIG. 1, in the cart 11 shown in FIGS. 1-4, or in the bed shown in FIGS. 5-14.

[0075] 20, the localization system 90 may include a localization module 95 that processes the input data 91-94 to generate location data 96 of the distal tip of the medical instrument. The location data 96 may be data or logic that represents the location and / or orientation of the distal tip of the instrument relative to a frame of reference. The frame of reference may be relative to a known object such as the patient's anatomy or an EM field generator (see below for a description of EM field generators).

[0076] The various input data 91-94 will now be described in more detail. Pre-operative mapping can be used by the localization module 95 to generate model data 91. Pre-operative mapping can be accomplished using the acquisition of low-dose CT scans. The pre-operative CT scans are reconstructed into three-dimensional images that are visualized, for example, as cutaway "slices" of the patient's internal anatomy. When analyzed as a whole, an image-based model can be generated for the anatomical cavities, spaces, and structures of the patient's anatomy, such as the patient's pulmonary network. Techniques such as center-line geometry can be determined and approximated from the CT images to create a three-dimensional volume of the patient's anatomy, referred to as model data 91 (also referred to as "pre-operative model data" when generated using only pre-operative CT scans). The use of center-line geometry is discussed in U.S. Patent Application Serial No. 14 / 523,760, the contents of which are incorporated herein in their entirety. Network topological models may also be derived from CT images and are particularly suited for bronchoscopy.

[0077] In some embodiments, the instrument may include a camera to provide visual data (or image data) 92 to a localization module 95. The localization module 95 may process the visual data 92 to enable one or more visual-based (or image-based) location tracking modules or functions. For example, the pre-operative model data 91 may be used in conjunction with the visual data 92 to enable computer vision-based tracking of a medical instrument (e.g., an endoscope or an instrument advancing through a working channel of the endoscope). For example, using the pre-operative model data 91, the robotic system may generate a library of predicted endoscopic images from the model based on the expected path of travel of the endoscope, with each image linked to a location in the model. During surgery, this library may be referenced by the robotic system to compare real-time images captured by a camera (e.g., a camera at the distal end of the endoscope) to those in the image library to assist with localization.

[0078] Other computer vision based tracking techniques use feature tracking to determine the movement of the camera, and therefore the endoscope. Some mechanisms of the localization module 95 may identify circular geometric shapes in the pre-operative model data 91 that correspond to the anatomical lumens and track changes in those geometries to determine which anatomical lumens have been selected, as well as the relative rotational and / or translational movement of the camera. The use of a phase map may further enhance vision-based algorithms or techniques.

[0079] Optical flow, another computer vision based technique, may analyze the displacement and translation of image pixels in a video sequence in the visual data 92 to infer camera movement. Examples of optical flow techniques may include motion detection, object segmentation computations, luminance, motion compensated coding, stereo disparity measurements, etc. By comparing multiple frames over multiple iterations, the movement and location of the camera (and therefore the endoscope) can be determined.

[0080] The localization module 95 can use the real-time EM tracking and EM data 93 to generate a real-time location of the endoscope in a global coordinate system that can be registered to the patient's anatomy represented by the pre-operative model. In EM tracking, an EM sensor (or tracker) including one or more sensor coils embedded in one or more locations and orientations in a medical instrument (e.g., an endoscopic instrument) measures variations in an EM field generated by one or more static EM field generators positioned at known locations. The location information detected by the EM sensor is stored as EM data 93. The EM field generator (or transmitter) can be placed in close proximity to the patient to generate a low-intensity magnetic field that can be detected by the embedded sensor. The magnetic field induces small currents in the sensor coils of the EM sensor, which can be analyzed to determine the distance and angle between the EM sensor and the EM field generator. These distances and orientations can be "registered" intraoperatively to the patient's anatomy (e.g., the pre-operative model) to determine a geometric transformation that aligns a single location in the coordinate system with a location in the pre-operative model of the patient's anatomy. Once aligned, an EM tracker embedded at one or more locations on the medical instrument (e.g., the distal tip of an endoscope) can provide a real-time indication of the medical instrument's progression through the patient's anatomy.

[0081] The robotic commands and kinematic data 94 may also be used by a localization module 95 to provide location data 96 for the robotic system. The device pitch and yaw resulting from the articulation commands may be determined during pre-operative calibration. Intraoperatively, these calibration measurements may be used in combination with known insertion depth information to estimate the location of the instrument. Alternatively, these calculations may be analyzed in combination with EM, vision, and / or phase modeling to estimate the location of the medical instrument within the network.

[0082] As Fig. 20 illustrates, multiple other input data can be used by the localization module 95. For example, although not shown in Fig. 20, an instrument that utilizes shape sensing fibers can provide shape data that the localization module 95 can use to determine the location and shape of the instrument.

[0083] The localization module 95 may use a combination of the input data 91-94. In some cases, such a combination may use a probabilistic approach in which the localization module 95 assigns a confidence weight to the location determined from each of the input data 91-94. Thus, if the EM data may be unreliable (such as when EM interference is present), which may reduce the reliability of the location determined by the EM data 93, the localization module 95 may rely more heavily on the vision data 92 and / or the robot command and kinematic data 94.

[0084] As discussed above, the robotic systems discussed herein can be designed to incorporate one or a combination of two or more of the above-mentioned technologies. The computer-based control system of the robotic system, whether based on the tower, bed, and / or cart, may store, for example, in a non-transitory computer-readable storage medium, such as a persistent magnetic storage drive, solid-state drive, etc., computer program instructions that, when executed, cause the system to receive and analyze sensor data and user commands, generate system-wide control signals, and display navigation and localization data, such as instrument position within a global coordinate system, anatomical maps, etc.

[0085] 2. Robot-Controllable Electromagnetic Field Generator Described herein are embodiments of a robotically controllable EM field generator that may be configured for use with a robotic medical system (e.g., those described above and others). The robotically controllable EM field generator may be configured to couple to (or otherwise be attached or integrated with) a robotic arm of a robotic medical system. Such a robotically controllable EM field generator may provide advantages over other EM field generators commonly used with robotic medical systems that typically require more complex alignment steps to relate the EM field coordinate frame to the robotic system coordinate frame or to a global coordinate frame, and generally cannot be repositioned during a procedure without having to redo the complex alignment steps.

[0086] By coupling the EM field generator to a robotic arm, the kinematics of the robotic arm can be used to align the coordinate frame of the EM field to the robot coordinate frame or to the global coordinate frame. Such alignment based on the kinematics of the robotic arm can be done automatically or without requiring an operator to perform a separate alignment step (such as those alignment steps described below with reference to FIGS. 22B and 22C). This can eliminate the need for more complicated or user-involved alignments that are common with other types of EM field generators used with robotic medical systems. Furthermore, with the EM field generator coupled to the robotic arm, the EM field generator becomes robotically controllable or positionable. The robotic arm can be commanded to translate, rotate, or a combination of both to physically move the EM field generator to various positions during a robotic medical procedure.

[0087] As described in more detail below, such robotically controllable EM field generators can provide multiple advantages and are useful in a wide variety of applications. For example, the use of such robotically controllable EM field generators can, among other things, eliminate the need to perform a separate alignment step to align the EM field coordinate frame with the robot coordinate frame or a global coordinate frame, provide a larger field of view through the ability to move the EM field generator with a robotic arm, enable the use of smaller EM field generators (e.g., compact electromagnetic field generators), simplify the setup of robotic medical systems, and increase the accuracy of the detected position of the EM sensor. Applications of such robotically controllable EM field generators can include, for example, automatic tool tracking, automatic or extended electromagnetic field generator setup, EM focusing, anatomical mapping, and others.

[0088] These and other features of the robotically-controllable electromagnetic field generator are described in more detail below, after first providing an overview of the use of EM sensors and EM field generators in robotic medical systems in general.

[0089] A. Overview of the use of EM sensors and EM field generators A robotic medical system can use various types of position sensors to facilitate tracking and navigation of various tools and instruments. As described above, one type of position sensor that can be used in a robotic medical system is an EM position sensor (also referred to herein as an EM sensor, beacon, or tracker). One or more EM sensors can be provided, for example, on a portion of a tool or instrument used during a medical procedure. The location of the EM sensor can be determined and used to track the location of the corresponding tool or instrument. One or more EM sensors can also be provided on the patient, for example, to track the patient's movement (such as movement due to breathing or other types of locomotion) during a procedure. As described above with respect to FIG. 20, the localization system 90 can use the EM data 93, along with various other types of data, to provide navigation and guidance information to an operator of the system.

[0090] The EM sensor is used in conjunction with one or more EM field generators configured to generate a low strength magnetic field. When the EM sensor is positioned within the magnetic field generated by the EM field generator (also referred to as the working volume of the EM field generator), the position of the EM sensor relative to the EM field generator can be determined. For example, as described above with reference to FIG. 20, the EM sensor may include one or more sensor coils that measure variations in the magnetic field generated by the EM field generator. The magnetic field can induce small currents in the sensor coils of the EM sensor that can be analyzed to determine, for example, the distance and angle between the EM sensor and the EM field generator.

[0091] In some embodiments, the position (e.g., three-dimensional position) of the EM sensor relative to the EM field generator can be determined. In some embodiments, the orientation (e.g., pitch, yaw, and / or roll) of the EM sensor can also be determined. A five degree of freedom (DoF) EM sensor can provide the three-dimensional position of the EM sensor, as well as the pitch and yaw of the EM sensor. A six DoF EM sensor can provide the three-dimensional position of the EM sensor, as well as the pitch, yaw, and roll of the EM sensor.

[0092] 21 illustrates an example of an EM field generator 202 and an EM sensor 204. The EM field generator 202 is configured to generate a magnetic field. In the illustrated embodiment, the magnetic field of the EM field generator 202 has a working volume 206. When the EM sensor 204 is positioned within the working volume 206 of the magnetic field generated by the EM field generator 202, the position (or position and orientation) of the EM sensor 204 may be determined relative to an EM coordinate frame 208 associated with the EM field generator 202. For example, the distance and angle between the EM sensor 204 and an origin of the EM coordinate frame 208 may be determined to determine the position (e.g., x, y, and z position) and / or orientation (e.g., pitch, yaw, and / or roll) of the EM sensor 204 within the working volume 206.

[0093] In the illustrated embodiment, the working volume 206 of the EM field generator 202 is represented as having the shape of a rectangular prism protruding from one side of the EM field generator 202. However, this represents just one type of working volume 206, and EM field generators 202 may be provided that generate working volumes 206 of various sizes, shapes, and positions relative to the EM field generator 202.

[0094] FIG. 22A illustrates an exemplary robotic medical system 200 that uses the EM field generator 202 of FIG. 21. The robotic medical system 200 may be similar to the robotic medical systems described above with reference to FIGS. 1-20. In the illustrated embodiment, the robotic medical system 200 includes a cart 210 that includes two robotic arms 212. Although two robotic arms 212 are illustrated, other numbers of robotic arms 212 are possible. For example, FIG. 2 (discussed above) illustrates an exemplary cart 11 that includes three robotic arms 12. Additionally, in some embodiments, the robotic arms 212 need not be attached to the cart 210. In some embodiments, the robotic arms may be coupled to a bed or patient platform 215, as shown, for example, in FIGS. 5-10 and 14 (discussed above). In some embodiments, the robotic medical system 200 may include a robotic arm 212 that is coupled to the cart 210, the patient platform, and / or other components of the system.

[0095] The robot arm 212 may be associated with a robot coordinate frame 216. Due to the known kinematics of the robot arm 212, the position of the robot arm 212 (e.g., the position of the distal end of the robot arm 212 and / or the position of an instrument 214 attached thereto) may be determined with reference to the robot coordinate frame 216. For example, the lengths of the various links that make up the robot arm 212 are known and the angles between the links of the robot arm 212 can be determined, so that the position of the robot arm is kinematically defined within the robot coordinate frame 216.

[0096] In some embodiments, the position and orientation of the cart 210 relative to the patient platform 215 and the patient may be determined, set, or controlled such that the robot coordinate frame 216 may be considered a global coordinate frame that may be aligned to the patient or a portion of the patient's anatomy. For example, in some embodiments, up, down, right, left, etc. in the robot coordinate frame 216 may correspond to up, down, right, left, etc. in a global coordinate frame that includes the patient, the patient platform 215, and / or other components.

[0097] 22A, a tool or instrument 214 (e.g., a laparoscopic or endoscopic instrument) may be coupled to one of the robotic arms 212. The instrument 214 may be inserted into a patient to perform a medical procedure. Although only a single instrument 214 is shown, it should be understood that other numbers of instruments 214 (e.g., two or more instruments) may be used in other embodiments. One or more robotic arms 212 may be used to position and control the instrument 214. An operator (not shown) may control the robotic arms 212 and the instrument 214 using a controller.

[0098] In the illustrated embodiment, the instrument 214 includes an EM sensor 204 positioned at its distal end. Additionally, the illustrated embodiment includes a second EM sensor 204 attached to the patient's chest, which may be used to track patient movement or motion, such as movement caused by the patient's breathing.

[0099] 22A illustrates that in some embodiments, the EM field generator 202 may be positioned relative to the patient such that the working volume 206 of the electromagnetic field generator 202 overlaps with a portion of the patient's anatomy where a medical procedure is performed. As used herein, the portion of the patient's anatomy where a medical procedure is performed may be referred to as a medical site, which may include a site where any medical procedure may be performed, including biopsy, endoscopy, surgery, therapy, etc. In some embodiments, the EM field generator 202 may be supported by a stand or other support structure (not shown) such that the working volume 206 is positioned relative to the patient. In other embodiments, the EM field generator 202 may be supported by or attached to the patient platform 215, or may be integrated into or supported by another component in the operating environment (such as the cart 30 described above with reference to FIGS. 1, 3, and 4).

[0100] 21 , once the EM sensor 204 is positioned within the working volume 206 of the EM field generator 202, the position of the EM sensor 204 in the EM coordinate frame 208 can be determined. However, determining the position of the EM sensor 204 in the EM coordinate frame 208 may not be particularly useful unless the EM coordinate frame 208 is aligned with the robot coordinate frame 216 or the global coordinate frame (which, as noted above, may be the robot coordinate frame 216). Thus, an alignment step that relates the EM coordinate frame 208 to the robot coordinate frame 216 is often required.

[0101] In some robotic medical systems, such as the robotic medical system 200 illustrated in FIG. 22A, aligning the EM coordinate frame 208 to the robot coordinate frame 216 may require the operator to perform certain steps to achieve alignment. The alignment process may add steps to a procedure and may increase procedure time. FIGS. 22B and 22C show example alignment steps or procedures that may be used to align the robot coordinate frame 216 associated with the robotic medical system 200 with the EM coordinate system 208 associated with the EM field generator 202. As described in more detail below, the need to perform these types of alignment steps or procedures may be eliminated or reduced by providing a robotic medical system with an EM field generator attached to its robotic arm, for example as shown in FIG. 23.

[0102] FIG. 22B illustrates an exemplary alignment procedure that may be used to align the robot coordinate frame 216 associated with the robotic medical system 200 with the EM coordinate system 208 associated with the EM field generator 202. To achieve the alignment, multiple points are identified in both the robot coordinate frame 216 and the EM coordinate frame 208. Once multiple points are identified in each of the robot coordinate frame 216 and the EM coordinate frame 208, an alignment algorithm can determine the transformation between the two coordinate frames. The alignment procedure of FIG. 22B uses one of the robot arms 212 of the robotic system, the EM field generator 202, the EM probe 203, and the alignment fixture 205. In the illustrated embodiment, the alignment fixture 205 comprises a cube with fiducials or markers located at the vertices of the cube. Thus, in the illustrated embodiment, the alignment fixture 205 includes eight fiducials. Other numbers of fiducials and other shapes for the alignment fixture 205 may also be used. With the alignment fixture 205 positioned at a rest location, an operator can command the robot arm 212 to contact each of the fiducials in the alignment fixture 205. Thus, the position of each fiducial in the robot coordinate frame 216 can be determined. The operator can then touch each of the fiducials with the EM probe 203. The EM probe 203 can include a handheld EM sensor 204 that has a well-defined tip position relative to the electromagnetic field generator 202. By contacting each of the fiducials with the EM probe 203, the position of each fiducial in the EM coordinate frame 208 can be determined. With the same points (the fiducials of the alignment fixture 205) now known in each of the robot coordinate frame 216 and the EM coordinate frame 208, an alignment algorithm can determine the transformation between the two coordinate frames.

[0103] 22C illustrates another exemplary alignment procedure that may be used to align the robot coordinate frame 216 associated with the robotic medical system 200 with the EM coordinate system 208 associated with the EM field generator 202. This alignment procedure uses one of the robot arms 212 of the robotic system, the EM field generator 202, and an EM sensor 204 attached to the end effector of the robot arm 212. Using the EM sensor 204 attached to the end effector of the robot arm 212, an operator can then command the robot arm 212 to move through or trace an alignment trajectory 207. In the illustrated embodiment, the alignment trajectory 207 includes a cubic shape, although other alignment trajectories including different shapes can be used. At various points along the alignment trajectory 207 (e.g., vertices of the cubic shape), the position of the end effector in the robot coordinate frame 216 and the position of the EM sensor 204 in the EM coordinate frame 208 can be recorded. Once a sufficient number of points have been identified in both the robot coordinate frame 216 and the EM coordinate frame 208, a registration algorithm can determine the transformation between the two coordinate frames.

[0104] In a robotic system such as the system 200 illustrated in FIG. 22A, an alignment procedure, such as any of the procedures described above with reference to FIGS. 22B and 22C, may be performed prior to initiating a procedure to align the EM coordinate frame 208 and the robot coordinate frame 216.

[0105] These types of alignments are associated with various drawbacks. For example, such alignments can be tedious and time consuming. Such alignments are generally not considered automatic because they may require user input and / or may need to be performed before starting a robotic medical procedure. Furthermore, such alignments can introduce inaccuracies into the system if, for example, the operator does not navigate to the fiducials precisely. In addition, such alignments require that the position of the EM field generator 202 remain fixed during the procedure. If the EM field generator 202 is moved (e.g., accidentally bumped by someone in the operating room or moved to allow access to the patient), the alignment must be performed again to re-establish the relationship. This can be problematic because, for example, the EM field generator 202 may need to be moved to allow access to a fluoroscopy C-arm. In this case, it would be necessary to remove the EM field generator 202, move the C-arm to a suitable position for capturing one or more images, then remove the C-arm, return the EM field generator 202 to its proper position, and have the operator redo the alignment step between the EM coordinate frame and the robot or global coordinate frame.

[0106] As described in more detail below, the need for these types of alignments may be reduced or eliminated through the use of a robot-controllable electromagnetic field generator that may be directly coupled to a robot arm, for example, as shown in Figures 23-25. By coupling the robot-controllable electromagnetic field generator to the robot arm, the relationship or alignment between the robot-controllable electromagnetic field generator and the robot arm may be kinematically determined, and thus the EM coordinate frame 208 may be located within the robot coordinate frame 216.

[0107] Additionally, in previous systems using alignment procedures such as those described with reference to FIGS. 22B and 22C, the position of the EM field generator 202 must remain fixed during the procedure, which may require the use of physically larger and sometimes more cumbersome EM field generators 202 to generate a working volume 206 large enough in size to encompass the relevant medical site. In some situations or applications, this may increase the difficulty in setting up and positioning the EM field generator 202 relative to the patient. In some situations or applications, this may limit the available space around the patient during the procedure and / or require more setup time. For example, the use of a robotically controllable electromagnetic field generator that may be directly coupled to a robotic arm, as shown in FIGS. 22-25, may reduce or eliminate one or more of these limitations associated with the EM field generator 202 as shown in FIG. 22A.

[0108] B. Overview of a robot-controllable electromagnetic field generator 23 illustrates an embodiment of a robotic medical system 300 that includes an embodiment of a robotically controllable EM field generator 302 coupled to a robotic arm 212. As described in more detail below, coupling the EM field generator 302 to the robotic arm 212 can provide several advantages including, for example, (1) allowing for more accurate and / or simpler alignment between the EM coordinate frame 308 of the EM field generator 302 and the robot coordinate frame 216 based on the kinematics of the robotic arm 212, and / or (2) allowing for the position of the EM field generator 302 to be moved or realigned using the robotic arm 212.

[0109] Similar to the robotic medical system 200 of FIG. 22A, in the robotic medical system 300 of FIG. 23, the robotic arm 212 can be associated with a robot coordinate frame 216. Due to the known kinematics of the robotic arm 212, the position of the robotic arm 212 can be determined with reference to the robot coordinate frame 216. For example, the position of the robotic arm 212 is kinematically defined in the robot coordinate frame 216 because the lengths of the various links that make up the robotic arm 212 are known and the angles between the links of the robotic arm 212 can be determined. Additionally, for some embodiments of the robotic medical system 300, the position and orientation of the cart 210 relative to the patient platform 215 and the patient can be determined, set, or controlled such that the robot coordinate frame 216 can be considered a global coordinate frame that can be aligned with the patient or a portion of the patient's anatomy. For example, in some embodiments, up, down, right, left, etc. in the robot coordinate frame 216 can correspond to up, down, right, left, etc. in a global coordinate frame that includes the patient, the patient platform 215, and / or other components. Doing so can facilitate the operator's ability to navigate within the patient's anatomy. As a more specific example, during a bronchoscopy or ureteroscopy procedure, where the robotic system 300 includes a cart from which the robotic arm 212 extends, the cart can be positioned parallel to the bed. The bed and cart have a common gravity vector (i.e., a lift point to the ceiling for both of them). Using this information, the system can provide spatial context to show the user which direction is up (anterior) or down (posterior) as the user is driving the bronchoscope or ureteroscope. For urological applications, the user may need to specify whether the ureteroscope is in the posterior or anterior calyx during target selection for percutaneous access. Since the orientation of the system is known, the system can provide anatomical context to facilitate this navigation.

[0110] As shown in FIG. 23, the EM field generator 302 may be coupled to one of the robotic arms 212. In some embodiments, the EM field generator 302 is coupled or attached to the distal end of the robotic arm 212, although the EM field generator 302 may be coupled or attached to the robotic arm 212 at other locations (e.g., at a location between the distal and proximal ends of the arm). FIGS. 24 and 25, described below, show more detailed embodiments of the EM field generator 302 and how, in some embodiments, it may be coupled to the robotic arm 212. Other methods and mechanisms for coupling the EM field generator 302 to the robotic arm 212 are also possible. Additionally, in some embodiments, the EM field generator 302 may be integrated into the robotic arm 212 itself (e.g., may be a component on or in the robotic arm 212). For example, the EM field generator 302 may be integrated into one of the links of the robotic arm 212 (such as the distal-most link) or into an instrument driver or instrument drive mechanism positioned, for example, at the distal end of the robotic arm 212. Exemplary instrument drivers 62, 75, 83, 146 that may incorporate the EM field generator 302 are shown in Figures 14-17 above, although these examples are not limiting.

[0111] By coupling the EM field generator 302 to the robotic arm 212, the kinematics of the robotic arm 212 can be used to provide alignment between the EM coordinate frame 308 of the EM field generator 302 and the robot coordinate frame 216 and / or the global coordinate frame. That is, the kinematics of the robotic arm 212 (which are known as described above) can be used to relate the EM coordinate frame 308 of the EM field generator 302 to the robot coordinate frame 216 or the global coordinate frame. This can provide one or more advantages over a robotic medical system that includes a separate EM field generator 202, such as the robotic system 200 of FIG. 22A.

[0112] One advantage may be that by coupling the EM field generator 302 to the robotic arm 212, the need for a separate alignment step (which may require user input, as discussed above) may be eliminated. For example, in the robotic medical system 300, it may not be necessary to manually perform a separate alignment step, such as the alignment steps described with reference to FIGS. 22B and 22C, to align the EM coordinate frame 308 to the robot coordinate frame 216 or the global coordinate frame. In the robotic medical system 300, such alignment may be performed automatically (e.g., by the system) by simply attaching the EM field generator 302 to the robotic arm 212. The robotic medical system 300 may then relate to the EM coordinate frame 308 and the robot coordinate frame 216 or the global coordinate frame using the known kinematics of the robotic arm 212.

[0113] Another advantage that may be achieved by coupling the EM field generator 302 to the robotic arm 212 is that the accuracy of the alignment between the EM coordinate frame 308 and the robot coordinate frame 216 may be improved. As mentioned above, in a system such as the robotic system 200 of FIG. 22A, the alignment step may be inaccurate due to, for example, navigational imprecision during the manual alignment step. In the robotic medical system 300, the accuracy of the alignment between the EM coordinate frame 308 and the robot coordinate frame 216 is determined primarily by the accuracy of the kinematics of the robotic arm 212. If these are known and well defined, the alignment between the EM coordinate frame 308 and the robot coordinate frame 216 can be very accurate.

[0114] A further advantage provided by some embodiments including coupling the EM field generator 302 to the robotic arm 212 may be that the position of the EM field generator 302 may be adjusted using the robotic arm 212. This, in itself, may provide one or more advantages. As an example, the robotic arm 212 may be used to move or reposition the EM field generator 302. This may allow, for example, an operator to adjust the position of the working volume 306 of the magnetic field of the EM field generator 302. As described below, for example, the robotic arm 212 may be used to move the EM field generator 302 such that the EM field generator 302 tracks the movement of the instrument 214 as it moves through the body. Additionally, the EM field generator 302 may be moved or repositioned by the robotic arm 212 to facilitate access to the patient, for example to allow a fluoroscopy C-arm to access the patient. For example, the robotic arm 212 may be used to move the EM field generator 302 out of the way. Furthermore, when the robot arm 212 moves the EM field generator 302, the relationship between the EM coordinate frame 308 and the robot coordinate frame 216, as determined by the kinematics of the robot arm 212, remains known. Therefore, it may not be necessary to realign the EM coordinate frame 308 with respect to the robot coordinate frame 216 after moving the EM field generator 302.

[0115] Another advantage achievable by coupling the EM field generator 302 to the robot arm 212 may include improved accuracy of the determined position of the EM sensor 204, as the orientation and distance between the EM field generator 302 and the EM sensor 204 may be adjusted to improve accuracy. For example, the accuracy of the determined position of the EM sensor 204 may decrease as it approaches the edge of the working volume 306. If it is determined that the EM sensor 204 is closer to the edge of the working volume 306, the robot arm 212 may move the EM field generator 302 such that the EM sensor 204 is positioned closer to the center of the working volume 306 (or away from the edge of the working volume 306), which may increase the accuracy of the determined position. As another example, the robot arm 212 may move the EM field generator 302 closer to the EM sensor 204 to focus on the EM sensor 204 and more closely track the position of the EM sensor 204 (referred to herein as EM focusing).

[0116] In addition, because the robot arm 212 can easily adjust the position of the EM field generator 302 without requiring a separate alignment step, smaller EM field generators can be used compared to other robotic systems (such as the robotic system 200) that generally use stationary EM field generators. If stationary EM field generators are used, they must be large enough to provide a working volume large enough to cover the medical site. Even so, they may not cover all parts of the patient's anatomy, such as the portion of the patient's anatomy that is navigated through to access the medical site. By coupling the EM field generator 302 to the robotic arm 212, smaller EM field generators can be used, which may have a smaller working volume in some cases, because the position of the working volume 306 can be adjusted by moving the EM field generator 302 with the robotic arm 202. In some embodiments, this may enable the use of compact electromagnetic field generators (cFGs). However, the use of compact electromagnetic field generators is not required, and in some embodiments, larger electromagnetic field generators can be coupled to the robotic arm.

[0117] When coupled to the robotic arm 212, the EM field generator 302 may function similarly to the EM field generator 202 described above. For example, in the illustrated embodiment of the system 300 of FIG. 23, one or more of the instruments 214 may include one or more EM sensors 204 positioned at their distal ends and / or on other portions of the instruments 214. In addition, the illustrated embodiment includes a second EM sensor 204 attached to the patient's chest that may be used to track motion caused by the patient's breathing. Once the EM sensor 204 is positioned within the working volume 206 of the EM field generator 302, the position of the EM sensor 204 within the EM coordinate frame 308 may be determined.

[0118] Additionally, the kinematics of the robot arm 212 can be used to align the EM coordinate frame 308 with respect to the robot coordinate frame 216, such that the position of the EM sensor 204 in the robot coordinate frame 216 or the global coordinate frame can be determined. By physically connecting the EM field generator 302 to the robot arm 212, the EM coordinate frame 308 and the robot coordinate frame 216 or the global coordinate frame can be linked such that the position of the EM sensor 204 can be determined in the robot coordinate frame 216 or the global coordinate frame.

[0119] For example, in the illustrated embodiment of FIG. 23 , the robotic medical system 300 may include an EM field generator 302 configured to generate an EM field. The robotic medical system 300 may include a robotic arm 212. The first robotic arm 212 may be coupled to the EM field generator 302 and configured to articulate to move the EM field generator 302. The robotic medical system 300 may also include one or more processors configured to determine a position of the EM sensor 204 within the EM field in an EM coordinate frame 302 associated with the EM field generator 302. The processor may also be configured to determine an alignment between the EM coordinate frame 308 and a robot coordinate frame 216 associated with the first robotic arm 212 based on determining the position of the EM field generator 302 within the robot coordinate frame. This may be accomplished by using the kinematics of the robotic arm 212, as described above. The processor may further be configured to determine a position of the EM sensor 204 within the robot coordinate frame 216 based on the alignment.

[0120] 23, the robotic medical system 300 may include a controller or control circuitry comprising one or more processors configured to control movement of one or more robotic arms 212. The robotic arm 212 may be configured to articulate in response to receiving commands from the control circuitry. The EM field generator 302 may be coupled to the robotic arm 212 such that the EM field generator also moves in response to one or more commands from the control circuitry.

[0121] In addition to the robot-controllable EM field generator 302 coupled to the robotic arm 212, the illustrated embodiment of the robotic system 300 of FIG. 23 may be similar in many respects to the robotic system 200 described above with reference to FIG. 22A, including a separate EM field generator 202, or other robotic medical systems described throughout this application or elsewhere. For example, in the illustrated embodiment, the robotic medical system 300 comprises a cart 210 including two robotic arms 212. Although shown with two robotic arms 212, the robotic system 300 may use other numbers of robotic arms 212. For example, FIG. 2 (above) shows an exemplary cart 11 including three robotic arms 12. Additionally, in some embodiments, the robotic arm 212 need not be attached to the cart 210. In some embodiments, the robotic arm may be coupled to a bed or patient platform 215, for example, as shown in FIGS. 5-10 and 14 (above). In some embodiments, the robotic medical system 300 may include multiple robotic arms 212 (e.g., two, three, four, five, six, or more robotic arms 212) coupled to the cart 210, the patient platform, and / or other components of the system.

[0122] Additionally, while the robotic system 300 is shown in FIG. 23 with a single tool or instrument 214 (e.g., a laparoscopic or endoscopic instrument) coupled to one of the robotic arms 212, it should be understood that in other embodiments, other numbers of instruments 214 (e.g., two, three, four, five, six, or more instruments including a laparoscope, endoscope, and camera) may be used.

[0123] 24 illustrates one embodiment of an EM field generator 302. In the illustrated embodiment, the EM field generator 302 is configured to couple to an instrument drive mechanism 402 that may be positioned on a robotic arm such that the EM field generator 302 is attachable to the robotic arm 212. As discussed above with reference to FIG. 16 , an instrument drive mechanism such as the instrument drive mechanism 402 may be configured with an interface 404 adapted for attachment to various tools or instruments for use by a robotic medical system. The EM field generator 302 may be configured with a corresponding interface 320 configured to couple to the interface 404 of the instrument drive mechanism 402. In this manner, the EM field generator 302 may be coupled to the robotic arm 212 in a manner similar to other robotic tools or instruments of a robotic system.

[0124] For example, the interface 320 of the EM field generator 302 includes one or more connectors 322 configured to couple to one or more corresponding connectors 406 on the interface 404 of the instrument drive mechanism 402. Such a configuration may allow an operator to easily removably couple the EM field generator 302 to the instrument drive mechanism 402.

[0125] In the illustrated embodiment, the EM field generator 302 includes a housing 324. A proximal end 326 of the housing 324 may be configured to couple to the instrument drive mechanism 402. For example, the proximal end 326 may include an interface 320 and a connector 322. The housing 324 may extend from the proximal end 326 to a distal end 328. The distal end 328 may include an EM field generator unit 330 configured to generate a magnetic field. In the illustrated embodiment, the distal end 328 includes a clip 332 configured to secure the EM field generator unit 330. Other mechanisms or methods for securing the EM field generator unit 330 are possible. In some embodiments, the EM field generator unit 330 may be removable from the housing 324, for example, as shown. In some embodiments, the EM field generator unit 330 is integrated into the housing 324. In the illustrated embodiment, the EM field generator unit 330 includes a cFG, although other types of electromagnetic field generators may be used.

[0126] The size and shape of the housing 326 may be determined or selected such that the kinematic relationship between the EM field generator unit 330 and the robot arm to which the EM field generator 302 is mounted is known, thereby allowing the kinematic alignment between the EM coordinate frame associated with the EM field generator and the robot coordinate frame associated with the robot arm to be determined.

[0127] In the illustrated embodiment, the EM field generator unit 330 includes one or more connectors 334. The connectors 334 may electrically connect the EM field generator unit 330 to a robotic system such that the robotic system may communicate with the EM field generator unit. In other embodiments, an electrical connection may be made between the interface 320 of the EM field generator 302 and the interface 404 of the instrument drive mechanism 402.

[0128] Figure 25 shows how the EM field generator 302 may be attached to the instrument drive mechanism 402, according to one embodiment. In the illustrated embodiment, the instrument drive mechanism 402 is shown covered with a sterile drape. A sterile adapter 410 is shown positioned over the interface 404 of the instrument drive mechanism 402. The interface 320 of the EM field generator 302 may be attached to the sterile adapter 410 such that the EM field generator 302 is coupled to the instrument drive mechanism 402 with the sterile adapter 410 positioned therebetween. As shown in Figure 25, the EM field generator unit 330 may also be covered with a sterile drape.

[0129] FIG. 26 is a flow chart illustrating one embodiment of a method 500 for performing a robotic medical procedure using an EM field generator configured to couple (or otherwise connect or integrate) with a robotic arm. In the illustrated embodiment, the method begins at block 502. Block 502 may include generating an EM field using an EM field generator coupled to a first robotic arm. The EM field may be associated with an EM coordinate frame. The method 500 then moves to block 504, where an alignment between the EM coordinate frame and a robot coordinate frame associated with the first robotic arm is determined. The alignment may be based on determining a position of the EM field generator in the robot coordinate frame based on the kinematics of the first robotic arm. At block 506, a position of an EM sensor in the EM coordinate frame is determined. At block 508, the method includes determining a position of the EM sensor in the robot coordinate frame based on the alignment determined at block 504.

[0130] In some embodiments, the method 500 may optionally include removably coupling the EM field generator to a first robotic arm, for example as shown in FIG. 25. The method 500 may also optionally include moving the EM field generator with the first robotic arm. Moving the EM field generator may be useful in that the working volume of the EM field may be positioned as needed to facilitate a procedure and / or the EM field generator may be moved out of the way to allow access to the patient during a medical procedure. Moving the EM field generator with the first robotic arm to adjust the position of the EM field generator relative to the EM sensor may also improve the accuracy of the determined position of the EM sensor in the EM field coordinate frame (and correspondingly, in the robot coordinate frame through alignment).

[0131] The method 500 can also include moving a medical instrument coupled to a second robotic arm, the robot coordinate frame also being associated with the second robotic arm. The EM sensor can be positioned on the medical instrument such that navigational guidance of the second instrument can be determined based at least in part on a determined position of the EM sensor.

[0132] C. Exemplary Applications of Robotically Controllable Electromagnetic Field Generators In addition to providing one or more of the advantages described above, a robotically-controllable electromagnetic field generator coupled to or otherwise integrated with a robotic arm may be used to enable, facilitate, and / or improve a variety of functionality during a robotic medical procedure. This section outlines several exemplary applications for a robotically-controllable electromagnetic field generator that may be advantageous over other robotic systems that include, for example, static or stationary electromagnetic field generators.

[0133] i. Electromagnetic field generator settings and device tracking As mentioned above, a robotically controllable electromagnetic field generator coupled to or incorporated into a robot arm can be used to facilitate setting the electromagnetic field generator and tracking the instrument. This section provides several examples to illustrate these concepts without limiting them.

[0134] As used herein, "setting" an EM field generator refers to determining where the EM field generator should be positioned to facilitate a procedure. The position may be based on a determined position of one or more EM sensors, for example. In some cases, the position may be adjusted during a procedure, for example, when one or more of the EM sensors are moved during a procedure. As described above with reference to FIGS. 21 and 22A, the EM field generator 202 is positioned relative to the patient such that the working volume 206 of the electromagnetic field generator 202 overlaps a portion of the patient's anatomy (e.g., medical site) where the medical procedure is to be performed. The EM field generator 202 may be supported by a stand or other support structure, such as a patient platform 215. Once positioned, an alignment step (such as that described with reference to FIGS. 22B and 22C) is performed to align the EM coordinate frame 208 associated with the EM field generator 202 with the robot coordinate frame 216. Additionally, the EM field generator 202 was not typically moved during the procedure because the movements would have to be done manually and the alignment steps would have to be repeated to realign the EM coordinate frame 208 associated with the EM field generator 202 to the robot coordinate frame 216. A robot-controllable EM field generator that can be repositioned using the system's robotic arm would facilitate the set-up procedure by allowing the system to determine where the EM field generator should be positioned and then using the system's robotic arm to move the EM field generator to those locations. Additionally, as the positions of the tools and instruments, including the EM sensors, change during the procedure, the system can adjust the position of the EM field generator so that the EM field generator remains positioned in an advantageous location.

[0135] 28A and 28B show an introductory example illustrating that a robotically controlled EM field generator 302 (e.g., as described above with reference to FIGS. 23-25) can be used to facilitate the setting up and positioning of various tools and devices used in a medical procedure. Additional examples are described in more detail below. FIG. 28A shows three EM position sensors 204 as well as an EM field generator 302 coupled to a robotic arm 212. The EM position sensors 204 can be positioned, for example, on a medical instrument (such as an endoscope, laparoscope, and / or other type of surgical tool) or directly on a patient (e.g., an EM patch sensor configured to track the patient's movements or respiration). For ease of illustration, the medical instruments are not shown.

[0136] As shown in FIG. 28A, the system may be configured to move the EM field generator 302 using the robotic arm 212 to locate the EM sensor 204. Such movement may include translation, rotation, and / or vibration of the EM field generator 302. An example of a circular path is shown in FIG. 28A. Once the location of the EM sensor 202 is determined, the robotic arm 212 may move to reposition the EM field generator 302 to a relatively improved (referred to herein as a "working location") or centralized location where the EM sensor 204 is positioned within the working volume 306 of the EM field generator 302, for example, as shown in FIG. 28B. This may facilitate setup since the system may determine the working location or central location where the EM field generator 302 should be automatically positioned. In a related aspect, the location of one or more EM sensors 204 may be adjusted to achieve a relatively improved placement of the EM sensor 204 based on the range of movement or vibration achievable by the EM field generator, the medical procedure step, and / or the anatomical characteristics of the patient.

[0137] In some embodiments, the system locates and determines the position and / or orientation of the EM sensor 204 and then calculates the centroid or geometric center of those positions, or a shape defined by one or more of the positions. The EM field generator 302 can then be positioned, for example using the robotic arm 212, such that the center of the working volume 306 is aligned with the determined centroid of the EM sensor 204 position.

[0138] 28A illustrates that the EM field generator 302 can increase the functional size of the working volume 306. For example, if the EM sensors 204 are spaced apart such that they cannot all fit within the working volume 306, the system can move the EM field generator (in a scanning motion) to create a functional working volume 306 that is large enough to track all of the EM sensors 204. In some embodiments, the system can prioritize certain EM sensors 204 such that they always remain positioned within the working volume 306, while other EM sensors 204 are sometimes inside and sometimes outside the working volume 306.

[0139] "Instrument tracking," as used herein, generally refers to moving a robot-controllable EM field generator using a robot arm to track or follow the movement of an EM sensor. As discussed above, conventional systems generally involve the use of stationary EM field generators that cannot be moved or cannot be moved (to avoid realignment) to track or follow the movement of an EM sensor. In such systems, if the EM sensor moves outside the working volume of the EM field generator, it will no longer be detectable until it is moved back into the working volume. A robot-controllable electromagnetic field generator that can be moved with the robot arm can facilitate instrument tracking by allowing the EM field generator to be moved or otherwise adjusted to track the movement of one or more instrument sensors. Thus, rather than moving the EM sensor outside the working volume of the EM field generator, the EM field generator can be moved with the EM sensor such that the EM sensor remains positioned within the working volume, facilitating continuous tracking of the EM sensor.

[0140] 27A and 27B illustrate an introductory embodiment of automatic instrument tracking that may be facilitated using a robotically controllable electromagnetic field generator, such as EM field generator 302 described above with reference to FIGS. 23-25. Specifically, FIGS. 27A and 27B illustrate an example of automatic instrument tracking during an exemplary medical procedure, for example, during a ureteroscopic procedure in which a ureteroscope 502 is robotically navigated through a patient's opening, through the ureter, and into the kidney. Although a ureteroscopic procedure is illustrated, automatic instrument tracking may be used in other types of procedures, including endoscopic and / or laparoscopic procedures. FIG. 27A illustrates the procedure at an earlier or first time / step (e.g., immediately after insertion of the ureteroscope 502) and FIG. 27B illustrates the procedure at a later or second time / step (e.g., after the distal tip of the ureteroscope 502 has been navigated into the kidney).

[0141] As shown in Figures 27A and 27B, the ureteroscope 502 may be coupled to one or more robotic arms 212 configured to manipulate and insert the ureteroscope 502, for example, as described with respect to Figure 3. In Figures 27A and 27B, the EM field generator 302 is coupled to a third robotic arm 212 configured to move to adjust the position of the EM field generator 302 (and correspondingly, the position of the working volume 306 of the magnetic field generated by the EM field generator 302). As described above, because the EM field generator 302 is attached to the robotic arm 212, the EM coordinate frame associated with the EM field generator 302 may be automatically aligned to the robot coordinate frame associated with the robotic arm 212 through the kinematics of the robotic arm 212 to which the EM field generator 302 is attached.

[0142] 27A and 27B, the system may be configured to move the EM field generator 302 with a corresponding robotic arm 212 to automatically track the position of the ureteroscope 502. In the illustrated embodiment, the ureteroscope 502 includes an EM sensor 204 positioned at its distal tip. As described above, the position of the EM sensor 204 can be determined relative to the EM coordinate frame when the EM sensor 204 is positioned within the working volume 306 of the EM field generator 302. Furthermore, the EM coordinate frame can be aligned to the robot coordinate frame, such that the position of the EM sensor 204 in the robot coordinate frame can also be determined.

[0143] As the ureteroscope 502 is inserted further into the patient, the robotic arm 212 to which the EM field generator is attached may be moved or adjusted to automatically reposition the EM field generator such that the position sensor 204 remains positioned within the working volume 306 of the EM field generator. In some embodiments, this automatic tracking of the ureteroscope 502 is configured such that the EM field generator 302 is moved or adjusted such that the EM sensor 204 remains positioned at the center of the working volume 306, although this need not be the case in all embodiments. Additionally, in some embodiments, the orientation of the EM field generator 302 may be adjusted relative to the EM sensor 204 to provide optimal or improved accuracy in determining its position.

[0144] This type of automatic tracking may advantageously allow the position of the ureteroscope 502 to be tracked throughout the entire procedure (e.g., from insertion into the kidney to operation), which may not be possible with robotic systems that use static or stationary electromagnetic field generators. As mentioned above, static or stationary electromagnetic field generators are often set such that their working volume is centered on the primary medical site (e.g., the kidney). In this position, the working volume may not be extensive enough to encompass insertion and navigation through the ureter. Thus, in systems that use static or stationary electromagnetic field generators, the operator may not be able to utilize EM-based navigation until the operator navigates the instrument into the working volume of the electromagnetic field generator. Also, moving the static or stationary electromagnetic field generator to follow the instrument during insertion and navigation of the instrument into the medical site is generally not possible, as it would require re-performing the alignment steps each time the electromagnetic field generator is moved, which is impractical and could cause delays during the medical procedure and potentially adversely affect the outcome of the procedure.

[0145] However, as shown in Figures 27A and 27B, a robotically controlled EM field generator 302 can easily track the position of the ureteroscope 502 from insertion into the kidney through operation, as the robotic arm 212 can reposition the EM field generator 302 as needed, and the kinematics of the robotic arm 212 can be used to provide continuous alignment between the EM coordinate frame and the robot coordinate frame.

[0146] An additional benefit that may be achieved using the automatic instrument tracking shown in Figures 27A and 27B is improved anatomical mapping. As the instrument is navigated through the patient's anatomy, the position of the EM sensor on the instrument may be used to build an anatomical map. For example, in the case of bronchoscopy, as the bronchoscope containing the EM sensor is navigated through the patient's airways, the determined position of the EM sensor may be used to generate a map of the patient's lungs. With static or stationary EM field generators, this is only possible for areas within the working volume of the EM field generator. However, using a robotically controllable EM field generator 302 that allows the robotic arm 212 to reposition the EM field generator 302 enhances the ability to generate an anatomical map.

[0147] Using the example of Figures 27A and 27B, the position of the EM sensor 204 can be determined from its insertion (Figure 27A) into the kidney (Figure 27B) to generate a near-complete anatomical map of the patient's urinary tract. In contrast, if a static electromagnetic field generator is used with its working volume centered on the kidney, EM sensor data that can be used to map the patient's ureters may not be available. Thus, the use of a robotically controllable EM field generator 302 as described herein can expand the ability to generate anatomical mapping using EM sensor data, resulting in a more complete anatomical map.

[0148] FIG. 29 is a block diagram depicting a robotic medical system 300, such as the example shown in FIG. 22A, including a robotically controllable EM field generator 302 mounted on a robotic arm 212. As shown in FIG. 29 (and also in FIG. 23), the system 300 may include one or more robotic arms 212. In the illustrated embodiment, the system 300 includes a first robotic arm 212 and a second robotic arm 212, although in other embodiments other numbers of robotic arms (e.g., one, two, three, four, five, six, or more) may be included. As shown, the first robotic arm 212 is coupled to the EM field generator 302. As discussed above, the EM field generator 302 may be attached or coupled to the first robotic arm 212, or may be directly incorporated into the first robotic arm 212 itself. The EM field generator 302 is configured to generate an EM field having a working volume 306 in which the position of the EM sensor 204 can be determined relative to an EM coordinate frame 308. The first robotic arm 212 can be configured to adjust the position of the EM field generator 302. For example, the first robotic arm 212 can move (e.g., change its pose or shape) to readjust the position of the EM field generator 302 and, accordingly, the position of the EM field working volume 306.

[0149] The first robotic arm 212 (as well as the other robotic arms 212 in the system) is associated with a robot coordinate frame 216 as described above. The position of the EM field generator 302 within the robot coordinate frame 216 can be determined based on the kinematic position of the arm 212. This allows an alignment or mapping between the EM coordinate frame 308 and the robot coordinate frame 216 to be determined as described above. As shown in FIG. 29 (and also in FIG. 23), the system 300 can include one or more EM sensors 204. The position of the EM sensor 204 can be determined relative to the EM coordinate frame 308 and mapped to the robot coordinate frame 216 using the alignment described above.

[0150] 29 (and FIG. 23 ), one of the EM sensors 204 is positioned on a medical instrument 214. The medical instrument 214 may, in some embodiments, be coupled to a second robotic arm 212. This may allow the second robotic arm 212 to manipulate and control the medical instrument 214. As the second robotic arm 212 manipulates and controls the medical instrument 214, the position of the medical instrument 214 may be determined based on the determined position of the EM sensor 204. As shown in FIG. 29 , the system 300 may include additional EM sensors 204 that may be tracked in a similar manner.

[0151] 29 block diagram further illustrates that the system 300 can include a processor 380 and a memory 382. The memory 382 can be used to store instructions that, when executed by the processor 380, can enable various functionalities of the system 300, such as determining and using alignment and mapping from the EM coordinate frame 308 to the robot coordinate frame 216, configuring the EM field generator 302, and tracking the medical instrument 214, as described in this section and throughout this application. For example, the processor 380 can be in communication with the first robotic arm 212 and the EM field generator 302, and can be configured to determine a position of one or more of the EM sensors 204 within the EM field generated by the EM field generator 302, and adjust the position of the EM field generator 302 by commanding movement of the first robotic arm 212 based on the determined position of the EM sensor 204. The commanded movement of the first robotic arm 212 based on the determined position of the EM sensor 204 can be used to set the position of the EM field generator 302 or to track the position of the EM sensor 204 as it moves during a procedure.

[0152] In some embodiments, the processor 302 is configured to first determine a position of the EM sensor 204 relative to the EM coordinate frame 308 and then map that position to the robot coordinate frame 216. As described above, this may be accomplished, for example, by (i) determining a position of the EM sensor 204 in the EM field relative to the EM coordinate frame 308 associated with the EM field generator 302, (ii) determining an alignment between the EM coordinate frame 308 and the robot coordinate frame 216 associated with the first robot arm 212 based on determining the position of the EM field generator in the robot coordinate frame 216, and (iii) determining a position of the EM sensor 204 in the robot coordinate frame 216 based on the alignment. The determination of the position of the EM field generator 302 in the robot coordinate frame 216 in step (ii) may be based on the kinematics of the first robot arm 212.

[0153] 30A-33D show various examples or approaches to moving the EM field generator 302 using the first robotic arm 204 based on the determined position of the EM sensor 204 to facilitate electromagnetic field generator setting and instrument tracking.

[0154] 30A and 30B are perspective views illustrating an example of using a robotic arm to move an EM field generator such that an EM sensor is positioned at a predetermined location within the EM field. This can be done, for example, during setup of the electromagnetic field generator and can be advantageously used to position the EM field generator 302 relative to the EM sensor 204. In some embodiments, it can be advantageous to set up or align the EM field generator 302 such that the EM sensor 204 is located at a predetermined location with the working volume 306 of the EM field. This can be useful for a number of reasons. For example, in some cases, this can center the working volume 306 of the EM field around the EM sensor 204. This can help ensure that if the EM sensor 204 moves, it can still be tracked since it can move without moving outside of the working volume 306. As another example, this can increase the accuracy of the determined location of the EM sensor 204 because in some examples, the ability to determine the location of the EM sensor 204 within the EM field can be more accurate toward the center of the working volume 306 than on the edges of the working volume 306. Thus, in some embodiments, the predetermined position comprises the center of the working volume 306, and the EM field generator 302 is moved such that the EM sensor 204 is positioned at the center of the working volume 306, as shown in Figures 30A and 30B.

[0155] In Figures 30A and 30B, the predetermined location 384 is the center of the working volume 306 and is represented in the figures as x. Figure 30A shows the system in a first state before the EM field generator 302 is moved such that the EM sensor 204 is positioned at the predetermined location 384. As shown in Figure 30A, the EM sensor 204 is positioned within the working volume 306 of the EM field generator 302 but is not located at the predetermined location 384. To position the EM sensor 204 at the predetermined location 384, the EM field generator 302 must be moved in the direction of the illustrated arrow, which causes a corresponding movement of the working volume 306 and the predetermined location 384. Figure 30B shows the system in a second state after the EM sensor 204 has been positioned at the predetermined location 384 within the EM field by adjusting the position of the EM field generator 302 based on the determined position of the EM sensor 204. In this example, the EM field generator 302 may be considered a "set" here since its position has been adjusted relative to the EM sensor 204. Notably, movement of the EM field generator 302 may be accomplished by the robot arm 212 to which the EM field generator 302 is attached. Furthermore, points in the EM coordinate frame, such as the position of the EM sensor 204 and the predetermined position 384, may be mapped to the robot coordinate frame using an alignment based on the kinematics of the robot arm such that all positions (e.g., the EM sensor position, the predetermined position 384, and the position of the EM field generator 302) may be processed within a single space (e.g., the robot coordinate frame).

[0156] 30A and 30B, the predetermined location 384 within the EM field working volume 306 includes the center of the working volume 306. However, this need not be the case in all embodiments and other predetermined locations 384 within the working volume 306 can be used. For example, another possible predetermined location 384 may include a location located at one extreme end of the working volume 306 such that the EM sensor 204 can be moved completely across the working volume 306 before the EM field generator 302 needs to be repositioned to continue to track the EM sensor 204.

[0157] In some embodiments, the EM field generator 302 may be positioned such that the EM sensor 204 is positioned within a predetermined region or portion of the working volume 306 of the EM field, rather than at a particular predetermined location. FIGS. 31A and 31B are perspective views illustrating an example of moving the EM field generator 302 using the robot arm 212 such that the EM sensor 204 is positioned within a predetermined region 386 within the working volume 306. In FIGS. 31A and 31B, a predetermined region 386 is shown that represents a subsection of the working volume 386. In the figures, both the working volume 306 and the predetermined region 386 are represented as a right angle prism. However, this is only one example, and other shapes of the working volume and the predetermined region 386 are possible. Furthermore, the shape of the predetermined region 386 need not correspond to the shape of the working volume 306. For example, the working volume 306 may include the shape of a right angle prism, and the predetermined region 386 may include the shape of a sphere within the working volume 306. In some embodiments, the predetermined area 386 may represent a portion of the working volume 306 where the position of the EM sensor 204 may be determined with greater accuracy, and therefore it may be desirable to position the EM field generator 302 relative to the EM sensor 204 such that the EM sensor 204 is within the predetermined area 386. In some embodiments, the predetermined area 386 may include the entirety of the working volume 306.

[0158] FIG. 31A illustrates the system in a first state, prior to adjusting the position of the EM field generator 302 such that the EM sensor 204 is positioned within the predetermined region 386. As shown, the EM sensor 204 is positioned within the working volume 306 of the EM field, but not within the predetermined region 386. To position the working volume 306 such that the EM sensor 204 is positioned within the predetermined region 386, the EM field generator 302 can be moved in the direction of the illustrated arrow using the robotic arm 212 to which it is attached. This movement causes a corresponding movement of the working volume 306 and the predetermined region 386. FIG. 31B illustrates the system in a second state, after adjusting the position of the EM field generator 302 such that the EM sensor 204 is positioned within the predetermined region 386. As shown, the EM field generator 302 has been moved using the robotic arm 212 such that the EM sensor 204 is now positioned within the predetermined region 386. As mentioned above, the above mentioned registration can map points in the EM coordinate frame to corresponding points in the robot coordinate frame based on the kinematics of the arm so that all positions can be represented in a single space.

[0159] 31A-31B and 32A-32B include adjusting the position of the EM field generator 302 relative to the EM sensor 204. In some embodiments, the system may additionally or alternatively adjust at least one of the orientation and position of the EM field generator 302 by determining an orientation of the EM sensor 204 within the EM field and commanding movement of the first robot arm 212 based on the determined orientation of the EM sensor 204. In certain examples, the accuracy with which the position of the EM sensor 204 may be determined within the working volume 306 may be influenced or affected by the orientation of the EM sensor 204 relative to the EM field generator 302. Thus, in some embodiments, the processor 380 is configured to adjust at least one of the orientation and position of the EM field generator based on the determined orientation of the EM sensor to increase accuracy of the determined position of the EM sensor 204 within the EM field. This may include, for example, adjusting the pitch, yaw, and / or roll of the EM field generator 302 relative to the EM sensor 204 using the robotic arm 212 to which the EM field generator 302 is attached.

[0160] As discussed above, the examples of FIGS. 31A-31B and 32A-32B generally relate to determining a position of the EM field generator 302 relative to a determined position of the EM sensor 204 in order to configure the EM field generator 302 for a procedure. This configuration is facilitated by mounting the EM field generator 302 on the robotic arm 212 such that its position can be adjusted by the robot. Robotic adjustment of the position of the EM field generator 302 also facilitates instrument tracking, which may occur when one or more of the EM sensors 204 move during a procedure. As discussed above, one or more of the EM sensors 204 may be positioned on medical tools or instruments that are moved during a procedure. The EM sensors 204 may enable tracking of the movement of these tools. As described in the examples below, when movement of the EM sensor 204 is detected, a new position of the EM field generator 302 can be determined and the EM field generator 302 can be moved using the robotic arm 212 to which it is attached.

[0161] 32A and 32B are perspective views illustrating an example of using the robotic arm 212 to move the EM field generator 302 along a path 390 that tracks the path 388 of movement of the EM sensor 204. In this manner, the path 390 of the electromagnetic field generator 302 tracks, follows, or mirrors the path 388 of the EM sensor 204. In the example of FIGS. 32A and 32B, the system is configured to maintain the position of the EM field generator 302 such that the EM sensor 204 remains positioned at a predetermined location 384 (in the illustrated example, the center of the working volume 306) as the EM sensor 204 moves along the path 388.

[0162] During a medical procedure, a medical instrument including an EM sensor 204 positioned thereon (such as the medical instrument 214 of FIGS. 23 and 29 ) may be navigated through a patient's anatomy. This movement of the EM sensor 204 (and corresponding instrument 214) is represented by a path 388. FIG. 32A illustrates a system with the EM sensor 204 positioned at a first location along the path 388. If the EM sensor 204 continues to travel along the path 388 while the EM field generator 302 remains stationary, the EM sensor 204 will no longer be positioned at the predetermined location 384 within the working volume 306. In order to maintain the EM sensor 204 at the predetermined location 384 within the working volume 306, the EM field generator 302 must move along a corresponding path 390. In this manner, the EM field generator 302 tracks or follows the movement of the EM sensor 204. FIG. 32B illustrates the system after the EM sensor 204 has moved along its path 388. The EM sensor 204 remains positioned at a predetermined location 384 within the working volume 306 as the EM field generator 302 is also moving along a corresponding path 390. This is made possible by moving the EM field generator 302 along with the robotic arm 212 to which it is attached.

[0163] In some embodiments, the tracking of the EM sensor 204 by the EM field generator 302 need not have a direct correspondence. For example, the path 390 of the EM field generator 302 need not directly correspond to the path 388 of the EM sensor 204. Figures 33A-33D show an example.

[0164] 33A-33D are perspective views illustrating an example of moving the EM field generator 302 using the robotic arm 212 such that the EM sensor 204, moving along a path 388, remains positioned within the predetermined region 386 of the working volume 306. As shown, in this example, the EM field generator 302 tracks the movement of the EM sensor 204, but does not move along a path that directly corresponds to the path 388 of the EM sensor 204. In this example, the system is configured to adjust the position of the EM field generator 302 when the EM sensor 204 is moved to a boundary or outside of the predetermined region 386 of the working volume 306. For example, the processor 380 may be configured to command the movement of the EM field generator 302 using the first robotic arm 212 such that the EM sensor 204 remains positioned within the predetermined region 386 of the EM field working volume 306 during movement of the medical instrument 214 including the EM sensor 204.

[0165] FIG. 33A illustrates the system in a first state. As shown, the EM sensor 204 is positioned within the predetermined region 386 of the working volume 306 and travels along a path 388. FIG. 33B illustrates the system in a second state. As shown, the EM sensor 204 continues to travel along the path 204, but is still positioned within the predetermined region 386. Since the EM sensor 204 is still positioned within the predetermined region 386, the EM field generator 302 has not yet moved. FIG. 33C illustrates a third state in which the EM sensor 204 has reached an edge or boundary of the predetermined region 386. If the EM sensor 204 continues to travel along the path 388 while the EM field generator 302 remains stationary, the EM sensor 204 will move outside of the predetermined region 386. In order to keep the EM sensor 204 within the predetermined region 386, the EM field generator 302 needs to move in the direction of the arrow shown. This movement may be determined, for example, by the processor 380, which may instruct the robotic arm 212 to move the EM field generator 302. Figure 33D illustrates the system in a fourth state after the EM field generator 302 has been moved. As shown, the movement may reposition the EM field generator 302 such that the EM sensor 204 is repositioned within the predetermined region 386 even as the EM sensor 204 continues to progress along the path 388. In this manner, the EM field generator 302 again moves to track or follow the movement of the EM sensor 204 even though the movement of the EM field generator 302 does not directly correspond to the movement of the EM sensor 204, as in the example of Figures 32A and 32B.

[0166] 34 is a flow chart providing an example method 400 for moving an EM field generator coupled to a robot arm based on a determined position of an EM sensor within an EM field. The method 400 can be executed, for example, by the processor 380 to provide electromagnetic field generator configuration and instrument tracking. The method 400 begins at block 401 with determining a position of an EM sensor of a medical instrument within a working volume of an EM field generated by an EM field generator coupled to a first robot arm. The position of the EM sensor can be determined relative to an EM coordinate frame associated with the EM field generator. In some embodiments, the position of the EM sensor can be further determined relative to a robot coordinate frame associated with the first robot arm by applying an alignment determined based on the kinematics of the robot arm as described above.

[0167] The method 400 includes a block 403 in which the medical instrument is moved. The movement of the medical instrument causes a corresponding movement of an EM sensor positioned thereon. In some embodiments, the medical instrument comprises a manually controlled instrument, and the movement of the medical instrument is accomplished manually. In other embodiments, the medical instrument includes a robotically controllable instrument. For example, the medical instrument can be coupled to a second robotic arm, and moving the medical instrument can include moving the instrument with the second robotic arm. In some embodiments, moving the instrument with the second robotic arm includes articulating the second robotic arm. The medical instrument can be coupled to an instrument drive mechanism, and moving the medical instrument with the second robotic arm can include actuating the medical instrument with the instrument drive mechanism. The movement of the instrument can be detected by determining that a position of an EM sensor attached to the instrument has moved.

[0168] The method 400 includes block 405, which includes instructing a first robotic arm to move the EM field generator in response to movement of the medical instrument such that the EM sensor remains positioned within a working volume of the EM field generator. For example, if movement of the medical instrument causes the EM sensor to move outside of the working volume, the EM field generator can be repositioned using the first robotic arm to maintain the EM sensor within the working volume of the EM field.

[0169] In some embodiments, commanding the first robotic arm to move the EM field generator causes the EM field generator to track the movement of the medical instrument. For example, commanding the first robotic arm to move the EM field generator may include moving the EM field generator such that an EM sensor is positioned or remains positioned within a predetermined region of the working volume of the EM field, as shown in the examples of Figures 31A-31B and 33A-33D. As another example, commanding the first robotic arm to move the EM field generator may include moving the EM field generator such that an EM sensor is positioned or remains positioned at a predetermined location within the working volume of the EM field, as shown in the examples of Figures 30A-30B and 32A-32B.

[0170] In some embodiments, method 400 may optionally include determining an orientation of the EM sensor within the EM field and adjusting at least one of an orientation and a position of the EM field generator by commanding movement of the first robotic arm based on the determined orientation of the EM sensor. As described above, adjusting at least one of the orientation and the position of the EM field can increase accuracy of the determined position of the EM sensor within the working volume of the EM field.

[0171] Additionally, method 400 may be performed as a loop, as indicated by dashed line 407, to continually track the EM sensor and adjust the position of the EM field generator accordingly. That is, at multiple discrete time steps, the position of the EM sensor may be determined and the EM field generator may be readjusted based on each newly determined position.

[0172] The examples provided above illustrate that a robotic system such as the system 300 shown in FIG. 23 and FIG. 29 may be configured to move the EM field generator 302 using the robot arm 312 to set the EM field generator 302 relative to one EM sensor 204 and / or track the movement of one EM sensor 204. However, the system 300 is not limited to an embodiment having one EM sensor 204. For example, the processor 380 may be configured to determine the positions of multiple EM sensors 204 within the EM field, determine a generator position of the EM field generator 302 based on the determined positions of the multiple EM sensors 204, and command the first robotic arm 212 to move the EM field generator 302 to the generator position. That is, the system 300 may be configured to set and track the EM field generator based on the multiple EM sensors 204. Examples of use based on multiple EM sensors 204 are provided with respect to FIG. 35A-FIG. 37.

[0173] 35A and 35B are perspective views illustrating an example of using the robotic arm 212 to move the EM field generator 302 to an electromagnetic field generator location based on the determined positions of the multiple EM sensors 204 within the EM field. In some ways similar to the previous example, this can be done during setup of the electromagnetic field generator and can be advantageously used to position the EM field generator 302 relative to the multiple EM sensors 204.

[0174] 35A illustrates the system in a first state. As shown, in the first state, the plurality of EM sensors 204 are positioned within the working volume 306 of the EM field generator 302. However, as shown, the plurality of EM sensors 204 are not centered within the working volume 306. To center the plurality of EM sensors 204 within the working volume 306, an electromagnetic field generator position 392 can be determined. The electromagnetic field generator position 392 can represent a position to which the EM field generator 302 may be moved to center the plurality of EM sensors 204 within the working volume. In the illustrated embodiment, the generator position 392 is represented as x. To reach the generator position 392, the EM field generator 302 must move in the direction of the illustrated arrow, which causes a corresponding movement of the working volume 306.

[0175] The generator position 392 can be determined based on the determined positions of the multiple EM sensors 302. In one example, the processor 380 is configured to determine the generator position 392 based on determining a centroid of the determined positions of the multiple EM sensors 204. The generator position 392 can then be determined by finding a position to which the EM field generator 302 is moved such that the centroid of the positions of the multiple EM sensors 204 is positioned at a predetermined position (such as a center or other desired position) within the working volume 306. As described above, the determined positions of the multiple EM sensors can be mapped to the robot coordinate frame using the aforementioned alignment based on the kinematics of the robot arm 212 to which the EM field generator 302 is attached.

[0176] 35B illustrates the system in a second state after adjusting the position of the EM field generator 302 to generator position 392. As shown, the multiple EM sensors 204 are now positioned closer to the center of the working volume 206. Thus, the system can facilitate setting up and positioning the EM field generator 302 relative to the multiple EM sensors 204. Although this example describes "centering" the multiple EM sensors 204 within the working volume 306, the generator position 392 may be determined to position the multiple EM sensors elsewhere within the working volume 306.

[0177] 36A, 36B, and 36C are perspective views illustrating an example of using a robotic arm 212 to realign an electromagnetic field generator position 392 of an EM field generator 302 based on determined positions of a plurality of EM sensors 204 within a working volume 306, where at least one of the plurality of EM sensors 204 has moved. For example, the processor 380 may be configured to detect movement of at least one of the EM sensors 204, determine a new generator position 392 of the EM field generator 302 based on the detected movement, and command the first robotic arm 212 to move the EM field generator 302 to the new generator position 392.

[0178] 36A, one of the EM sensors 204 is moving along a path 388. It may be desirable to adjust the position of the electromagnetic field generator 302 based on this movement. For example, if not adjusted, the EM sensor 204 may move along the path to a location outside the working volume 306, at which point tracking of the EM sensor 204 is lost.

[0179] Thus, as shown in FIG. 36B, a new electromagnetic field generator position 392 may be determined based on the positions of the multiple EM sensors 204 and taking into account the movement of one EM sensor 204 along the path 388. The new electromagnetic field generator position 392 may be based, for example, on the current center of gravity of the positions of the multiple EM sensors 204. As shown, the EM field generator 302 needs to move in the direction of the illustrated arrow to reach the new generator position 392. FIG. 36C shows the system after movement to the new electromagnetic field generator position 392. As shown, the positions of the multiple EM sensors 204 are now again centered within the working volume 306. This may advantageously allow the position of the EM field generator 302 to be continually adjusted to optimize the position of the working volume 306 such that the positions of the multiple EM sensors 204 remain positioned within the working volume 306 for as long as possible, even as they move away from one another.

[0180] 37 is a flow chart illustrating an example method 410 for determining an electromagnetic field generator position of an EM field generator mounted on a robotic arm based on determined positions of a plurality of EM sensors within an EM field. The method 410 begins at block 411 with generating an EM field with an EM field generator coupled to a first robotic arm.

[0181] At block 413 of method 410, an embodiment may determine positions of the multiple EM sensors within the EM field. This may be accomplished by determining positions of the multiple EM sensors within the EM field relative to an EM coordinate frame associated with the EM field generator, determining an alignment between the EM coordinate frame and a robot coordinate frame associated with the first robot arm based on determining the position of the EM field generator in the robot coordinate frame, and determining positions of the multiple EM sensors in the robot coordinate frame based on the alignment. Determining the position of the EM field generator in the robot coordinate frame may be based on the kinematics of the first robot arm.

[0182] At block 415, the method 410 includes determining a generator position of the EM field generator based on the determined positions of the multiple EM sensors. Determining the generator position may be based on determining a centroid of the positions of the multiple EM sensors 204. The EM field generator position may be a position to which the EM field generator may be moved that will desirably position the multiple EM sensors within the working volume of the EM field.

[0183] Block 417 includes commanding the first robotic arm to move the EM field generator to a generator position. For example, the system can use the first robotic arm to move the EM field generator to the generator position. As the EM field generator is moved, the position of the working volume is correspondingly repositioned relative to the position of the EM sensor.

[0184] As indicated by dashed line 419, method 410 may be performed as a loop to continuously adjust the position of the EM field generator, as described above with reference to Figures 36A-36C. For example, method 410 may also include detecting movement of at least one of the multiple EM sensors, determining a new generator position for the EM field generator based on the detected movement, and commanding a first robotic arm to move the EM field generator to the new generator position.

[0185] ii. Electromagnetic field generator settings and instrument tracking with extended working volume In the previous section, a robotic medical system was described that includes a robotically controllable electromagnetic field generator that can be moved with the robotic arm of the system to facilitate setting of the electromagnetic field generator and tracking of the instrument. In some of the examples, it was shown that the EM field generator can be moved with the robotic arm to track the EM sensor, e.g., to keep the moving EM sensor within the working volume of the EM field generated by the EM field generator. By moving the EM field generator, and correspondingly the working volume of the EM field, the system described herein can be considered to have an "extended" working volume. That is, by repositioning the EM field generator and the working volume, the position of the EM sensor can be detected and determined over an area larger than the working volume of a stationary electromagnetic field generator.

[0186] This section provides additional examples to elaborate on the concept of "extending" the working volume of an EM field generator by moving the EM field generator with a robot arm. In particular, the examples show, among other things, how EM sensors that are too far away to simultaneously fit within the working volume of the EM field generator can be tracked by moving the EM field generator, detecting the position of the EM sensor, and mapping the detected position of the EM sensor into the robot coordinate frame. This provides a significant advantage over conventional systems utilizing stationary electromagnetic field generators that may "lose track" of the EM sensor if the EM sensor moves outside the stationary working volume. The examples in this section refer to the exemplary system 300 of FIG. 23 and FIG. 29, but may be implemented with other systems that include a robot-controllable electromagnetic field generator.

[0187] A first example will now be described with reference to Figures 38A-38C, which show the system 300 at different stages, to illustrate how the system 300 expands the working volume 306 of the EM field generator 302. As shown in Figure 38A, which shows some components of the system at a first stage, the system includes an EM field generator 302 configured to generate an EM field having a working volume 306 within which the position of the EM sensor 204 can be determined relative to an EM coordinate frame 308, as previously described. The EM field generator 302 is coupled to a robotic arm 212, which is configured to move to adjust the position of the EM field generator 302. As the robotic arm 212 moves the EM field generator 302, the position of the working volume 306 of the EM field generator 302 also moves.

[0188] 38A also shows two EM sensors 204A and 204B. As shown, in a first stage, neither of the two EM sensors 204A, 204B is positioned within the working volume 306 of the electromagnetic field generator 302, and thus the positions of the EM sensors 204A, 204B are determinable in the first stage. Furthermore, the two EM sensors 204A, 204B are positioned far enough apart that they cannot both be positioned within the working volume 306 at the same time. Thus, as shown in FIGS. 38B and 38C, in order to track the positions of the EM sensors 204A, 204B, the EM field generator 302 can be moved using the robot arm 212 to different positions where each of the EM sensors 204A, 204B can be determined and mapped to the robot coordinate frame 216 to "expand" the working volume 306 of the EM field generator.

[0189] For example, a processor 380 of the system 300 in communication with the EM field generator 302 and the robotic arm 212 to which the EM field generator 302 is attached can be configured to move the EM field generator 302 to a first position where the EM sensor 204 is positioned within the working volume 306. With the EM field generator 302 in the first position, the processor 380 can determine a position of the first EM sensor 204A within the working volume 306 relative to the EM coordinate frame 308 and map the position of the first EM sensor 204A to the robot coordinate frame 216 based on the kinematics of the robot arm with the EM field generator 302 in the first position. The processor 380 can then cause the robotic arm 212 to move the EM field generator 302 to a second position where the second EM sensor 204B is positioned within the working volume. Also, with the EM field generator 302 in the second position, the processor 380 can determine a position of the second EM sensor 204B in the working volume 306 relative to the EM coordinate frame 308 and map the position of the second EM sensor 204B to the robot coordinate frame 216 based on the kinematics of the robot arm 212 with the EM field generator 302 in the second position. In this manner, the positions of both EM sensors 204A, 204B can be presented in the robot coordinate frame 216.

[0190] 38B shows the system in an exemplary second phase, where the EM field generator 302 has been moved along with the robot arm 212 to a first position where the first EM sensor 204A is positioned within the working volume 306. With the EM field generator 302 in this position, the position of the EM sensor 204A can be determined relative to the EM coordinate frame 308, which can then be mapped to the robot coordinate frame 216 using the kinematics-based alignment of the robot arm 212 described above. In FIG. 38B, the EM sensor 204A is shaded black to indicate that its position was initially determined in the EM coordinate frame 308, but has been mapped to the robot coordinate frame 216.

[0191] 38C shows the system in an exemplary third phase, where the EM field generator 302 has been moved along with the robot arm 212 to a second position where the second EM sensor 204B is positioned within the working volume 306. With the EM field generator 302 in this position, the position of the EM sensor 204B can be determined relative to the EM coordinate frame 308, which can then be mapped to the robot coordinate frame 216 using the kinematics-based alignment of the robot arm 212 described above. Again, the EM sensor 204B is shaded black to indicate that its position has been mapped to the robot coordinate frame 216. The EM sensor 204A is also still shaded black to indicate its position within the robot coordinate frame 216.

[0192] 38C, it can be seen that the EM sensors 204A, 204B are spaced far enough apart that they cannot both fit within the working volume 306 of the EM field generator 302 at the same time, but the positions of both EM sensors 204A, 204B may be represented simultaneously or substantially contemporaneously within the robot coordinate frame 216. In some embodiments, the processor 380 is further configured to frequently (e.g., at about 40 Hz, or at both smaller and larger time steps) update and track the positions of the first EM sensor 204A and the second EM sensor 204B as the EM field generator 302 moves back and forth between the first and second positions.

[0193] Further, the examples described with reference to FIGS. 38A-38C are presented to simplistically illustrate the EM field generator 302 in a first position (FIG. 38B) and a second position (FIG. 38C). In practice, the system may be configured to continuously or near-continuously detect any EM sensors 204 positioned within the EM field and map their positions to the robot coordinate frame 216. For example, the processor 380 may be configured to move the EM field generator 302 using the robot arm 212, and when an EM sensor is detected within the working volume 306, the processor 380 may be configured to (i) determine a position of the EM sensor 204 within the working volume 306 relative to the EM coordinate frame 308, and (ii) map the position of the EM sensor 204 to the robot coordinate frame 216 based on the kinematic pose of the first robot arm 212.

[0194] In some examples, the initial position of the EM sensor 204 may not be known. To identify the location of the EM sensor 204, the system can be configured to move the EM field generator 302 along a search path or trajectory (e.g., along a predetermined path / pattern or range of motion) using the robotic arm 212. The search path can be configured to sweep the working volume 306 through a treatment volume that is larger than the working volume 306 and identify the location of the EM sensor within the treatment volume. By sweeping through the treatment volume, the initial position of any EM sensor 204 within the treatment volume can be determined.

[0195] 39A-39D illustrate an embodiment of a robotic medical system including a robotically controllable electromagnetic field generator 302 that can be moved with the robot arm 212 along a search path 510 to detect the location of the EM sensor 204. In the example of FIG. 39A, the initial positions of the two EM sensors 204A, 204B are not known. To identify the locations of the EM sensors 204A, 204B, FIG. 39A illustrates an exemplary search path 510 along which the EM field generator 302 moves using the robot arm 212. In the illustrated embodiment, the search path 510 comprises a generally sinusoidal path configured for the EM sensor 204 to sweep the working volume 306 across the anticipated treatment volume or treatment site. The search path 510 illustrated in FIG. 39A is provided as an example, and any number of paths including different shapes are possible.

[0196] FIG. 39B shows the system in a second phase as the EM field generator 302 moves along the search path 510. In the second phase illustrated in FIG. 39B, the first EM sensor 204A is now positioned and detectable within the working volume 306 of the EM field generator 302. The position of the EM sensor 204A can be determined relative to the EM coordinate frame 308, which can then be mapped to the robot coordinate frame 216 using the kinematics-based alignment of the robot arm 212 described above. In FIG. 39B, the EM sensor 204A is shaded black to indicate that its position was initially determined in the EM coordinate frame 308, but has been mapped to the robot coordinate frame 216. The EM field generator 302 can proceed along the search path 510.

[0197] FIG. 39C shows the system in a third phase as the EM field generator 302 moves further along the search path 510. In the third phase illustrated in FIG. 39C, the second EM sensor 204B is now positioned and detectable within the working volume 306 of the EM field generator 302. The position of the EM sensor 204B can be determined relative to the EM coordinate frame 308, which can then be mapped to the robot coordinate frame 216 using the kinematics-based alignment of the robot arm 212 described above. In FIG. 39B, the EM sensor 204B is shaded black to indicate that it has been mapped to the robot coordinate frame 216. The EM field generator 302 can proceed along the search path 510. The position of the EM sensor 204A is also shown shaded black to indicate that it has been mapped to the robot coordinate frame.

[0198] 39D illustrates the system in a fourth stage where the EM field generator 302 has completed a search path 510. As shown, all EM sensors 204 (in the illustrated example, EM sensors 204A, 204B) have been located and mapped to the robot coordinate frame 216. In some embodiments, the system is configured to follow the search path 510 at the start of a procedure to locate the EM sensors 204. The search path 510 can be repeated periodically to determine if any new EM sensors 204 have been introduced.

[0199] In some embodiments, the initial position of the EM sensor 204 may be determined in another manner. For example, if the EM sensor 204 is included on a robotically controllable instrument, the system may estimate or determine the initial position of the EM sensor 204 based on the kinematics of the robot. For an EM sensor 204 that is not coupled to a robotic component of the system, in some embodiments, a user may indicate or input the initial position of the EM sensor 204 to the system (e.g., via a user input device, controller, graphical user interface, etc.). For example, in the case of an EM patch sensor placed on a patient's chest to track respiration, a user placing the EM patch sensor may indicate to the system where the patch sensor was placed.

[0200] As mentioned above, in some examples, the position of the EM sensor 204 mapped to the robot coordinate frame 216 may not represent the live position of the sensor. Thus, after the initial position of the EM sensor is determined, the system may determine a new tracking path along which the EM field generator 302 may move to continue tracking the position of the EM sensor 204. The tracking path may be shorter than the search path, for example, which may allow for faster or more frequent remapping of the position of the EM sensor 204. An example is described with reference to FIGS. 40A-40D.

[0201] 40A-40D continue the example of FIGS. 39A-39D and further illustrate an exemplary tracking path 512 for the EM field generator 302. The tracking path 512 can be determined based on a determined position of the EM sensor 204 in the robot coordinate frame 216. For example, the tracking path 512 can be determined such that the EM field generator 302 is moved back and forth between determined positions of the EM sensor 204 in the robot coordinate frame 216 such that the position can be frequently remapped and frequently updated. For example, as the EM field generator 302 is moved along the tracking path using the robot arm 212, the processor 380 can be configured to (i) re-determine a position of the EM sensor 204 in the working volume 306 relative to the EM coordinate frame 216, (ii) remap the position of the EM sensor 204 to the robot coordinate frame 308 based on the kinematic pose of the robot arm 212, and (iii) determine an updated tracking path 510.

[0202] As shown in FIG. 40A, the tracking path 512 can be shorter than the search path 510 (of FIGS. 39A-39D). This can be advantageous because it can limit unnecessary movement of the EM field generator (which can cause collisions with other objects in space) and limits the time that the EM sensor is not positioned within the working volume 306. In the illustrated embodiment, the tracking path 512 includes a line that moves the EM field generator 302 back and forth between positions where each of the EM sensors 204A, 204B can be detected within the working volume 306 and remapped to the robot coordinate frame 216. Other shapes of the tracking path 512 are possible.

[0203] 40A shows the EM field generator 302 positioned at a first end of the tracking path 512 where the first EM sensor 204A is positioned within the working volume 306. In this position, the position of the first EM sensor 204A can be detected relative to the EM coordinate frame 308 and remapped to the robot coordinate frame 216. FIG. 40B shows the EM field generator 302 positioned at a second end of the tracking path 512 where the second EM sensor 204B is positioned within the working volume 306. In this position, the position of the second EM sensor 204B can be detected relative to the EM coordinate frame 308 and remapped to the robot coordinate frame 216. In Figures 40A and 40B, the EM sensors 204A, 204B are shaded black to indicate that their positions have been mapped into the robot coordinate frame 216 (e.g., as detected as the EM field generator 302 moves along the search path 510, as described in the previous example of Figures 39A-39D).

[0204] FIG. 40C illustrates how the location of EM sensor 204A mapped to the robot coordinate frame may not be live. For example, as shown in FIG. 40C, the first EM sensor 204A may be moved to a new location shown as EM sensor 204A' while the EM field generator 302 is positioned at the second end of the tracking path 512 such that the first EM sensor 204A is not positioned within the working volume 306. Notably, the previously mapped location of EM sensor 204A is still shown (shaded in black). In this position, the system has not yet determined that EM sensor 204A has moved to the new location of EM sensor 204A'. In some embodiments, each sensor 204 is connected to a dedicated port of the system so that the system can identify and distinguish the sensors. Other mechanisms and methods for distinguishing the sensors are possible as well.

[0205] However, Figure 40D shows that as the EM field generator 302 moves back along the tracking path 512, a new position of the EM sensor 204A' can be detected. At this position, a new position of the first EM sensor 204A' can be detected relative to the EM coordinate frame 308 and remapped into the robot coordinate frame 216. The EM sensor 204A' is shaded black in Figure 40D to indicate that the new position of the EM sensor 204A' has been remapped into the robot coordinate frame 216.

[0206] 40D also illustrates that a new tracking path 512′ may be determined based on the new position of the EM sensor 204A′. Thus, as the EM sensor 204 moves, the new position of the EM sensor 204 may be remapped into the robot coordinate frame 216 and an updated tracking path 512′ may be determined. This process may be repeated throughout the procedure, beneficially enabling tracking of various EM sensors 204 within an expanded working volume, made possible by moving the EM field generator 302 with the robot arm 212 and mapping the detected EM sensor 204 position into the robot coordinate frame based on the kinematic pose of the robot arm 212.

[0207] FIG. 41A is a flow chart illustrating an example method 600 for expanding a working volume of a robot-controllable electromagnetic field generator. Method 600 may be used, for example, to implement the functions described above with reference to FIGS. 38A-38C. Method 600 begins at block 601 where the EM field generator is moved to a first position. The first position may be a position where an EM sensor is detected within the working volume of the EM field generator. The EM field generator may be moved using a robot arm to which it is attached. In some embodiments, moving the EM field generator to the first position includes moving the EM field generator along a search path or a track path.

[0208] The method proceeds to block 602. In block 602, a position of the first EM sensor is determined relative to an EM coordinate frame. The EM coordinate frame is associated with the EM field of the EM field generator. Then, in block 603, the position of the EM sensor in the EM coordinate frame (determined in block 602) is mapped to a robot coordinate frame. The robot coordinate frame is associated with, for example, a robot arm to which the EM field generator is coupled, and other robotic components of the system. Mapping the position of the EM sensor to the robot coordinate frame can be based on the kinematic pose of the first robot arm, which establishes a relationship between the EM coordinate frame and the robot coordinate frame, as described above, since the EM field generator is coupled to the robot arm.

[0209] Method 600 may then proceed to block 604. In block 604, the EM field generator is moved to a second location where the second EM sensor is detected within the working volume. As described above, the EM field generator may be moved to the second location using the robot arm to which it is attached. In block 605, the position of the second EM sensor may be determined relative to the EM coordinate frame. Then, in block 606, the position of the second EM sensor in the EM coordinate frame (determined in block 605) is mapped to the robot coordinate frame, again using an alignment based on the kinematics of the robot arm. Finally, line 607 indicates that method 600 may be performed as a loop to continuously redetect and remap the positions of the first EM sensor and the second EM sensor to the robot coordinate frame.

[0210] 41B is a flow chart illustrating another exemplary method 610 for expanding the working volume of a robot-controllable electromagnetic field generator. The method 610 begins at block 611 with moving the EM field generator along with the robot arm to which it is attached. In block 612, whenever the EM sensor is detected within the working volume of the EM field generator, the position of the EM sensor may be determined relative to the EM coordinate frame. In block 613, the position of the EM sensor may be mapped to the robot coordinate frame. The mapping may be based on the kinematics of the robot arm, as described above. Finally, line 614 indicates that the method 610 may be implemented as a loop to continuously redetect and remap the positions of the first EM sensor and the second EM sensor to the robot coordinate frame.

[0211] iii. Alignment of percutaneously insertable devices with EM targets The robotically controllable EM field generator 302 can also facilitate alignment of percutaneously inserted tools with other tools endoscopically inserted into the body. For example, as shown in FIG. 27B, a ureteroscope 502 can be endoscopically guided to a location within the kidney. The operator may then wish to perform percutaneous insertion of a needle (or other instrument) or percutaneous insertion with a needle (or other instrument) to rendezvous with the ureteroscope 502. In some embodiments, the EM field generator 302 can include a needle guide (e.g., a channel, tube, or other structure) through which a needle (or other instrument) can be inserted. In other embodiments, the EM field generator 302 can define a space through which a needle guide or equivalent can be advanced and / or positioned. For example, the needle guide can be configured to allow only one degree of freedom (insertion and / or retraction) for a needle inserted therethrough. The EM field generator 302 may be aligned with the EM sensor 204 on the ureteroscope such that the needle is guided directly towards the EM sensor 204 as it is inserted through the needle guide. One such use case is a percutaneous procedure where access to an internal organ or other tissue is made through a needle puncture of the skin. The needle, rigidly attached to the robotic end effector and having a known translation (rigid body definition) relative to the robotic end effector, may thus be robotically aligned with an EM beacon (target) inside the patient's body. The needle may then be inserted through the skin along a robotically aligned trajectory.

[0212] 42A-42C illustrate an embodiment of a robotic medical system, such as the system 300 of FIG. 23 and FIG. 29, having a robotically controllable electromagnetic field generator 302 configured to facilitate alignment of the percutaneously insertable instrument 214 with the EM target 204A. The percutaneously insertable instrument 214 may include, for example, a needle, an access sheath, a laparoscopic instrument, or other type of percutaneously insertable instrument. In the example of FIG. 42A-42C, it may be desirable to precisely insert the percutaneously insertable instrument 214 into a specific location within the body. To facilitate percutaneous insertion, the EM target 204A (which may be an EM sensor) may be positioned within the body at a desired location for the percutaneously insertable instrument 214. In some embodiments, the percutaneously insertable instrument 214 may also include an EM sensor 204B positioned thereon.

[0213] In the illustrated embodiment of FIG. 42A, the percutaneously insertable instrument 214 may be a robotically controllable instrument. The percutaneously insertable instrument 214 is shown attached to a second robotic arm 212B configured to position and insert the percutaneously insertable instrument 214. As shown, the percutaneously insertable instrument 214 extends along an axis 502. In general, it is desirable for the percutaneously insertable instrument 214 to be inserted along the axis 502. However, in order to accurately insert the percutaneously insertable instrument 214, the axis 502 needs to be aligned with the EM target 204A.

[0214] Thus, the system's processor 380 may be configured to determine an alignment that maps a position in an EM coordinate frame associated with the EM field generator 302 to a position in the robot coordinate frame 216 based on the kinematic pose of the first robotic arm 212A to which the EM field generator 302 is attached. The processor 380 may be further configured to determine a position of the EM target 204A in the robot coordinate frame 216 based on the alignment, and based on the position of the EM target 204A in the robot coordinate frame 216, the processor may be configured to move the second robotic arm 212B to align the axis 502 of the percutaneously insertable instrument 214 with the EM target. Finally, the processor 380 may be configured to insert the percutaneously insertable instrument 214 along the axis 502 toward the EM target 204A using the second robotic arm 212B.

[0215] As described above, the alignment may be determined based on the position of the EM field generator 302 in the robot coordinate frame 216, and the position of the EM field generator with the robot coordinate frame 216 may be determined based on the kinematic pose of the first robot arm 212A. The processor 380 may be further configured to determine the position of the EM target 204A in the EM coordinate frame 308 and determine the position of the EM target 204A in the robot coordinate frame 216 by mapping the position of the EM target 204A in the EM coordinate frame 308 to the position of the EM target 204A in the robot coordinate frame 216 using the alignment.

[0216] In some embodiments, the EM target 204A may be an EM sensor positioned on another robotic medical instrument configured to be inserted into a patient, such as an endoscope navigated through the body, which may be coupled to a third robotic arm configured to control the robotic medical instrument.

[0217] Figure 42A shows the system in a first state where the axis 502 of the percutaneously insertable device 214 is not aligned with the EM target 204A. Figure 42B shows the system in a second state after the axis 502 has been aligned with the EM target 204A. Figure 42C shows the system in a third state showing the insertion of the percutaneously insertable device 214 along the axis 502 and rendezvous with the EM target 204A.

[0218] 43A and 43B illustrate an embodiment of a robotic medical system, such as the system 300 of FIGS. 23 and 29, in which a robotically controllable electromagnetic field generator 302 is configured to facilitate alignment of an instrument guide 520 mounted on the electromagnetic field generator 302 with the EM target 204A. The instrument guide 520 may be configured such that a percutaneously insertable instrument may be inserted therethrough along an axis 522 during insertion into a patient. The instrument guide 520 may restrict motion of the percutaneously insertable instrument to only motion along the axis 522 such that when the axis 522 is aligned with the EM target 204, the percutaneously insertable instrument may be guided towards the EM target 204. In some embodiments, the instrument guide 520 is removably coupled to the EM field generator 302. In other embodiments, the instrument guide 520 may be permanently coupled to or integral with the EM field generator 302. It should be noted that in related aspects, the instrument guide 520 may be configured to have different shapes / curves and / or lengths than the example of Figures 43A and 43B.

[0219] Similar to the previous example, the axis 522 of the instrument guide 520 can be aligned with the EM target 204 to guide the percutaneously insertable instrument towards the EM target 20A. For example, the processor 380 can be configured to determine an alignment that maps a position in the EM coordinate frame 308 associated with the EM field generator 308 to a position in the robot coordinate frame 216 based on the kinematic pose of the first robotic arm 212. The processor 380 can also determine a position of the EM target 204 in the robot coordinate frame 216 based on the alignment, and based on the position of the EM target 204 in the robot coordinate frame 216, the processor 280 can cause movement of the first robotic arm 212 to align the insertion axis 522 of the instrument guide 520 with the EM target 204.

[0220] Figure 43A shows the system in a first state where axis 522 of instrument guide 520 is not aligned with EM target 204. Figure 43B shows the system in a second state after axis 522 has been aligned with EM target 204. With the system in the second state, a percutaneously insertable instrument may be inserted through instrument guide 520 along axis 522 towards EM target 204.

[0221] The system may also include a percutaneously insertable instrument. The percutaneously insertable instrument may include a needle, an access sheath, a laparoscopic instrument, or another type of percutaneously insertable instrument. The percutaneously insertable instrument may extend along an axis. In some embodiments, the system includes a second robotic arm coupled to and configured to move the percutaneously insertable instrument. In such a case, the processor may be further configured to use the second robotic arm to align the axis of the percutaneously insertable instrument with the insertion axis and to use the second robotic arm to insert the percutaneously insertable through the instrument guide along the insertion axis toward the EM target 204. In some embodiments, the percutaneously insertable instrument may include an instrument that is manually inserted through the instrument guide 520.

[0222] In some embodiments, the EM target 204 comprises an EM sensor positioned on a medical instrument configured to be inserted into a patient. The robotic medical instrument may be an endoscope. The system may include a third robotic arm coupled to and configured to control the robotic medical instrument.

[0223] 44A and 44B illustrate an embodiment of a robotic medical system having a robotically controllable electromagnetic field generator 302 configured to facilitate alignment of an instrument guide 530 with an EM target 204. This example is similar to the example of FIG. 43A and 43B, except that the instrument guide 530 is coupled to a second robotic arm 212B rather than the EM field generator 302. Similarly, the processor 380 may be configured to determine an alignment that maps a position in the EM coordinate frame 308 associated with the EM field generator 302 to a position in the robot coordinate frame 216 based on the kinematic pose of the first robotic arm 212A to which the EM field generator 302 is attached. The processor 380 may further determine a position of the EM target 204 in the robot coordinate frame based on the alignment, and move the second robotic arm 212B to align the insertion axis 532 of the instrument guide with the EM target 204 based on the position of the EM target in the robot coordinate frame. Figure 44A shows the system before and Figure 44B shows the system after alignment of the axis 532 of the instrument guide 530 with the EM target 204. In the position shown in Figure 44B, a percutaneously insertable instrument can be inserted through the instrument guide 530 towards the EM target 204.

[0224] In some embodiments, the percutaneously insertable instrument is coupled to a third robotic arm configured to move the percutaneously insertable instrument. In such a case, the processor may be further configured to use the third robotic arm to align an axis of the percutaneously insertable instrument with an insertion axis 532 of the instrument guide 530 and to insert the percutaneously insertable instrument through the instrument guide 530 along the insertion axis 532 toward the EM target 204 using the third robotic arm. In other embodiments, the percutaneously insertable instrument may be manually inserted through the instrument guide 530.

[0225] The system may also include a robotic medical instrument configured to be inserted into the patient. The EM target 204 may include an EM sensor on the robotic medical instrument. The robotic medical instrument may be coupled to another robotic arm configured to control the robotic medical instrument.

[0226] 45A is a flow chart illustrating a method 700 for aligning a percutaneously insertable instrument with an EM target using a robotically controllable electromagnetic field generator. The method 700 begins at block 701 where an alignment is performed that maps positions in the EM coordinate frame to positions in the robot coordinate frame. The alignment may be determined based on the kinematic pose of the robot arm to which the EM field generator is attached, as described above.

[0227] A position of the EM target may then be determined in the robot coordinate frame based on the alignment, in block 702. As described above, this may include determining a position of the EM target within the EM field generated by the EM field generator and mapping that position to the robot coordinate frame.

[0228] Based on the determined location of the EM target, in block 703, an axis of a percutaneously insertable instrument is aligned with the EM target. An example is shown in FIG. 42B, supra. Finally, in block 704, the percutaneously insertable instrument may be inserted along the axis toward the EM target. An example is shown in FIG. 42C, supra.

[0229] FIG. 45B is a flow chart illustrating a method 710 for aligning an instrument guide for a percutaneously insertable instrument with an EM target using a robotically controllable electromagnetic field generator. At block 711, the method 700 includes determining an alignment that maps a position in the EM coordinate frame to a position in the robot coordinate frame. At block 712, a position of the EM target is determined in the robot coordinate frame based on the alignment. Finally, at block 713, the method 700 includes aligning an axis of the instrument guide with the EM target. Examples are described above with reference to FIGS. 43A and 43B showing an instrument guide on an EM field generator and FIGS. 44A and 44B showing an instrument guide mounted on a second robotic arm.

[0230] iv. Distortion detection The robotically controlled EM field generator 302 can also be used for distortion detection. For example, the robotically controlled EM field generator can be commanded to move while a static EM sensor is present in the region of interest, e.g., needle insertion site, biopsy site, or any stationary position within the working volume of the EM field generator. The EM field generator 302 can be moved by the robot arm 212 to which it is attached, and the commanded robot motion (in the robot coordinate frame) can be compared to the recorded EM sensor trajectory (in the EM coordinate frame). The difference between the two trajectories can be a measure or indicator of the distortion of the EM signal within the working volume of the EM field generator 302. Similar principles can also be used with non-static EM sensors, whose motion is either known or has bounded uncertainty. The accuracy of distortion detection can depend on or be limited by the uncertainty in the knowledge of the motion / location of the EM sensor.

[0231] FIG. 46A illustrates an embodiment of a robotic medical system that moves an EM field generator 302 relative to a fixed EM position sensor 204, for example, to detect EM distortion. As shown in the illustrated embodiment, the EM field generator 302 may be coupled to a first robotic arm 212. The EM field generator 302 may be configured to generate an EM field and associated with an EM coordinate frame in which the position of an EM sensor, such as the illustrated EM sensor 204, may be determined with reference to the EM coordinate frame. The first robotic arm 212 may be configured to move to reposition the EM field generator 302. The first robotic arm 212 may be associated with a robot coordinate frame. As described above, the motion of the robotic arm 212 within the robot coordinate frame is known due to the known kinematics of the arm. Thus, because the EM field generator 302 is coupled to the first robotic arm 212, the position of the EM field generator 302 within the robot coordinate frame 212 may also be determined based on the kinematic pose of the arm. As explained in more detail above, this relationship can establish an alignment that can be used to determine the position of the EM sensor 204 in the EM coordinate frame of the EM field relative to the robot coordinate frame.

[0232] In the illustrated embodiment of Figure 46A, to determine or detect EM distortions within the EM field, the EM sensor 204 may be provided in a fixed and stationary position. For example, the EM sensor 204 may be attached to a non-moving object, such as a patient platform or other non-moving piece of equipment. In some embodiments, the EM sensor 204 is positioned on a medical instrument, such as a robotically controlled medical instrument. In this case, the medical instrument may remain stationary during the distortion detection steps described with reference to Figures 46A-47, and thus the position of the EM sensor 204 remains fixed and stationary.

[0233] To detect EM distortion, a processor in communication with the first robotic arm 212 (and, by extension, the EM field generator 302) may be configured to cause the first robotic arm 212 to move the EM field generator 302 along a robotic trajectory 850. In FIG. 46A, the robotic trajectory 850 is represented by a dashed line, and the system is shown approximately midway through its movement, such that the EM field generator 302 is positioned approximately halfway along the robotic trajectory 850. In some embodiments, movement of the EM field generator 302 by the first robotic arm 212 along the robotic trajectory 850 may be accomplished under the direction or control of an operator, such as a physician, who may command the movement using a controller or other user interface. In other embodiments, movement of the EM field generator 302 by the first robotic arm 212 along the robotic trajectory 850 may be provided automatically, for example, as part of an automated EM distortion detection process operated by the system.

[0234] In FIG. 46A, it is important to realize that in this example, the EM field generator 302 is moved along a robot trajectory 850, and this is done while the EM sensor 204 remains stationary or fixed as described above. FIG. 46A also shows an EM sensor trajectory 852, which is represented diagrammatically as a dashed dotted line. However, the EM sensor trajectory 852 is not the result of the physical movement of the EM sensor 204, which remains fixed as described above. Rather, the EM sensor trajectory 852 is generated by the movement of the EM field generator 302 relative to the stationary EM sensor 204. For example, as the first robot arm 212 moves the EM field generator 302 along the robot trajectory 850, the position of the EM sensor in the EM coordinate frame associated with the EM field generator 302 is recorded to generate the EM trajectory 852.

[0235] Thus, the robot trajectory 850 is caused by the movement of the EM field generator 302 by the first robotic arm 212. The robot trajectory 850 may be detected, determined, and / or recorded based on the known kinematic movements of the first robotic arm 212 during motion. In contrast, the EM sensor trajectory 852 is caused by the movement of the EM field generator 302 relative to the stationary EM sensor 204. The EM sensor trajectory 852 may be detected, determined, and / or recorded based on detecting the EM sensor 204 in the EM field of the EM field generator 302 as the EM field generator 302 is moved and the EM sensor 204 remains stationary.

[0236] Because the EM sensor 204 remains fixed during the movement of the EM field generator 302 by the first robotic arm 212, it is expected that the robot trajectory 850 will correspond to the EM sensor trajectory 852 in the absence of any EM distortions. This is because the movement of the EM field generator 302 should match the relative motion between the moving EM field generator 302 and the static EM sensor 204. Thus, the difference between the robot trajectory 850 and the EM sensor trajectory 852 can be analyzed to determine whether an EM distortion is present. In some embodiments, the analysis can also provide a measure of the degree or severity of the EM distortion.

[0237] FIG. 46B shows a comparison of the robot trajectory 850 and the EM sensor trajectory 852. In this example, the starting points of each trajectory are aligned, although this is not necessary in all embodiments. As shown in FIG. 46B, it is clear that the robot trajectory 850 and the EM sensor trajectory 852 do not correspond exactly. To detect the EM distortion, the difference between the trajectories can be determined. In some embodiments, the detected EM distortion is indicative of the EM distortion at the location of the fixed EM sensor 204.

[0238] Not all differences between the robot trajectory 850 and the EM sensor trajectory 852 are necessarily caused by EM distortion. For example, a robotic system may not be able to determine the robot motion with perfect accuracy for various reasons including manufacturing tolerances, motor control limitations, sensor accuracy, etc. Thus, some of the differences between the robot trajectory 850 and the EM sensor trajectory 852 may be caused by inaccuracies in determining the robot motion. However, generally, the inaccuracies in determining the robot motion may be small, e.g., in the sub-millimeter range. Similarly, the EM field generator 302 may be subject to EM noise that may lead to reduced resolution of the EM sensor positions determined within the field. This may lead to inaccurate recording of the EM sensor trajectory 852, causing differences that are not directly attributable to EM distortion. However, again, the EM field generator may be configured or selected such that the EM sensor noise is, in some embodiments, about 1 millimeter or less. Thus, for some embodiments of the robotic system, inaccuracies in determining the robot motion, and also in recording the EM sensor positions within the EM field, may lead to potential inaccuracies of about 1 millimeter or less.

[0239] Thus, to determine the presence of EM distortion, it may be beneficial to analyze the difference between the robot trajectory 850 and the EM sensor trajectory 852 with respect to a threshold value selected to eliminate or reduce contributions from other factors. In some embodiments, the threshold value may be, for example, 1 millimeter, 1.25 millimeters, 1.5 millimeters, 1.75 millimeters, 2 millimeters, 2.25 millimeters, 2.5 millimeters, 2.75 millimeters, 3 millimeters, or more. In many instances, it is beneficial to set the threshold value to be greater than one or both of an error factor associated with the motion of the first robot arm or an error factor associated with noise in the EM sensor or generator.

[0240] The comparison of the robot trajectory 850 and the EM sensor trajectory 852 can be accomplished in a variety of ways. For example, the shape of the robot trajectory 850 and the shape of the EM sensor trajectory 852 can be directly compared, as shown in FIG. 46B. The shapes can be analyzed to determine how closely they correspond, and if the deviation between the shapes exceeds a threshold, the system can determine that EM distortion is present and / or the degree of EM distortion. In another embodiment, analyzing the robot trajectory 850 and the EM sensor trajectory 852 to determine the difference therebetween can include comparing a plurality of points along the robot trajectory 850 to a corresponding plurality of points along the EM sensor trajectory 852. In some embodiments, the plurality of points are determined with respect to a time associated with the movement of the EM field generator 302 using the first robot arm 212. For example, a point on the robot trajectory 850 at the start of the movement (e.g., t=0) can be compared to a corresponding point on the EM sensor trajectory 852. Subsequent points can then be compared at subsequent discrete time steps.

[0241] In some embodiments, if an EM distortion is detected, the system can be configured to reposition the EM field generator 302 with the first robot arm 212 to a location that reduces the EM distortion. For example, in some embodiments, the system can position the EM field generator 302 to a location along the robot trajectory 850 where the difference between the robot trajectory 850 and the EM sensor trajectory 852 is reduced.

[0242] In some embodiments, this can be accomplished by moving the EM field generator 302 along a second robot trajectory to identify locations that may reduce EM distortion. Thus, the EM field generator 302 can be moved along a second robot trajectory that is different from the first robot trajectory 850. For example, the EM field generator 302 can be moved along the second robot trajectory through a space that is different from the space covered along the first robot trajectory 850. As described above, this is done while the EM sensor 204 remains stationary. The second robot trajectory can be recorded along with a second corresponding EM sensor trajectory. These trajectories can be analyzed to determine if a location that reduces EM distortion can be determined. If so, the EM field generator 302 can be moved to that location using the robot arm 212. If not, additional robot trajectories can be attempted until a suitable location for the EM field generator 302 that reduces EM distortion can be found.

[0243] 46A and 46B, the robot trajectory 850 is defined relative to the global robot reference frame, and the EM sensor trajectory 852 is defined relative to the EM field generator coordinate frame. In this case, in order to compare the robot trajectory 850 to the EM sensor trajectory 852, the EM field generator coordinate frame needs to be aligned to the global robot reference frame. This can be accomplished, for example, as described above, since the EM field generator 302 is mounted to the robot arm 212, establishing a kinematic relationship between the global robot reference frame and the EM field generator coordinate frame.

[0244] 46A and 46B is somewhat simplified for ease of understanding and illustration, and assumes that only the position of the EM field generator 302 with respect to the EM sensor 204 is changed, and not the relative orientation. In a more complex example, both the position and orientation of the EM field generator 302 with respect to the EM sensor 204 may be changed, and the detected position (and / or orientation) of the EM sensor 204 may be analyzed to determine whether (and / or the extent of) EM distortion is present. In such a case, the determination of EM distortion may be made with respect to a global reference frame 854 associated with a cart to which the robot arm 212 is attached, as shown in FIG. 46C, which illustrates the case where the robot arm 212 to which the EM field generator 302 is attached moves along a robot trajectory 850 that changes the position and orientation of the EM field generator 302 with respect to the EM sensor 204. Because the cart and EM sensor 204 are not moving, any detected change in position and / or orientation measured with respect to the global reference frame 854 is due to distortion (and negligible kinematic error).

[0245] EM distortions may be caused by any or all components positioned within the working volume of the EM field generator, and therefore a similar process may be performed while moving other components to find their optimal or preferred positions.

[0246] Figure 47 is a flow chart illustrating an exemplary method 800 for EM distortion detection. Method 800 is similar in many respects to the process described above with respect to Figures 46A-C. Method 800 can be performed using a robotic medical system, such as the systems previously described, in which the EM field is generally robotically controllable via attachment to a robotic arm of the system.

[0247] Method 800 begins at block 801. Block 801 may include moving an EM field generator coupled to a first robot arm along a robot trajectory by the first robot arm while an EM sensor remains stationary at a location. As described above, the robot trajectory may be determined based on the kinematics of the arm to which the EM field generator is attached. In some embodiments, the robot trajectory may be determined relative to a global reference frame, which may be associated, for example, with a cart to which the robot arm is attached. In some embodiments, block 801 is executed by a processor (or multiple processors) of the robotic medical system. For example, the processor may be configured to send commands to the first robot arm to cause movement of the EM field generator along the robot trajectory. In some cases, the EM sensor may be fixed at a determinable position while the EM field generator moves along the robot trajectory.

[0248] At block 802, an EM sensor trajectory of the EM sensor within the EM field associated with the EM field generator may be detected based on the sensor data generated by the EM sensor as the EM field generator moves along the robot trajectory. In this manner, as previously described, the EM trajectory may be the recorded positions of the EM sensor within the EM field during the same period as the EM field generator is moved by the robot arm according to the robot trajectory.

[0249] In block 803, the robot trajectory and the EM sensor trajectory are analyzed to determine differences between them. The analysis can be done in a number of ways, including, among others, comparing the shapes of the trajectories and / or comparing multiple discrete points along the trajectories. Such comparisons can take into account changes in position and / or orientation.

[0250] At block 804, EM distortions may be detected at the locations of the EM sensors based on a comparison of the difference between the trajectories to a threshold value. The threshold value may be selected or determined to reduce or eliminate contributions caused by factors other than EM distortions, such as errors in recording or determining the motion of the robot and / or in detecting the EM sensor position within the EM field.

[0251] FIG. 48 is a flow chart illustrating another exemplary method 900 for EM distortion detection. The method 900 can be performed using a robotic medical system, such as the system described above, where the EM field is generally robotically controllable via attachment to a robotic arm of the system. The method 900 begins at block 901 where a first robotic arm is coupled to an EM field generator. The EM field generator is configured to generate an EM field and is associated with an EM coordinate frame. The first robotic arm is associated with the robot coordinate frame, and a position of the coupled EM field generator can be determined in the robot coordinate frame based on the kinematic pose of the first robotic arm. In some embodiments, the position can be further determined relative to a global coordinate frame, such as a coordinate frame associated with a cart to which the robotic arm is attached. The position of the EM sensor within the working volume of the EM field of the EM field generator can be determined based on the pose of the arm and mapped to the robot coordinate frame (or global coordinate frame), as described above.

[0252] In block 902, with the EM field generator in a first EM field generator position relative to the EM sensor, the method 900 includes determining a first alignment between the EM coordinate frame and the robot coordinate frame based on kinematics of the first robot arm with the EM field generator in the first position, and determining a first EM sensor position of the EM sensor in the robot coordinate frame based on the first alignment.

[0253] At block 903, the method 900 includes using the first robotic arm to move the EM field generator from a first EM field generator position to a second EM field generator position relative to the EM sensor, the EM sensor remaining stationary during the movement. In some embodiments, the movement of the EM field generator by the first robotic arm may be accomplished under the direction or control of an operator, such as a physician, who may command the movement using a controller or other user interface. In other embodiments, the movement of the EM field generator by the first robotic arm along a robotic trajectory may be provided automatically, for example, as part of an automated EM distortion detection process operated by the system.

[0254] At block 904, with the EM field generator at the second EM field generator position, a second alignment between the EM coordinate frame and the robot coordinate frame is determined based on the kinematics of the first robot arm with the EM field generator at the second EM field generator position, and a second EM sensor position of the EM sensor in the robot coordinate frame (or global coordinate frame) is also determined based on the second alignment. The second alignment is necessary because the EM field generator has been moved, and the new kinematic pose of the robot arm at the second position is used to establish a new relationship between the robot coordinate frame and the EM coordinate frame.

[0255] At block 905, the method 900 includes determining a difference between the first EM sensor position and the second EM sensor position. Since the EM sensor is not moving, the determined position is expected to be the same. However, EM distortions caused by different positions of the robot arm and the EM field generator (or other components that are moved) may cause the determined position to vary. In some embodiments, changes in orientation may also be considered. At block 906, the EM distortion may be detected based on a comparison between the difference and a threshold (e.g., the difference exceeds the threshold). As discussed above, other factors may contribute to the difference. As discussed above, the threshold may be selected or determined to reduce the contribution of these other non-EM distortion factors.

[0256] A difference exceeding the threshold, when viewed relative to the threshold, may indicate the presence of EM distortion. However, by only testing two EM field generator positions (and / or orientations), it may be difficult to determine which position provided the most accurate position determination. Therefore, it may be advantageous to continue testing multiple other EM field generator positions (and / or orientations) by repeating the associated steps of moving the EM field generator to a new position using the robot arm, determining a new EM-to-robot coordinate alignment based on the new position of the EM field generator based on the kinematics of the robot arm, and determining a position of the EM sensor in the robot coordinate frame based on the newly determined alignment.

[0257] In some embodiments, this process can continue to build a map of EM distortion by determining distortion at multiple locations. The location of the EM field generator that provides a determination of the EM sensor position that varies the least from other locations can be determined to have low EM distortion, while the location of the EM field generator that provides a determination of the EM sensor position that varies the most from other locations can be determined to have high EM distortion. In this way, the system or operator can determine how best to position the EM field generator and other components to reduce EM distortion.

[0258] The EM distortions (and EM distortions in general) detected using the methods described above can be both static or dynamic. Static EM distortions are invariant and can be determined and removed from the EM measurements by applying corresponding correction values. Static EM distortions can be caused by multiple factors, including objects in the operating room that generally do not move and therefore do not change during the procedure, as well as larger scale factors such as the Earth's magnetic field. Dynamic EM distortions are caused by factors that change over the course of the procedure. These can include any of the moving robotic components of the system, as well as other components (or even people) that may move through the operating room. Distortion detection can be used both to detect and correct static distortions, as well as to find optimal positions for moving components that reduce dynamic distortions.

[0259] v. Multimodal sensor fusion The robotically controlled EM field generator 302 may also be useful to enable multimodal sensor fusion. As used herein, multimodal sensor fusion may refer to the synergistic simultaneous or parallel use of different sensor types during a single procedure. As an example, the EM field generator 302 rigidly mounted on the robot arm 212 may be used to align EM sensing technology with other imaging and sensing modalities mounted on the robot arm 212 or within the robotic system. Aligning the EM sensing modality with other imaging and sensing modalities attached to or part of the robot arm, cart, or base may facilitate, for example, providing simplified displays (where data from multiple sensing modalities may be displayed together in a unified manner), augmented reality, and the like. In this section, examples are described in which EM sensing technology may be used with ultrasound technology to provide multimodal capabilities. This may enable, for example, precise identification of a needle tip (where the needle tip includes an EM sensor) within an ultrasound image. In general, determining the needle tip in an ultrasound image may not be easy since artifacts may distort the image. Aligning the ultrasound and EM field imaging planes in a single space, such as the robot coordinate frame, may facilitate determining the needle tip in the ultrasound image. To achieve this, the ultrasound probe may be rigidly mounted on a robot arm, as described in more detail below. If the ultrasound imaging plane is kinematically known relative to the coordinate frame of the EM field generator 302, the EM sensor inside the needle may be tracked in real time and overlaid on the ultrasound image.

[0260] Attaching the ultrasound probe to a robotic arm can facilitate alignment or calibration of the imaging plane of the ultrasound probe. Calibration of the imaging plane of the ultrasound probe can generally require capturing three different ultrasound images from three different known positions. The three ultrasound images, along with the corresponding known positions, can be used to calibrate the imaging plane. Traditionally, additional equipment has been required to calibrate an ultrasound probe used in a surgical robotic system. For example, an external position tracking system needs to be set up to determine the positions at which the three ultrasound images are captured. As a specific example, an optical sensor (such as an infrared LED) can be attached to the ultrasound probe. A position tracking system, such as Optitrack, can detect the position of the optical sensor attached to the ultrasound probe and used to determine its position. In a surgical robotic setup, this can be disadvantageous as additional equipment is required.

[0261] By mounting the ultrasound probe on a robotic arm, the position of the ultrasound probe can be readily determined relative to the robot coordinate frame associated with the arm based on the kinematics of the arm in a manner similar to that described above with respect to an EM field generator mounted on a robotic arm. A reference ultrasound image can be captured at this reference position. The robotic arm can then move the ultrasound probe to a first position where a first ultrasound image is captured, the first position can be determined based on the kinematics of the arm. The robotic arm can then move the ultrasound probe to a second position where a second ultrasound image is captured, the second position can be determined based on the kinematics of the arm at the second position. The three images (base, first, and second) and positions (base, first, and second) can then be used to calibrate the imaging plane of the ultrasound probe without the need for external or additional equipment such as conventional position tracking equipment.

[0262] FIG. 49A illustrates a robotic medical system in which an ultrasound probe 1050 is attached to a robot arm 212 during a procedure for calibrating the imaging plane of the ultrasound probe. In the illustrated embodiment, the robot arm 212 is coupled to the ultrasound probe 1050. For example, the ultrasound probe 1050 may be attached to the distal end of the robot arm 212 or an instrument device manipulator (IDM) positioned on the arm. The robot arm 212 is configured to move to adjust the position of the ultrasound probe 1050. The robot arm 212 is also associated with a robot coordinate frame, as described above. In the illustrated embodiment of FIG. 49A, three positions Bi, Bj, and Bk are shown. The robot arm 212 moves to move the ultrasound probe 212 between these three positions. At each position, the kinematics of the arm can be used to determine the position of the ultrasound probe in the robot coordinate frame.

[0263] At the first position Bi, the ultrasound probe 1050 can capture a first image. The ultrasound probe 1050 is then moved to a second position Bj using the robot arm 212. At the second position Bj, a second ultrasound image is captured by the ultrasound probe 1050. The ultrasound probe 1050 is then moved to a third position Bj using the robot arm 212. At the third position Bj, a third ultrasound image is captured by the ultrasound probe 1050. The system can calibrate the imaging plane of the ultrasound probe 1050 relative to the robot coordinate frame based on the first ultrasound image, a first kinematics of the first robot arm with the ultrasound probe at the first ultrasound probe position, the second ultrasound image, a second kinematics of the first robot arm with the ultrasound probe at the second ultrasound probe position, and the third ultrasound image, a third kinematics of the first robot arm with the ultrasound probe at the second ultrasound probe position.

[0264] Once the imaging plane is calibrated and aligned to the robot coordinate frame, the robotic system can utilize multiple sensing modalities simultaneously. FIG. 49B illustrates an exemplary multi-modal robotic system. As shown in FIG. 49B, the robotic system can include a first robotic arm 212A, a second robotic arm 212B, and a third robotic arm 212C. The robotic arms can be mounted to a cart, patient platform, or other common structure such that each is associated with a common robot coordinate frame. In the illustrated embodiment, an ultrasound probe 1050 with a calibrated imaging plane is mounted to the first robotic arm 212A. An EM field generator 302 is mounted to the second robotic arm 212B. The EM field generator 302 is configured to generate an EM field within which the position of an EM sensor, such as the EM sensor 204, can be determined. Using the EM to robot coordinate frame alignment procedure described above, the position of the EM sensor 204 can be determined in the robot coordinate frame. Since the imaging plane of the ultrasound probe is also aligned to the robot coordinate frame, the imaging planes of the EM sensor 204 positions are both determined in a common space (the robot coordinate frame) and therefore both can be displayed on a common display, as shown in FIG. 49B.

[0265] In FIG. 49B, the EM sensor 204 can be positioned at the distal end of a scope that is inserted into the treatment area of ​​the patient. The scope can be a robotically controlled medical instrument that is attached to and controlled by a different robotic arm. Alternatively, the scope can be a manually controlled scope. FIG. 49B also shows that a medical instrument such as a needle can be attached to a third robotic arm 212C. The position of the needle, which can be rigidly attached to the third robotic arm 212C, can also be determined relative to the robot coordinate frame. Thus, the position of the needle can also be displayed along with the imaging plane and the EM sensor data. In this way, various sensor technologies can be fused into a single robotic system and used simultaneously during a single procedure. In this example, the rendezvous of the needle and the scope can be facilitated by the robotic data, the ultrasound data, and the EM data, improving the accuracy of the rendezvous and providing a comprehensive and improved surgical experience.

[0266] FIG. 50 is a flow chart illustrating an exemplary method 1000 for calibrating an imaging plane of an ultrasound probe for use with a robotic medical system. The method 1000 begins at block 1001 where an ultrasound probe is coupled to a first robotic arm. The first robotic arm is configured to move to adjust the position of the ultrasound probe, and the first robotic arm is associated with a robot coordinate frame. In block 1002, the ultrasound probe is moved to a first ultrasound probe position using the first robotic arm. The first ultrasound probe position can be a position where a first ultrasound image used to calibrate the imaging plane of the ultrasound probe is captured. The first position can be determined based on the kinematics of the first robotic arm. In block 1003, a first ultrasound image is acquired with the ultrasound probe positioned at the first ultrasound probe position. In block 1004, the ultrasound probe is moved to a second ultrasound probe position using the first robotic arm. The second ultrasound probe position can be a position where a second image can be captured, which is done in block 1005. The second image may also be used to calibrate the imaging plane of the ultrasound probe. The second position may be determined based on the kinematics of the arm. As indicated by arrow 1006, blocks 1004 and 1005 may be repeated to capture a third image at a third position, for example, as shown in FIG. 49A. In some embodiments, these steps may be repeated additional times to capture additional images at additional positions.

[0267] In block 1007, an imaging plane of the ultrasound probe relative to the robot coordinate frame can be calibrated based on the first ultrasound image, a first kinematics of the first robot arm with the ultrasound probe at the first ultrasound probe position, the second ultrasound image, a second kinematics of the first robot arm with the ultrasound probe at the second ultrasound probe position, and the third ultrasound image, a third kinematics of the first robot arm with the ultrasound probe at the third ultrasound probe position. Once calibrated, the imaging plane is determined relative to the robot coordinate frame and can be used in conjunction with other sensing modalities (such as EM) that may also be determined relative to the robot coordinate frame.

[0268] In some embodiments, the method 1000 may also include coupling the EM field generator to a second robotic arm, the EM field generator configured to generate an EM field, the EM field generator associated with an EM coordinate frame, and the second robotic arm configured to move to adjust a position of the EM field generator and the second robotic arm associated with the robot coordinate frame. The method 1000 may also include determining an alignment between the EM field coordinate frame and the robot coordinate frame based on the kinematics of the second robotic arm. This may be accomplished as described above. The first robotic arm and the second robotic arm may be mounted to a cart or patient platform that defines a relationship between the first robotic arm and the second robotic arm in the robot coordinate frame. This may allow both the EM field generator and the ultrasound probe to be brought into the robot coordinate frame. The method 1000 may also include positioning an EM sensor within the EM field and determining an EM sensor position in the robot coordinate frame based on the alignment. In addition, the method 1000 may also include displaying an imaging plane of the ultrasound probe and overlaying an indication of the determined EM sensor position on the displayed imaging plane of the ultrasound probe, for example, as shown in FIG. 49B.

[0269] Although the multimodal capabilities have been described with respect to EM and ultrasound, other modalities may also be used. For example, in some embodiments, a depth sensor may be attached to a robotic arm. The depth sensor may be used to align the position of the patient, bed or platform, fluoroscopic C-arm used during the procedure, and / or other items in the surgical space to a global robot reference frame. In some embodiments, these positions may be represented as a heat map or point cloud, as shown in FIG. 51A and FIG. 51B, respectively. The depth sensor camera may be configured to measure distances to objects within its field of view, as shown in FIG. 51C, for example. Calibration for the depth sensor (e.g., to align the output of the depth sensor to the robot coordinate frame or the global coordinate frame) may proceed in the same manner as the calibration of the ultrasound probe described above. Generally, the depth sensor is calibrated by the manufacturer with respect to pixel and depth information. For use with a robotic system, it may be necessary to calibrate the pose of the depth sensor relative to the robot frame, for example, as described herein, which may be accomplished in a similar manner as for the ultrasound probe described above, as described above.

[0270] In some embodiments, the robotic medical system may utilize multiple different sensor functions in combination with each other. For example, the system may include an ultrasound probe, a camera or depth sensor, and an EM field generator, each of which may have its output calibrated and aligned to the robot or global coordinate frame. In these embodiments, the ultrasound probe may provide imaging data about the inside of the patient's body. Similarly, the camera or depth sensor may provide imaging or other information about external anatomy, e.g., the patient's position, as well as external devices located in the operating room, such as a C-arm. This may be advantageous because knowing where the patient is relative to the robot coordinate frame may allow the system to identify, for example, a needle insertion site. Additionally, knowing the position of other items in the operating room relative to the robot coordinate frame, such as a C-arm, may allow the system to know when an object that may cause distortion is in proximity to the electromagnetic field generator.

[0271] 3. System Implementation and Terminology. The implementations disclosed herein provide systems, methods, and apparatus for a robot-controllable electromagnetic field generator.

[0272] It should be noted that, as used herein, the terms "couple," "coupled," "coupled," or other variations of the word coupled, may indicate either an indirect connection or a direct connection. For example, when a first component is "coupled" to a second component, the first component may be either indirectly connected to the second component through another component, or directly connected to the second component.

[0273] Phrases referring to certain computer-implemented processes and functions described herein may be stored as a processor-readable medium or one or more instructions on a computer-readable medium. The term "computer-readable medium" refers to any available medium that can be accessed by a computer or processor. By way of example, and not limitation, such media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, compact disc read-only memory (CD-ROM), or other optical disk storage, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be noted that computer-readable media may be tangible and non-transitory. As used herein, the term "code" may refer to software, instructions, code, or data that is executable by a computing device or processor.

[0274] The methods disclosed herein include one or more steps or acts for achieving the described method. Method steps and / or acts may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or acts is required for the proper operation of the described method, the order and / or use of specific steps and / or acts may be modified without departing from the scope of the claims.

[0275] As used herein, the term "plurality" refers to two or more than two. For example, a plurality of components refers to two or more than two components. The term "determining" encompasses a wide variety of acts, and thus "determining" can include calculating, computing, processing, calculating, investigating, looking up (e.g., looking in a table, database, or another data structure), ascertaining, and the like. "Determining" can also include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory), and the like. "Determining" can also include resolving, selecting, choosing, establishing, and the like.

[0276] The phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" describes both "based only on" and "based at least on."

[0277] The previous description of the disclosed implementations is provided to enable any person skilled in the art to make or use the invention disclosed herein. Various modifications to these implementations will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other implementations without departing from the scope of the invention. For example, those skilled in the art will recognize that many corresponding alternative and equivalent structural details may be employed, such as equivalent methods of fastening, mounting, coupling, or engaging tool components, equivalent mechanisms for producing specific actuation motions, and equivalent mechanisms for delivering electrical energy. Thus, the disclosure herein is not intended to be limited to the implementations expressly described, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0278] [Embodiment] (1) A robotic medical system, comprising: a first robotic arm configured to couple to an electromagnetic (EM) field generator and configured to move the EM field generator; one or more processors, determining an EM position of an EM sensor within the EM field in an EM coordinate frame associated with the EM field generator; determining a position of the EM field generator within a robot coordinate frame associated with the first robot arm; determining an alignment between the EM coordinate frame and the robot coordinate frame based on the position of the EM field generator; and determining a position of the EM sensor in the robot coordinate frame based on the alignment. (2) The system of embodiment 1, wherein the one or more processors are configured to determine the position of the EM field generator based on the kinematics of the first robotic arm. (3) The system of embodiment 1, wherein the EM field generator comprises a compact electromagnetic field generator (cFG). (4) The system of embodiment 1, wherein the first robotic arm is configured to be removably coupled to the EM field generator. (5) further comprising a second robotic arm configured to control movement of the medical instrument; The system of embodiment 1, wherein the robot coordinate frame is further associated with the second robot arm.

[0279] (6) The system of embodiment 5, wherein the first robotic arm and the second robotic arm are coupled to a movable cart. (7) The system of embodiment 5, wherein the first robotic arm and the second robotic arm are coupled to a patient platform configured to support a patient during a medical procedure. (8) The system of embodiment 5, wherein the EM sensor is positioned on the medical instrument. (9) The system of embodiment 1, wherein the one or more processors are further configured to cause the first robotic arm to move the EM field generator to enable access to a patient during a medical procedure. (10) The one or more processors: causing the first robotic arm to move the EM field generator to a new position relative to the EM sensor; 2. The system of claim 1, further configured to determine a new EM position of the EM sensor within the EM field in the EM coordinate frame associated with the EM field generator, the new EM position having improved accuracy compared to the EM position.

[0280] (11) The system of embodiment 1, wherein the one or more processors are further configured to cause the first robotic arm to move the EM field generator to center the EM sensor within a working volume of the EM field generator based on the position of the EM field generator and the position of the first robotic arm. (12) A method for performing a robotic medical procedure, the method comprising: generating an electromagnetic (EM) field using an EM field generator coupled to a first robotic arm, the EM field being related to an EM coordinate frame; determining a position of the EM field generator within a robot coordinate frame based on the kinematics of the first robot arm; determining an alignment between the EM coordinate frame and the robot coordinate frame associated with the first robot arm based on the determined position of the EM field generator; determining a position of an EM sensor within the EM coordinate frame; and determining the position of the EM sensor in the robot coordinate frame based on the alignment. (13) The method of embodiment 12, wherein the determination of the position of the EM field generator in the robot coordinate frame is based on the kinematics of the first robot arm. (14) The method of embodiment 12, further comprising removably coupling the EM field generator to the first robotic arm. (15) The method of embodiment 12, further comprising having the first robotic arm move the EM field generator.

[0281] (16) The method of embodiment 15, wherein having the first robotic arm move the EM field generator includes having the first robotic arm move the EM field generator to a position that allows access to a patient during a medical procedure. (17) The method of embodiment 15, wherein having the first robotic arm move the EM field generator includes moving the EM field generator to a new position associated with improved accuracy in tracking the EM sensor within the EM field coordinate frame. (18) The method of embodiment 15, wherein having the first robotic arm move the EM field generator includes centering the EM sensor within a working volume of the EM field generator. (19) The method of embodiment 12, further comprising controlling a medical instrument coupled to a second robotic arm, the robot coordinate frame being associated with the second robotic arm. (20) The method of embodiment 19, wherein the first robotic arm and the second robotic arm are coupled to a movable cart.

[0282] (21) The method of embodiment 20, further comprising positioning the mobile cart relative to a patient platform configured to support a patient during the robotic medical procedure such that a known spatial relationship exists between a robot coordinate frame and the patient platform. (22) The method of embodiment 19, wherein the first robotic arm and the second robotic arm are coupled to a patient platform configured to support a patient during the robotic medical procedure. (23) The method of embodiment 19, wherein the EM sensor is positioned on the medical instrument. (24) The method of claim 12, wherein the EM field generator comprises a compact electromagnetic field generator (cFG). (25) A non-transitory computer-readable storage medium containing computer program instructions that, when executed by one or more processors, cause the one or more processors to: determining a position of an electromagnetic (EM) field generator in a robot coordinate frame based on kinematics of a first robot arm, the EM field generator being coupled to the first robot arm; determining an alignment between an EM coordinate frame and the robot coordinate frame associated with the first robot arm based on the determined position of the EM field generator, the EM coordinate frame being associated with an EM field generated by the EM generator; determining a position of an EM sensor within the EM coordinate frame; and determining the position of the EM sensor within the robot coordinate frame based on the alignment.

[0283] (26) The non-transitory computer-readable storage medium of embodiment 25, wherein the EM field generator comprises a compact electromagnetic field generator (cFG). (27) The non-transitory computer-readable storage medium of embodiment 25, wherein the first robotic arm is configured to be removably coupled to the EM field generator. (28) The computer program instructions, when executed, further cause the one or more processors to: controlling a second robotic arm configured to control the movement of the medical instrument; 26. The non-transitory computer-readable storage medium of embodiment 25, wherein the robot coordinate frame is further associated with the second robot arm. (29) The non-transitory computer-readable storage medium of embodiment 28, wherein the first robotic arm and the second robotic arm are coupled to a movable cart. (30) The non-transitory computer-readable storage medium of embodiment 28, wherein the first robotic arm and the second robotic arm are coupled to a patient platform configured to support a patient during a medical procedure.

[0284] (31) A robotic medical system, comprising: a first robotic arm coupled to an electromagnetic (EM) field generator configured to generate an EM field, the first robotic arm configured to move the EM field generator; a medical device configured to be inserted into a patient, the medical device comprising an EM sensor; one or more processors, determining a position of the EM sensor within the EM field; and adjusting a position of the EM field generator by commanding movement of the first robotic arm based on the determined position of the EM sensor. (32) The system of embodiment 1, wherein the EM sensor is positioned at a predetermined location within the EM field by adjusting the position of the EM field generator based on the determined position of the EM sensor. (33) The system of embodiment 32, wherein the predetermined location within the EM field comprises a center of a working volume of the EM field. (34) The system of embodiment 1, wherein adjusting the position of the EM field generator based on the determined position of the EM sensor includes adjusting the position of the EM field generator such that the EM sensor is positioned within a predetermined region of a working volume of the EM field. (35) The one or more processors: determining an orientation of the EM sensor within the EM field; 2. The system of claim 1, further configured to: adjust at least one of an orientation and the position of the EM field generator by commanding movement of the first robotic arm based on the determined orientation of the EM sensor.

[0285] (36) The system of embodiment 35, wherein the one or more processors are configured to adjust at least one of the orientation and the position of the EM field generator based on the determined orientation of the EM sensor to increase accuracy of the determined position of the EM sensor within the EM field. (37) The one or more processors: 2. The system of embodiment 1, further configured to command movement of the first robotic arm to move the EM field generator so that the EM sensor is positioned within a working volume of the EM field during movement of the medical instrument. (38) The system of embodiment 37, wherein the movement of the EM field generator via the first robotic arm tracks the movement of the medical instrument. (39) The system of embodiment 37, wherein the one or more processors are configured to command movement of the first robotic arm to move the EM field generator during movement of the medical instrument such that the EM sensor is positioned within a predetermined region of the working volume of the EM field. (40) The system of embodiment 37, wherein the one or more processors are configured to command movement of the first robotic arm to move the EM field generator during movement of the medical instrument such that the EM sensor is positioned at a predetermined location within the working volume of the EM field.

[0286] (41) The system of embodiment 40, wherein the predetermined location comprises a center of the working volume of the EM field. (42) A second robotic arm is further provided, the medical instrument is coupled to the second robotic arm; The system of embodiment 37, wherein the one or more processors are configured to command movement of the medical instrument by the second robotic arm. (43) The system of embodiment 42, wherein the first robotic arm and the second robotic arm are coupled to one of a cart and a patient platform, the patient platform configured to support a patient during a medical procedure. (44) The system of embodiment 43, further comprising a third robotic arm coupled to one of the cart and the patient platform. (45) The one or more processors: determining the position of the EM sensor within the EM field relative to an EM coordinate frame associated with an EM field generator; determining an alignment between the EM coordinate frame and a robot coordinate frame associated with the first robot arm based on determining a position of the EM field generator in the robot coordinate frame; 2. The system of claim 1, further comprising: determining the position of the EM sensor within the robot coordinate frame based on the alignment.

[0287] (46) The system of embodiment 45, wherein the one or more processors are configured to determine the position of the EM field generator within the robot coordinate frame based on the kinematics of the first robot arm. (47) A robotic medical method, comprising: determining a position of an electromagnetic (EM) sensor of the medical instrument within a working volume of an EM field generated by an EM field generator coupled to a first robotic arm; moving the medical device; in response to the movement of the medical instrument, instructing the first robotic arm to move the EM field generator such that the EM sensor remains positioned within the working volume of the EM field generator. (48) The medical instrument is coupled to a second robotic arm; 48. The method of embodiment 47, wherein moving the medical instrument comprises moving the medical instrument using the second robotic arm. (49) The method of embodiment 48, wherein moving the instrument with the second robotic arm includes articulating the second robotic arm. (50) The medical instrument is coupled to an instrument drive mechanism; 49. The method of embodiment 48, wherein moving the medical instrument using the second robotic arm includes actuating the medical instrument using the instrument drive mechanism.

[0288] (51) The method of embodiment 47, wherein the EM field generator tracks the movement of the medical instrument by commanding the first robotic arm to move the EM field generator. (52) The method of embodiment 47, wherein instructing the first robotic arm to move the EM field generator includes moving the EM field generator such that the EM sensor is positioned within a predetermined region of the working volume of the EM field. (53) The method of embodiment 47, wherein instructing the first robotic arm to move the EM field generator includes moving the EM field generator such that the EM sensor is positioned at a predetermined location within the working volume of the EM field. (54) determining an orientation of the EM sensor within the EM field; 48. The method of claim 47, further comprising: adjusting at least one of an orientation and the position of the EM field generator by commanding movement of the first robot arm based on the determined orientation of the EM sensor. (55) The method of embodiment 54, wherein adjusting at least one of the orientation and the position of the EM field increases accuracy of the determined position of the EM sensor within the working volume of the EM field.

[0289] (56) determining the position of the EM sensor within the working volume of the EM field relative to an EM coordinate frame associated with an EM field generator; determining an alignment between the EM coordinate frame and a robot coordinate frame associated with the first robot arm based on determining a position of the EM field generator in the robot coordinate frame; 48. The method of claim 47, further comprising determining the position of the EM sensor in the robot coordinate frame based on the alignment. (57) The method of embodiment 56, wherein determining the position of the EM field generator within the robot coordinate frame is based on the kinematics of the first robot arm. (58) A robotic medical system, comprising: an electromagnetic (EM) field generator configured to generate an EM field; a first robotic arm coupled to the EM field generator and configured to move the EM field generator; and one or more processors in communication with the EM field generator and the first robotic arm, the one or more processors: determining positions of a plurality of EM sensors within the EM field; determining a generator position of the EM field generator based on the determined positions of the plurality of EM sensors; and commanding the first robotic arm to move the EM field generator to the determined generator position. (59) The system of embodiment 58, wherein the one or more processors are configured to determine the generator position based on determining a centroid of the determined positions of the multiple EM sensors. (60) A medical instrument further comprising an elongate shaft configured to be inserted into a patient; The system of embodiment 58, wherein at least one of the plurality of EM sensors is positioned on the elongated shaft.

[0290] (61) A second robotic arm is further provided, The system of embodiment 60, wherein the second robotic arm is coupled to the medical instrument such that the second robotic arm can move the medical instrument. (62) The system of embodiment 61, wherein the first robotic arm and the second robotic arm are coupled to a cart or a patient platform configured to support a patient during a robotic medical procedure. (63) The system of embodiment 60, wherein at least one of the EM sensors comprises a patch sensor positioned on the patient and configured to track respiration or other movement of the patient. (64) The one or more processors: determining the positions of the plurality of EM sensors within the EM field relative to an EM coordinate frame associated with an EM field generator; determining an alignment between the EM coordinate frame and a robot coordinate frame associated with the first robot arm based on determining a position of the EM field generator in the robot coordinate frame; 59. The system of claim 58, further configured to: determine the position of the EM sensor in the robot coordinate frame based on the alignment. (65) The system described in embodiment 64, wherein the determination of the position of the EM field generator in the robot coordinate frame is based on the kinematics of the first robot arm.

[0291] (66) The one or more processors: Detecting movement of at least one of the EM sensors; determining a new generator position for the EM field generator based on the detected movement; and 59. The system of embodiment 58, further configured to command the first robotic arm to move the EM field generator to the new generator position. (67) A method for positioning an electromagnetic (EM) field generator during a robotic medical procedure, said method comprising: generating an EM field using an EM field generator coupled to a first robotic arm; determining positions of a plurality of EM sensors within the EM field; determining a generator position of the EM field generator based on the determined positions of the plurality of EM sensors; and commanding the first robotic arm to move the EM field generator to the generator position. (68) The method of embodiment 67, further comprising determining the generator position based on determining a centroid of the determined positions of the plurality of EM sensors. (69) The method of embodiment 67, wherein at least one of the plurality of EM sensors is positioned on an elongated shaft of a medical instrument. (70) The method of embodiment 69, wherein the medical instrument is coupled to a second robotic arm.

[0292] (71) The method of embodiment 70, wherein at least one of the EM sensors comprises a patch sensor positioned on a patient and configured to track respiration or other movement of the patient. (72) determining the positions of the plurality of electromagnetic (EM) sensors within the EM field relative to an EM coordinate frame associated with an EM field generator; determining an alignment between the EM coordinate frame and a robot coordinate frame associated with the first robot arm based on determining a generator position of the EM field generator in the robot coordinate frame; 68. The method of claim 67, further comprising determining the positions of the multiple EM sensors in the robot coordinate frame based on the alignment. (73) The method of embodiment 68, wherein the determination of the generator position of the EM field generator in the robot coordinate frame is based on the kinematics of the first robot arm. (74) detecting movement of at least one of the plurality of EM sensors; determining a new generator position for the EM field generator based on the detected movement; and 68. The method of embodiment 67, further comprising instructing the first robotic arm to move the EM field generator to the new generator position. (75) A robotic medical system comprising: an electromagnetic (EM) field generator configured to generate an EM field; a first robotic arm coupled to the EM field generator and configured to move it; a percutaneously insertable instrument extending along an axis; a second robotic arm coupled to and configured to move the percutaneously insertable instrument; and an EM target configured to be positioned within a patient; and one or more processors in communication with the EM field generator and the first robotic arm, the one or more processors: determining an alignment that maps positions in an EM coordinate frame associated with the EM field generator to positions in a robot coordinate frame based on a kinematic pose of the first robot arm; determining a position of the EM target within the robot coordinate frame based on the alignment; and moving the second robotic arm to align the axis of the percutaneously insertable instrument with the EM target based on the position of the EM target in the robot coordinate frame; and causing insertion of the percutaneously insertable instrument along the axis towards the EM target using the second robotic arm.

[0293] (76) The alignment is determined based on the position of the EM field generator in the robot coordinate frame; The system of embodiment 75, wherein the position of the EM field generator having the robot coordinate frame is determined based on the kinematic posture of the first robot arm. (77) The one or more processors: determining a position of the EM target within the EM coordinate frame; The system of embodiment 76, wherein the system is configured to determine the position of the EM target in the robot coordinate frame by using the alignment to map the position of the EM target in the EM coordinate frame to the position of the EM target in the robot coordinate frame. (78) A robotic medical instrument configured to be inserted into the patient; a third robotic arm coupled to and configured to control the robotic medical instrument, The system of embodiment 75, wherein the EM target comprises an EM sensor positioned on the robotic medical instrument. (79) The system of embodiment 75, wherein the percutaneously insertable instrument comprises one of a needle, an access sheath, and a laparoscopic instrument. (80) A robotic medical system comprising: an instrument guide configured to guide a percutaneously insertable instrument along an insertion axis, the instrument guide configured to be positioned over an electromagnetic (EM) field generator, the EM field generator configured to generate an EM field; a first robotic arm configured to be coupled to the EM field generator, the first robotic arm further configured to move the EM field generator and the instrument guide; one or more processors, determining an EM target located within the patient; determining an alignment that maps positions in an EM coordinate frame associated with the EM field to positions in a robot coordinate frame associated with a kinematic pose of the first robot arm; determining a position of the EM target within the robot coordinate frame based on the alignment; and and one or more processors configured to: move the first robot arm to align the insertion axis of the instrument guide with the EM target based on the position of the EM target in the robot coordinate frame.

[0294] (81) A method for performing a percutaneously insertable instrumentation system, comprising: a first robot arm configured to couple with a percutaneously insertable instrument extending along an axis; the first robot arm further configured to move the percutaneously insertable instrument; the one or more processors: aligning the axis of the percutaneously insertable instrument with the insertion axis using the second robotic arm; The robotic medical system of embodiment 80, further configured to use the second robotic arm to insert the percutaneously insertable instrument through the instrument guide along the insertion axis toward the EM target. (82) The system of embodiment 81, further comprising a third robotic arm configured to couple with a robotic medical instrument, the third robotic arm further configured to control the robotic medical instrument while the robotic medical instrument is inserted into the patient, and the EM target comprises an EM sensor disposed on the robotic medical instrument. (83) The alignment is determined based on a position of the EM field generator within the robot frame; The system of embodiment 80, wherein the position of the EM field generator having the robot coordinate frame is determined based on the kinematic posture of the first robot arm. (84) The one or more processors: determining a position of the EM target within the EM coordinate frame; The system of embodiment 83, wherein the system is configured to determine the position of the EM target in the robot coordinate frame by mapping the position of the EM target in the EM coordinate frame to the position of the EM target in the robot coordinate frame based on the alignment. (85) The system of embodiment 80, wherein the percutaneously insertable instrument comprises one of a needle, an access sheath, and a laparoscopic instrument.

[0295] (86) A robotic medical system comprising: a first robotic arm configured to couple to and move an electromagnetic (EM) field generator, the EM field generator configured to generate an EM field; a second robotic arm configured to couple to move an instrument guide, the instrument guide configured to guide a percutaneously insertable instrument along an insertion axis; and one or more processors, determining an EM target located within the patient; determining an alignment that maps positions in an EM coordinate frame associated with the EM field generator to positions in a robot coordinate frame based on a kinematic pose of the first robot arm; determining a position of the EM target within the robot coordinate frame based on the alignment; and and one or more processors configured to: move the second robot arm to align the insertion axis of the instrument guide with the EM target based on the position of the EM target in the robot coordinate frame. (87) A method for treating a patient, comprising: a) providing a first robot arm configured to couple to and move the percutaneously insertable instrument; and b) providing a second robot arm configured to couple to and move the percutaneously insertable instrument, the percutaneously insertable instrument extending along an axis; the one or more processors: aligning the axis of the percutaneously insertable instrument with the insertion axis using the third robotic arm; The robotic medical system of embodiment 86, further configured to use the third robotic arm to insert the percutaneously insertable instrument through the instrument guide along the insertion axis toward the EM target. (88) A fourth robotic arm configured to couple to and control a robotic medical instrument, the robotic medical instrument configured to be inserted into the patient; The system of embodiment 87, wherein the EM target comprises an EM sensor positioned on the robotic medical instrument. (89) The alignment is determined based on a position of the EM field generator within the robot frame; The system of embodiment 86, wherein the position of the EM field generator having the robot coordinate frame is determined based on the kinematic posture of the first robot arm. (90) The one or more processors: determining a position of the EM target within the EM coordinate frame; The system of embodiment 89, further configured to determine the position of the EM target in the robot coordinate frame based on the alignment being configured to map the position of the EM target in the EM coordinate frame to the position of the EM target in the robot coordinate frame.

[0296] (91) The system of embodiment 86, wherein the percutaneously insertable instrument comprises one of a needle, an access sheath, and a laparoscopic instrument. (92) A robotic medical method, comprising: determining an alignment that maps positions in an electromagnetic (EM) coordinate frame associated with an EM field generator coupled to a first robotic arm to positions in a robot coordinate frame based on a kinematic pose of the first robotic arm; determining a position of an EM target in the robot coordinate frame based on the alignment; and moving a second robotic arm coupled to a percutaneously insertable instrument based on the position of the EM target in the robot coordinate frame to align an axis of the percutaneously insertable instrument with the EM target; and causing insertion of the percutaneously insertable instrument along the axis toward the EM target using the second robotic arm. (93) The method of embodiment 92, wherein determining the alignment includes determining the alignment based on the position of the EM field generator within the robot frame, and the position of the EM field generator with the robot coordinate frame is determined based on the kinematic attitude of the first robot arm. (94) The determining the position of the EM target within the robot coordinate frame comprises: determining a position of the EM target within the EM coordinate frame; 94. The method of claim 93, further comprising: using the alignment to map the position of the EM target in the EM coordinate frame to the position of the EM target in the robot coordinate frame. (95) The method of claim 1, further comprising navigating the robotic medical instrument within the patient using a third robotic arm coupled to and configured to control the robotic medical instrument; 93. The method of embodiment 92, wherein the EM target comprises an EM sensor positioned on the robotic medical instrument.

[0297] (96) The method of embodiment 92, wherein the percutaneously insertable instrument comprises one of a needle, an access sheath, and a laparoscopic instrument. (97) A robotic medical method, comprising: determining an alignment that maps positions in an electromagnetic (EM) coordinate frame associated with an EM field generator coupled to a first robotic arm to positions in a robot coordinate frame based on a kinematic pose of the first robotic arm; determining a position of an EM target in the robot coordinate frame based on the alignment; and and moving the first robotic arm to align an insertion axis of an instrument guide positioned on the EM field generator with the EM target based on the position of the EM target in the robot coordinate frame, the instrument guide being configured to guide a percutaneously insertable instrument along the insertion axis. (98) A robotic medical method, comprising: determining an alignment that maps positions in an electromagnetic (EM) coordinate frame associated with an EM field generator coupled to a first robotic arm to positions in a robot coordinate frame based on a kinematic pose of the first robotic arm; determining a position of an EM target in the robot coordinate frame based on the alignment; and and moving a second robotic arm to align an insertion axis of an instrument guide coupled to the second robotic arm with the EM target based on the position of the EM target in the robot coordinate frame. (99) A robotic medical system comprising: a first robotic arm coupled to and configured to move an electromagnetic (EM) field generator, the EM field generator configured to generate an EM field having a working volume in which a position of an EM sensor can be determined relative to an EM coordinate frame; one or more processors, moving the EM field generator using the first robotic arm; When an EM sensor is detected within the working volume, (i) determining a position of the EM sensor within the working volume relative to the EM coordinate frame; (ii) mapping the position of the EM sensor to a robot coordinate frame associated with the first robot arm based on a kinematic pose of the first robot arm. (100) The system of embodiment 99, wherein the one or more processors are configured to use the first robotic arm to move the EM field generator along a search path, the search path being configured to sweep the working volume through a treatment volume larger than the working volume and to identify a position of an EM sensor within the treatment volume.

[0298] (101) The one or more processors: determining a tracking path based on the determined position of the EM sensor within the robot coordinate frame; moving the EM field generator along the tracking path using the first robotic arm to track the position of the EM sensor; The system described in embodiment 100, further configured to move the EM field generator along the tracking path using the first robotic arm. (102) When the EM field generator moves along the tracing path, the one or more processors: (i) re-determining the position of the EM sensor within the working volume relative to the EM coordinate frame; (ii) remapping the position of the EM sensor into the robot coordinate frame based on the kinematic pose of the first robot arm; (iii) determining an updated tracking route. (103) A robotic medical system, comprising: a first robotic arm coupled to and configured to move an electromagnetic (EM) field generator, the EM field generator configured to generate an EM field having a working volume in which a position of an EM sensor can be determined relative to an EM coordinate frame; one or more processors, (i) with the EM field generator in a first position; determining a position of a first EM sensor within the working volume relative to the EM coordinate frame; mapping the position of the first EM sensor into a robot coordinate frame associated with the first robot arm based on kinematics of the first robot arm with the EM field generator in the first position; (ii) moving the EM field generator to a second position; (iii) with the EM field generator in the second position; determining a position of a second EM sensor within the working volume relative to the EM coordinate frame; and mapping the position of the second EM sensor to the robot coordinate frame based on the kinematics of the first robot arm with the EM field generator in the second position. (104) The system of embodiment 103, wherein the one or more processors are further configured to move the first robot arm such that a first EM sensor and the second EM sensor are not simultaneously located within the working volume of the EM field generator. (105) The system of embodiment 103, wherein the one or more processors are further configured to move the EM field generator between the first position and the second position to track the positions of the first EM sensor and the second EM sensor.

[0299] (106) A method comprising the steps of: using a first robotic arm to move an electromagnetic (EM) field generator positioned on the first robotic arm; When an EM sensor is detected within the working volume, (i) determining a position of the EM sensor within a working volume of an EM field generated by the EM field generator relative to an EM coordinate frame; (ii) mapping the position of the EM sensor to a robot coordinate frame associated with the first robot arm based on a kinematic pose of the first robot arm. (107) A method comprising the steps of: (i) with an electromagnetic (EM) field generator coupled to a first robotic arm in a first position; determining a position of a first EM sensor within a working volume of an EM field generated by the EM field generator relative to the EM coordinate frame; mapping the position of the first EM sensor into a robot coordinate frame associated with the first robot arm based on kinematics of the first robot arm with the EM field generator in the first position; (ii) moving the EM field generator to a second location using the first robotic arm; (iii) with the EM field generator in the second position; determining a position of a second EM sensor within the working volume relative to the EM coordinate frame; and mapping the position of the second EM sensor into the robot coordinate frame based on the kinematics of the first robot arm with the EM field generator in the second position. (108) A robotic medical system, comprising: a first robotic arm coupled to an electromagnetic (EM) field generator configured to generate an EM field; EM sensor and a processor, the processor comprising: sending a command to the first robot arm to cause movement of the EM field generator along a robot trajectory while the EM sensor remains in place; detecting an EM sensor trajectory of the EM sensor within the EM field corresponding to a period of time that the EM field generator moved along the robot trajectory; analyzing the robot trajectory and the EM sensor trajectory to determine a difference between the robot trajectory and the EM sensor trajectory; and detecting EM distortion at the location based on a comparison between the difference and a threshold. (109) The system of embodiment 108, wherein the processor is configured to determine the robot trajectory based on kinematic data corresponding to the first robot arm. (110) The system of embodiment 108, wherein the processor is configured to determine the threshold value based on at least one of an error factor associated with movement of the first robot arm or an error factor associated with EM sensor noise.

[0300] (111) The system of embodiment 108, wherein the processor is configured to determine the difference between the robot trajectory and the EM sensor trajectory based on a comparison of a shape of the robot trajectory and a shape of the EM sensor trajectory. (112) The system of embodiment 108, wherein the processor is configured to determine the difference between the robot trajectory and the EM sensor trajectory based on a comparison of a plurality of points along the robot trajectory to a corresponding plurality of points along the EM sensor trajectory. (113) The system of embodiment 112, wherein each of the plurality of points is determined with respect to a time associated with the movement of the EM field generator. (114) The system of embodiment 108, wherein the processor is further configured to, upon detecting the EM distortion at the location, reposition the EM field generator using the first robotic arm to reduce the EM distortion at the location. (115) The processor, sending an additional command to the first robot arm to cause movement of the EM field generator along a second robot trajectory while the EM sensor remains at the location; detecting a second EM sensor trajectory of the EM sensor within the EM field corresponding to a period of time that the EM field generator moved along the second robot trajectory; analyzing the second robot trajectory and the second EM sensor trajectory to determine a difference between the second robot trajectory and the second EM sensor trajectory; The system described in embodiment 114, wherein the system is configured to reposition the EM field generator using the first robot arm to reduce the EM distortion at the location by comparing the difference between the second robot trajectory and the second EM sensor trajectory with the difference between the robot trajectory and the EM sensor trajectory.

[0301] (116) The system of embodiment 115, wherein the processor is configured to position the EM field generator at a point along the second robot trajectory when the difference between the second robot trajectory and the second EM sensor trajectory is less than the difference between the robot trajectory and the EM sensor trajectory. (117) A method executed by a set of one or more processors of a robotic system, the method comprising: commanding a first robotic arm to move an electromagnetic (EM) field generator from a first location to a second location while the EM sensor remains at a location; determining a robot trajectory associated with the EM field generator moving from the first location to the second location; detecting an EM sensor trajectory of the EM sensor within an EM field, the EM sensor trajectory being related to a period of time that the EM field generator moved along the robot trajectory; determining a difference between the robot trajectory and the EM sensor trajectory; and detecting EM distortion at the location based on a comparison of the difference to a distortion threshold. (118) The method of embodiment 117, wherein determining the robot trajectory is further based on using kinematic data corresponding to the first robot arm. (119) The method of embodiment 117, wherein determining the robot trajectory associated with the EM field generator moving from the first location to the second location includes compensating for error factors associated with movement of the first robot arm. (120) The method of embodiment 117, wherein detecting the EM sensor trajectory includes compensating for error factors associated with EM sensor noise.

[0302] (121) The method of embodiment 117, wherein determining the difference between the robot trajectory and the EM sensor trajectory further comprises comparing a shape of the robot trajectory to a shape of the EM sensor trajectory. (122) The method of embodiment 117, wherein determining the difference between the robot trajectory and the EM sensor trajectory further includes comparing a plurality of points along the robot trajectory to a corresponding plurality of points along the EM sensor trajectory. (123) The method of embodiment 122, wherein each of the plurality of points is determined with respect to a time associated with the movement of the EM field generator. (124) The method of embodiment 117, further comprising, upon detecting the EM distortion at the location, repositioning the EM field generator using the first robotic arm to reduce the EM distortion at the location. (125) commanding the first robot arm to cause movement of the EM field generator along a second robot trajectory while the EM sensor remains at the location; detecting a second EM sensor trajectory of the EM sensor within the EM field, the second EM sensor trajectory being associated with a period of time that the EM field generator moved along the second robot trajectory; analyzing the second robot trajectory and the second EM sensor trajectory to determine a difference between the second robot trajectory and the second EM sensor trajectory; 118. The method of claim 117, further comprising comparing the difference between the second robot trajectory and the second EM sensor trajectory with the difference between the robot trajectory and the EM sensor trajectory.

[0303] (126) The method of embodiment 125, further comprising positioning the EM field generator at a point along the second robot trajectory when the difference between the second robot trajectory and the second EM sensor trajectory is less than the difference between the robot trajectory and the EM sensor trajectory. (127) A robotic medical system comprising: a first robotic arm coupled to an electromagnetic (EM) field generator configured to generate an EM field, the first robotic arm associated with a robot coordinate frame, the EM field generator associated with the EM coordinate frame; EM sensor and a processor in communication with the first robotic arm, the processor comprising: determining a first alignment between the EM coordinate frame and the robot coordinate frame based on kinematics of the first robot arm with the EM field generator in a first EM field generator position relative to a location of the EM sensor; determining a first EM sensor position of the EM sensor in the robot coordinate frame based on the first alignment; moving the EM field generator from the first EM field generator position to a second EM field generator position relative to the EM sensor using the first robotic arm; determining a second alignment between the EM coordinate frame and the robot coordinate frame based on the kinematics of the first robot arm with the EM field generator at the second EM field generator position relative to the location of the EM sensor; determining a second EM sensor position of the EM sensor in the robot coordinate frame based on the second alignment; determining a difference between the first EM sensor position and the second EM sensor position; and detecting EM distortion at the location based on a comparison between the difference and a threshold. (128) The system of embodiment 127, wherein the processor is further configured to determine position distortion. (129) The system of embodiment 127, wherein the processor is further configured to determine orientation distortion. (130) The system of embodiment 127, wherein the threshold is determined based on at least one of an error factor associated with movement of the first robot arm or an error factor associated with EM sensor noise.

[0304] (131) A method comprising the steps of: coupling an electromagnetic (EM) field generator to a first robotic arm, the EM field generator configured to generate an EM field associated with an EM coordinate frame, the first robotic arm associated with a robot coordinate frame; determining a first alignment between the EM coordinate frame and the robot coordinate frame based on kinematics of the first robot arm with the EM field generator in a first EM field generator position relative to an EM sensor location; determining a first EM sensor position of the EM sensor in the robot coordinate frame based on the first alignment; moving the EM field generator from the first EM field generator position to a second EM field generator position relative to the EM sensor using the first robotic arm, wherein the EM sensor remains stationary during the movement; determining a second alignment between the EM coordinate frame and the robot coordinate frame based on the kinematics of the first robot arm and the location of the EM sensor with the EM field generator in the second EM field generator position; determining a second EM sensor position of the EM sensor in the robot coordinate frame based on the second alignment; determining a difference between the first EM sensor position and the second EM sensor position; and detecting EM distortion at the location based on the difference exceeding a threshold. (132) A robotic medical system comprising: a first robotic arm coupled to an ultrasonic probe, the first robotic arm configured to move to adjust a position of the ultrasonic probe, the first robotic arm associated with a robot coordinate frame; a processor in communication with the first robotic arm and the ultrasound probe, the processor comprising: moving the ultrasonic probe to a first ultrasonic probe position using the first robotic arm; capturing a first ultrasound image with the ultrasound probe positioned at the first ultrasound probe position; moving the ultrasonic probe to a second ultrasonic probe position using the first robotic arm; capturing a second ultrasound image with the ultrasound probe positioned at the second ultrasound probe position; moving the ultrasonic probe to a third ultrasonic probe position using the first robotic arm; capturing a third ultrasound image with the ultrasound probe positioned at the third ultrasound probe position; an imaging plane of the ultrasound probe relative to the robot coordinate frame; the first ultrasound image and a first kinematics of the first robotic arm with the ultrasound probe at the first ultrasound probe position; the second ultrasound image and second kinematics of the first robotic arm with the ultrasound probe at the second ultrasound probe position; and calibrating based on the third ultrasound image and a third kinematics of the first robotic arm with the ultrasound probe at the third ultrasound probe position. (133) A method for generating an electromagnetic (EM) field, comprising: a second robotic arm configured to move to adjust a position of the EM field generator, the second robotic arm being associated with the robot coordinate frame; The processor, The system of embodiment 132, further configured to determine alignment between the EM field coordinate frame and the robot coordinate frame based on the kinematics of the second robot arm. (134) The system described in embodiment 133, wherein the first robotic arm and the second robotic arm are mounted to a cart or patient platform that defines a relationship between the first robotic arm and the second robotic arm within the robot coordinate frame. (135) The system of embodiment 133, further comprising an EM sensor detectable within the EM field, and the processor is further configured to determine an EM sensor position within the robot coordinate frame based on the alignment.

[0305] (136) The processor, displaying the imaging plane of the ultrasound probe; The system of embodiment 135, further configured to overlay an indication of the determined EM sensor position on the displayed imaging surface of the ultrasound probe. (137) A method comprising the steps of: coupling an ultrasonic probe to a first robotic arm, the first robotic arm configured to move to adjust a position of the ultrasonic probe, the first robotic arm associated with a robot coordinate frame; moving the ultrasonic probe to a first ultrasonic probe position using the first robotic arm; capturing a first ultrasound image with the ultrasound probe positioned at the first ultrasound probe position; moving the ultrasonic probe to a second ultrasonic probe position using the first robotic arm; capturing a second ultrasound image with the ultrasound probe positioned at the second ultrasound probe position; moving the ultrasonic probe to a third ultrasonic probe position using the first robotic arm; capturing a third ultrasound image with the ultrasound probe positioned at the third ultrasound probe position; an imaging plane of the ultrasound probe relative to the robot coordinate frame; the first ultrasound image and a first kinematics of the first robotic arm with the ultrasound probe at the first ultrasound probe position; the second ultrasound image and second kinematics of the first robotic arm with the ultrasound probe at the second ultrasound probe position; and calibrating based on the third ultrasound image and a third kinematics of the first robotic arm with the ultrasound probe at the third ultrasound probe position. (138) Coupling an electromagnetic (EM) field generator to a second robotic arm, the EM field generator configured to generate an EM field, the EM field generator associated with an EM coordinate frame, the second robotic arm configured to move to adjust a position of the EM field generator, the second robotic arm associated with the robot coordinate frame; 138. The method of claim 137, further comprising determining alignment between the EM field coordinate frame and the robot coordinate frame based on the kinematics of the second robot arm. (139) The method of embodiment 138, wherein the first robotic arm and the second robotic arm are mounted to a cart or patient platform that defines a relationship between the first robotic arm and the second robotic arm within the robot coordinate frame. (140) Positioning an EM sensor within the EM field; 139. The method of claim 138, further comprising determining an EM sensor position in the robot coordinate frame based on the alignment.

[0306] (141) Displaying the imaging plane of the ultrasound probe; 141. The method of embodiment 140, further comprising overlaying an indication of the determined EM sensor position on the displayed imaging surface of the ultrasound probe. (142) A robotic medical system comprising: a first robotic arm coupled to a first imaging device, the first robotic arm configured to move to adjust a position of the first imaging device, the first robotic arm associated with a robot coordinate frame; a processor in communication with the first robotic arm and the first imaging device, the processor comprising: moving the first imaging device to a first position using the first robotic arm; capturing a first image with the first imaging device positioned at the first location; moving the first imaging device to a second position using the first robotic arm; capturing a second image with the first imaging device positioned at the second location; moving the first imaging device to a third position using the first robotic arm; capturing a third image with the first imaging device positioned at the third position; the first imaging device relative to the robot coordinate frame; the first image and a first kinematics of the first robot arm with the first imaging device in the first position; the second image and second kinematics of the first robot arm with the first imaging device in the second position; calibrating based on the third image and a third kinematics of the first robotic arm with the first imaging device in the third position. (143) A system for imaging a subject, further comprising: a second robot arm coupled to a second imaging device, the second robot arm configured to move to adjust a position of the second imaging device, the second robot arm being associated with the robot coordinate frame; the processor is in communication with the second robotic arm and the second imaging device, the processor: moving the second imaging device to a first position using the second robotic arm; capturing a first image with the second imaging device positioned at the first location; moving the second imaging device to a second position using the second robotic arm; capturing a second image with the second imaging device positioned at the second location; moving the second imaging device to a third position using the second robotic arm; capturing a third image with the second imaging device positioned at the third position; the second imaging device relative to the robot coordinate frame; the first image and a first kinematics of the second robot arm with the second imaging device in the first position; the second image and second kinematics of the second robot arm with the second imaging device in the second position; The system of embodiment 142, wherein the system is configured to calibrate based on the third image and a third kinematics of the second robot arm when the second imaging device is in the third position. (144) The system of embodiment 142, wherein the first imaging device comprises one of an ultrasound probe, a camera, or a depth sensor. (145) The system of embodiment 144, wherein the second imaging device comprises one of an ultrasound probe, a camera, or a depth sensor.

[0307] (146) A method for manufacturing a gyroscope comprising: an electromagnetic (EM) field generator configured to generate an EM field, the EM field generator being associated with an EM coordinate frame; and a third robotic arm configured to move to adjust a position of the EM field generator, the third robotic arm being associated with the robot coordinate frame; The processor, The system described in embodiment 143, further configured to determine alignment between the EM field coordinate frame and the robot coordinate frame based on the kinematics of the third robot arm. (147) The system described in embodiment 146, further comprising an EM sensor detectable within the EM field, and the processor is further configured to determine an EM sensor position within the robot coordinate frame based on the alignment. (148) The processor, Displaying an image from the first imaging device or the second imaging device; and The system of embodiment 147, further configured to overlay an indication of the determined EM sensor position on the displayed image.

Claims

1. A robotic medical system, comprising: a first robotic arm coupled to an electromagnetic field generator configured to generate an electromagnetic field (EM field); An EM sensor; one or more processors, sending commands to the first robot arm to move the electromagnetic field generator along a robot trajectory while the EM sensor remains in one position; Detecting an EM sensor trajectory of the EM sensor within the electromagnetic field corresponding to a period of time that the electromagnetic field generator moved along the robot trajectory; analyzing the robot trajectory and the EM sensor trajectory to determine a difference between the robot trajectory and the EM sensor trajectory; determining a presence of EM distortion at the location based at least in part on a difference between the robot trajectory and the EM sensor trajectory being greater than a threshold; and one or more processors configured to

2. The system of claim 1 , wherein the one or more processors are configured to determine the robot trajectory based on motion data corresponding to the first robotic arm.

3. 2. The system of claim 1, wherein the one or more processors are configured to determine the threshold value based on at least one of an error factor associated with a movement of the first robotic arm or an error factor associated with noise of the EM sensor.

4. The system of claim 1 , wherein the one or more processors are configured to determine a difference between the robot trajectory and the EM sensor trajectory based on a comparison of shapes of the robot trajectory and the EM sensor trajectory.

5. 2. The system of claim 1, wherein the one or more processors are configured to determine a difference between the robot trajectory and the EM sensor trajectory based on a comparison of a plurality of points along the robot trajectory to a corresponding plurality of points along the EM sensor trajectory.

6. The system of claim 5 , wherein each of the plurality of points is determined with respect to a time relative to a movement of the electromagnetic field generator.

7. 2. The system of claim 1, wherein the one or more processors are further configured, upon detecting a presence of the EM distortion at the location, to reposition the electromagnetic field generator on the first robotic arm to reduce the EM distortion at the location.

8. the one or more processors: sending additional commands to the first robot arm to move the electromagnetic field generator along a second robot trajectory while the EM sensor remains at the position; sensing a second EM sensor trajectory of the EM sensor within the electromagnetic field corresponding to a period of time that the electromagnetic field generator moved along the second robot trajectory; analyzing the second robot trajectory and the second EM sensor trajectory to determine a difference between the second robot trajectory and the second EM sensor trajectory; comparing a difference between the second robot trajectory and the second EM sensor trajectory with a difference between the robot trajectory and the EM sensor trajectory; 8. The system of claim 7, wherein the system is configured to reposition the electromagnetic field generator with the first robotic arm to reduce the EM distortion at the location.

9. 10. The system of claim 8, wherein the one or more processors are configured to position the electromagnetic field generator at a point along the second robot trajectory when a difference between the second robot trajectory and the second EM sensor trajectory is less than a difference between the robot trajectory and the EM sensor trajectory.

10. A robotic medical system, comprising: a first robotic arm coupled to an electromagnetic field generator configured to generate an electromagnetic field (EM field), the first robotic arm configured to move the electromagnetic field generator; A medical instrument comprising an EM sensor; one or more processors, determining that the EM sensor is located at a boundary of a predetermined region within the electromagnetic field; one or more processors to adjust a position of the electromagnetic field generator by commanding movement of the first robotic arm in response to determining that the EM sensor is located at a boundary of the predetermined region; A system comprising:

11. The system of claim 10 , wherein adjusting the position of the electromagnetic field generator places the EM sensor at a predetermined location within the electromagnetic field.

12. The system of claim 11 , wherein the predetermined location within the electromagnetic field comprises a center of a working volume of the electromagnetic field.

13. 11. The system of claim 10, wherein adjusting a position of the electromagnetic field generator based on determining a position of the EM sensor comprises adjusting a position of the electromagnetic field generator such that the EM sensor is located within a predetermined region of the electromagnetic field.

14. the one or more processors further comprising: determining an orientation of the EM sensor within the electromagnetic field; adjusting at least one of an orientation and a position of the electromagnetic field generator by commanding movement of the first robotic arm based on the determined orientation of the EM sensor. The system of claim 10 configured to:

15. 15. The system of claim 14, wherein the one or more processors are configured to adjust at least one of an orientation and a position of the electromagnetic field generator based on the determined orientation of the EM sensor to increase accuracy in determining a position of the EM sensor within the electromagnetic field.

16. further including a second robotic arm; the medical instrument is coupled to the second robotic arm; The system of claim 10 , wherein the one or more processors are configured to direct movement of the medical instrument with the second robotic arm.

17. 17. The system of claim 16, wherein the first and second robotic arms are coupled to one of a cart and a patient platform.

18. 20. The system of claim 17, further comprising a third robotic arm coupled to one of the cart and the patient platform.

19. the one or more processors: determining a position of the EM sensor within the electromagnetic field relative to an EM coordinate system associated with the electromagnetic field generator; determining a registration between the EM coordinate system and a robot coordinate system associated with the first robot arm based on determining a position of the electromagnetic field generator within the robot coordinate system; determining a position of the EM sensor in the robot coordinate system based on the registration; The system of claim 10 configured to:

20. 20. The system of claim 19, wherein the one or more processors are configured to determine a position of the electromagnetic field generator in the robot coordinate system based on kinematics of the first robot arm.

Images