Robot-controllable electromagnetic field generator

Abstract

This relates to a system having a robot-controllable electromagnetic field generator and its applications. [Solution] One application of a robot-controllable electromagnetic field generator is to use a robotic arm 12 to move an electromagnetic field generator coupled to the robotic arm 12, facilitating the setup of the electromagnetic field generator and tracking of the instrument. Another application of such a robot-controllable electromagnetic field generator is to use a robotic arm 12 to move an electromagnetic field generator coupled to it, facilitating the alignment of an insertable instrument with a target detected within the volume of the electromagnetic field generator. Another application of such a robot-controllable electromagnetic field generator is to use a robotic arm 12 to move an electromagnetic field generator coupled to it, detecting strain occurring within a medical system. Another application of such a robot-controllable electromagnetic field generator is to use a robotic arm 12 to facilitate multimodal sensor fusion.

Description

Technical Field

[0001] The systems and methods disclosed herein are directed to electromagnetic field generators for robotic medical systems, and more specifically, to electromagnetic field generators that can be positioned and / or controlled by a robot for detecting distortion in medical systems and related devices, systems, and methods.

Background Art

[0002] Medical procedures such as laparoscopy or endoscopy may involve accessing and visualizing an internal region of a patient. In a laparoscopic procedure, medical instruments can be inserted into the internal region through a laparoscopic access port. In an endoscopic procedure, a thin, flexible, tubular medical instrument can be inserted into the internal region through a natural opening of the patient. The medical instrument can include an end effector configured to perform a function during the procedure.

[0003] In certain procedures, a robotic medical system can be used to control the insertion and / or operation of medical instruments and end effectors. A robotic medical system can include a robotic arm or other instrument positioning device having a manipulator assembly used to control the positioning of the instrument during the procedure.

[0004] A robotic medical system can be configured to determine the position of a medical instrument based on the output of one or more position sensors that can be positioned on the medical instrument.

Summary of the Invention

Means for Solving the Problems

[0005] A robotic medical system may include an electromagnetic (EM) field generator configured to be coupled to (or otherwise integrated with) the system's robotic arm. Such an EM field generator can be considered robot-controllable or positionable because it can be controlled or repositioned using the robotic arm. The EM field generator can generate a magnetic field capable of determining the position of one or more EM sensors. Because the EM field generator is coupled to the robotic arm, the kinematics of the robotic arm can be used to establish alignment between the EM coordinate frame of the EM field generator and the system's robotic coordinate frame or global coordinate frame.

[0006] This configuration can reduce or eliminate the need for more complex alignment steps that may require user input to establish the relationship between the EM coordinate frame and the robot or global coordinate frame. This configuration can also improve the accuracy of determining the position of EM sensors and facilitate sensor fusion with additional modalities, for example, by improving the setting of electromagnetic field generators corresponding to robot arms, tracking medical instruments, and detecting strains that may affect system sensors.

[0007] These and other features and advantages of robot-controllable or positionable EM field generators are described in more detail below. Each of the systems, methods, and devices of this disclosure has several innovative aspects, none of which alone contribute to the desirable attributes disclosed herein. [Brief explanation of the drawing]

[0008] The disclosed embodiments will be described below in conjunction with the accompanying drawings provided, which are illustrative and not limiting, and similar designations will indicate similar elements. [Figure 1] This figure shows one embodiment of a cart-based robotic system arranged for diagnostic and / or therapeutic bronchoscopy, according to an exemplary embodiment. [Figure 2]This figure shows a further embodiment of the robot system of Figure 1, according to an exemplary embodiment. [Figure 3] This figure shows one embodiment of the robotic system of Figure 1, arranged for ureteroscopy, according to an exemplary embodiment. [Figure 4] This figure shows one embodiment of the robotic system of Figure 1, arranged for vascular procedures, according to an exemplary embodiment. [Figure 5] This figure shows one embodiment of a table-based robotic system positioned for bronchoscopy, according to an exemplary embodiment. [Figure 6] This is an alternative diagram of the robot system in Figure 5, according to an exemplary embodiment. [Figure 7] This figure shows an exemplary system configured to house a robotic arm, according to an exemplary embodiment. [Figure 8] This figure shows one embodiment of a table-based robotic system configured for ureteroscopy, according to an exemplary embodiment. [Figure 9] This figure shows one embodiment of a table-based robotic system configured for laparoscopic procedures, according to an exemplary embodiment. [Figure 10] This figure shows one embodiment of the table-based robot system shown in Figures 5 to 9, which has pitch or tilt adjustment, according to an exemplary embodiment. [Figure 11] Figures 5 to 10 show detailed diagrams of the interface between the table and columns of the table-based robot system according to an exemplary embodiment. [Figure 12] This figure shows an alternative embodiment of a table-based robot system, according to an exemplary embodiment. [Figure 13] Figure 12 is an end view of a table-based robot system according to an exemplary embodiment. [Figure 14] This is an end view of a table-based robot system with a robotic arm attached, according to an exemplary embodiment. [Figure 15]This figure shows an exemplary device driver according to an exemplary embodiment. [Figure 16] This figure shows an exemplary medical device having a pair of instrument drivers according to an exemplary embodiment. [Figure 17] This figure shows an exemplary embodiment of an alternative design for a tool driver and tool, in which the shaft of the drive unit is parallel to the shaft of the tool's elongated shaft. [Figure 18] This figure shows a device having a device-based insertion architecture according to an exemplary embodiment. [Figure 19] This figure shows an exemplary controller according to an exemplary embodiment. [Figure 20] This block diagram shows a localization system for estimating the location of one or more elements of the robot system shown in Figures 1 to 10, such as the location of the devices shown in Figures 16 to 18, according to an exemplary embodiment. [Figure 21] This figure shows an exemplary EM field generator that generates a magnetic field with a working volume. An exemplary EM sensor positioned within the working volume of the EM field generator is also shown according to an exemplary embodiment. [Figure 22A] This is a perspective view showing one embodiment of a robotic medical system including the EM field generator shown in Figure 21, according to an exemplary embodiment. [Figure 22B] This figure shows an exemplary alignment step that may be used to align the robot coordinate frame associated with the robotic medical system in Figure 22A with the EM coordinate system associated with the EM field generator, according to an exemplary embodiment. [Figure 22C] This figure shows an exemplary alignment step that may be used to align the robot coordinate frame associated with the robotic medical system in Figure 22A with the EM coordinate system associated with the EM field generator, according to an exemplary embodiment. [Figure 23] This is a perspective view showing one embodiment of a robotic medical system, which includes one embodiment of a robot-controllable electromagnetic field generator coupled to the robotic arm of the system, according to an exemplary embodiment. [Figure 24]Perspective view of one embodiment of a robot - controllable electromagnetic field generator configured to be coupled to a robotic arm, according to an exemplary embodiment. [Figure 25] Diagram showing that, according to an exemplary embodiment, the robot - controllable electromagnetic field generator of FIG. 24 can be configured to be coupled to an instrument drive mechanism of a robotic arm. [Figure 26] Flowchart showing one embodiment of a method for performing a robotic medical procedure using a robot - controllable electromagnetic field generator configured to be coupled to a robotic arm, according to an exemplary embodiment. [Figure 27A] Diagram showing one embodiment of a robotic medical system including a robot - controllable electromagnetic field generator configured for automatic instrument tracking and mapping of anatomical features, according to an exemplary embodiment. [Figure 27B] Diagram showing one embodiment of a robotic medical system including a robot - controllable electromagnetic field generator configured for automatic instrument tracking and mapping of anatomical features, according to an exemplary embodiment. [Figure 28A] Diagram showing one embodiment of a robotic medical system including a robot - controllable electromagnetic field generator configured to facilitate the setup and placement of the electromagnetic field generator and to expand its working volume, according to an exemplary embodiment. [Figure 28B] Diagram showing one embodiment of a robotic medical system including a robot - controllable electromagnetic field generator configured to facilitate the setup and placement of the electromagnetic field generator and to expand its working volume, according to an exemplary embodiment. [Figure 29] Block diagram representing one embodiment of a robotic medical system including a robot - controllable electromagnetic field generator coupled to a robotic arm of the system, according to an exemplary embodiment. [Figure 30A] Perspective view showing an example of moving an EM field generator using a robotic arm so that an EM sensor is positioned at a predetermined position within the EM field, according to an exemplary embodiment. [Figure 30B]This perspective view shows an example of moving an EM field generator using a robotic arm so that an EM sensor is positioned in a predetermined location within the EM field, according to an exemplary embodiment. [Figure 31A] This perspective view shows an example of moving an EM field generator using a robotic arm so that an EM sensor is positioned within a predetermined area of ​​the EM field, according to an exemplary embodiment. [Figure 31B] This perspective view shows an example of moving an EM field generator using a robotic arm so that an EM sensor is positioned within a predetermined area of ​​the EM field, according to an exemplary embodiment. [Figure 32A] This perspective view shows an example of moving an EM field generator using a robotic arm along a path that tracks the movement path of an EM sensor, according to an exemplary embodiment. [Figure 32B] This perspective view shows an example of moving an EM field generator using a robotic arm along a path that tracks the movement path of an EM sensor, according to an exemplary embodiment. [Figure 33A] This perspective view shows an example of an exemplary embodiment in which an EM field generator is moved using a robotic arm so that an EM sensor moving along a path remains positioned within a predetermined area of ​​the EM field. [Figure 33B] This perspective view shows an example of an exemplary embodiment in which an EM field generator is moved using a robotic arm so that an EM sensor moving along a path remains positioned within a predetermined area of ​​the EM field. [Figure 33C] This perspective view shows an example of an exemplary embodiment in which an EM field generator is moved using a robotic arm so that an EM sensor moving along a path remains positioned within a predetermined area of ​​the EM field. [Figure 33D] This perspective view shows an example of an exemplary embodiment in which an EM field generator is moved using a robotic arm so that an EM sensor moving along a path remains positioned within a predetermined area of ​​the EM field. [Figure 34]This flowchart provides an exemplary method for moving an EM field generator, coupled to a robotic arm, based on the determined position of an EM sensor within the EM field, according to an exemplary embodiment. [Figure 35A] This perspective view shows an example of moving an EM field generator to its position using a robotic arm based on the determined positions of multiple EM sensors within the EM field, according to an exemplary embodiment. [Figure 35B] This perspective view shows an example of moving an EM field generator to its position using a robotic arm based on the determined positions of multiple EM sensors within the EM field, according to an exemplary embodiment. [Figure 36A] This is a perspective view illustrating an example of readjusting the position of an EM field generator using a robotic arm based on the determined positions of multiple EM sensors within the EM field, in an exemplary embodiment, where at least one of the multiple EM sensors is moving. [Figure 36B] This is a perspective view illustrating an example of readjusting the position of an EM field generator using a robotic arm based on the determined positions of multiple EM sensors within the EM field, in an exemplary embodiment, where at least one of the multiple EM sensors is moving. [Figure 36C] This is a perspective view illustrating an example of readjusting the position of an EM field generator using a robotic arm based on the determined positions of multiple EM sensors within the EM field, in an exemplary embodiment, where at least one of the multiple EM sensors is moving. [Figure 37] This flowchart shows an exemplary method for determining the position of an electromagnetic field generator mounted on a robot arm, based on the determined positions of multiple EM sensors within the EM field, according to an exemplary embodiment. [Figure 38A] This figure shows one embodiment of a robotic medical system, which includes a robot-controllable electromagnetic field generator that can be moved with a robotic arm to expand the working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 38B]This figure shows one embodiment of a robotic medical system, which includes a robot-controllable electromagnetic field generator that can be moved with a robotic arm to expand the working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 38C] This figure shows one embodiment of a robotic medical system, which includes a robot-controllable electromagnetic field generator that can be moved with a robotic arm to expand the working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 39A] This figure shows one embodiment of a robotic medical system, which includes a robot-controllable electromagnetic field generator that can be moved with a robotic arm to detect the position of an EM sensor and expand the working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 39B] This figure shows one embodiment of a robotic medical system, which includes a robot-controllable electromagnetic field generator that can be moved with a robotic arm to detect the position of an EM sensor and expand the working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 39C] This figure shows one embodiment of a robotic medical system, which includes a robot-controllable electromagnetic field generator that can be moved with a robotic arm to detect the position of an EM sensor and expand the working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 39D] This figure shows one embodiment of a robotic medical system, which includes a robot-controllable electromagnetic field generator that can be moved with a robotic arm to detect the position of an EM sensor and expand the working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 40A] This figure shows one embodiment of a robotic medical system, which includes a robot-controllable electromagnetic field generator that can be moved with a robotic arm to track the position of an EM sensor within an extended working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 40B] This figure shows one embodiment of a robotic medical system, which includes a robot-controllable electromagnetic field generator that can be moved with a robotic arm to track the position of an EM sensor within an extended working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 40C] This figure shows one embodiment of a robotic medical system, which includes a robot-controllable electromagnetic field generator that can be moved with a robotic arm to track the position of an EM sensor within an extended working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 40D] This figure shows one embodiment of a robotic medical system, which includes a robot-controllable electromagnetic field generator that can be moved with a robotic arm to track the position of an EM sensor within an extended working volume of the electromagnetic field generator, according to an exemplary embodiment. [Figure 41A] This flowchart shows an exemplary method for extending the working volume of a robot-controllable electromagnetic field generator according to an exemplary embodiment. [Figure 41B] This flowchart shows another exemplary method for extending the working volume of a robot-controllable electromagnetic field generator, according to an exemplary embodiment. [Figure 42A] This figure shows one embodiment of a robotic medical system having a robot-controllable electromagnetic field generator configured to facilitate alignment between a percutaneously insertable device and an EM target, according to an exemplary embodiment. [Figure 42B] This figure shows one embodiment of a robotic medical system having a robot-controllable electromagnetic field generator configured to facilitate alignment between a percutaneously insertable device and an EM target, according to an exemplary embodiment. [Figure 42C] This figure shows one embodiment of a robotic medical system having a robot-controllable electromagnetic field generator configured to facilitate alignment between a percutaneously insertable device and an EM target, according to an exemplary embodiment. [Figure 43A] This figure shows one embodiment of a robotic medical system having an instrument guide mounted on an electromagnetic field generator and a robot-controllable electromagnetic field generator configured to facilitate alignment with an EM target, according to an exemplary embodiment. [Figure 43B]This figure shows one embodiment of a robotic medical system having an instrument guide mounted on an electromagnetic field generator and a robot-controllable electromagnetic field generator configured to facilitate alignment with an EM target, according to an exemplary embodiment. [Figure 44A] This figure shows one embodiment of a robotic medical system having a robot-controllable electromagnetic field generator configured to facilitate alignment between an instrument guide and an EM target, according to an exemplary embodiment. [Figure 44B] This figure shows one embodiment of a robotic medical system having a robot-controllable electromagnetic field generator configured to facilitate alignment between an instrument guide and an EM target, according to an exemplary embodiment. [Figure 45A] This flowchart shows a method, according to an exemplary embodiment, for aligning a percutaneously insertable device with an EM target using a robot-controllable electromagnetic field generator. [Figure 45B] This flowchart shows a method, according to an exemplary embodiment, for aligning an instrument guide for a percutaneously insertable instrument with an EM target using a robot-controllable electromagnetic field generator. [Figure 46A] This figure shows an embodiment of a robotic medical system in which an EM field generator is moved relative to a fixed EM position sensor in order to detect EM strain, according to an exemplary embodiment. [Figure 46B] This figure shows a comparison of the robot trajectory and the EM sensor trajectory of the EM field generator in Figure 46A during movement, according to an exemplary embodiment. [Figure 46C] This figure shows an embodiment of a robotic medical system in which an EM field generator is moved relative to a fixed EM position sensor to detect EM strain that can take into account changes in position and orientation, according to an exemplary embodiment. [Figure 47] This flowchart shows an exemplary method for EM strain detection according to an exemplary embodiment. [Figure 48] This flowchart shows another exemplary method for EM strain detection according to an exemplary embodiment. [Figure 49A] This figure shows a robotic medical system to which an ultrasound probe is attached during a procedure for calibrating the imaging surface of the ultrasound probe, according to an exemplary embodiment. [Figure 49B] This figure shows a robotic medical system including an EM field generator and an ultrasonic probe. According to an exemplary embodiment, the imaging plane of the ultrasonic probe is calibrated to a robot coordinate frame and an EM coordinate frame so that the positions of the robotic tool and EM sensor can be overlaid on the imaging plane. [Figure 50] This is a flowchart illustrating an exemplary method for calibrating the imaging plane of an ultrasound probe for use with a robotic medical system, according to an exemplary embodiment. [Figure 51A] This figure shows exemplary thermal and point maps generated using a depth sensor mounted on the robotic arm of a robotic medical system, according to an exemplary embodiment. [Figure 51B] This figure shows exemplary thermal and point maps generated using a depth sensor mounted on the robotic arm of a robotic medical system, according to an exemplary embodiment. [Figure 51C] This figure shows an exemplary embodiment of an exemplary depth sensor according to an exemplary embodiment. [Modes for carrying out the invention]

[0009] 1. Overview. Aspects of this disclosure may be integrated into a robot-enabled medical system capable of performing a variety of medical procedures, including both minimally invasive procedures such as laparoscopy and non-invasive procedures such as endoscopy. Among endoscopic procedures, the system can perform bronchoscopy, ureteroscopy, gastroscopy, and the like.

[0010] In addition to performing a wide range of procedures, the system can offer additional benefits to assist physicians, such as enhanced imaging and guidance. Furthermore, the system can provide physicians with the ability to perform procedures from an ergonomic position without requiring awkward arm movements and postures. Moreover, the system can provide physicians with improved ease of use, allowing one or more of the system's instruments to be controlled by a single user.

[0011] Various embodiments are described below, accompanied by drawings, for illustrative purposes. It should be understood that many other implementations of the disclosed concepts are possible, and various advantages may be achieved in the disclosed implementations. Headings are included herein for reference and to help locate the various sections. These headings are not intended to limit the scope of the concepts described therein. Such concepts may be applicable throughout this specification.

[0012] A. Robot system - cart. Robot-enabled medical systems can be configured in various ways depending on the specific procedure. Figure 1 shows one embodiment of a cart-based robot-enabled system 10 positioned for diagnostic and / or therapeutic bronchoscopy. During bronchoscopy, the system 10 may include a cart 11 having one or more robotic arms 12 for delivering medical instruments, such as a maneuverable endoscope 13 which may be a bronchoscope specifically designed for bronchoscopy, to a natural orifice access point for delivering the diagnostic and / or therapeutic instrument (i.e., the patient's mouth positioned on a table in this example). As shown, the cart 11 can be positioned close to the patient's upper torso to provide access to the access point. Similarly, the robotic arms 12 can be operated to position the bronchoscope relative to the access point. The configuration in Figure 1 can also be used when performing gastrointestinal (GI) procedures using a gastroscope, which is an endoscope specifically designed for GI procedures. Figure 2 shows an exemplary embodiment of the cart in more detail.

[0013] Continuing to refer to Figure 1, once the cart 11 is properly positioned, the robotic arm 12 can insert the maneuverable endoscope 13 into the patient robotically, manually, or a combination thereof. As shown, the maneuverable endoscope 13 may include at least two nesting parts, such as an inner leader portion and an outer sheath portion, each portion coupled to a separate instrument driver from a set of instrument drivers 28, each instrument driver coupled to the distal end of an individual robotic arm. This linear configuration of the instrument drivers 28, which facilitates coaxial alignment of the leader portion with the sheath portion, forms a “virtual rail” 29 that can be repositioned in space by manipulating one or more robotic arms 12 at different angles and / or positions. The virtual rail described herein is illustrated in the figure using dashed lines, and therefore the dashed lines do not illustrate the physical structure of the system. Translation of the instrument drivers 28 along the virtual rail 29 causes the inner leader portion to nest with the outer sheath portion, or moves the endoscope 13 forward or backward from the patient. The angle of the virtual rail 29 may be adjusted, translated, and pivoted based on clinical use or physician preference. For example, in bronchoscopy, the angle and position of the virtual rail 29 as shown represent a compromise that provides physician access to the endoscope 13 while minimizing friction caused by bending the endoscope 13 into the patient's mouth.

[0014] The endoscope 13 may be directed downstream of the patient's trachea and lungs after insertion using precise commands from the robotic system until it reaches the target destination or surgical site. To facilitate navigation through the patient's lung network and / or to reach the desired target, the endoscope 13 may be maneuvered to extend the inner leader portion in a nested manner from the outer sheath portion to obtain increased articulation and a larger bending radius. The use of a separate instrument driver 28 also allows the leader portion and the sheath portion to be driven independently of each other.

[0015] For example, the endoscope 13 may be directed to deliver a biopsy needle to a target, such as a lesion or nodule in the patient's lung. The needle may be deployed down the working channel along the length of the endoscope to obtain a tissue sample to be analyzed by a pathologist. Depending on the pathological results, additional tools may be deployed down the working channel of the endoscope for further biopsies. After identifying the nodule as malignant, the endoscope 13 may endoscopically deliver tools to excise the potentially cancerous tissue. In some cases, diagnostic and therapeutic procedures may be delivered during a separate procedure. In these situations, the endoscope 13 may also be used to deliver a criterion to “mark” the location of the target nodule. In other cases, diagnostic and therapeutic procedures may be delivered during the same procedure.

[0016] System 10 may also include a movable tower 30 connected to the cart 11 via support cables, which can provide support for control, electronics, fluid mechanics, optics, sensors, and / or power to the cart 11. Placing such functions within the tower 30 allows for a smaller form factor cart 11 that can be more easily adjusted and / or repositioned by the physician and their staff performing the procedure. Additionally, the separation of functions between the cart / table and the support tower 30 reduces clutter in the operating room and facilitates improvements in the clinical workflow. The cart 11 may be positioned close to the patient, while the tower 30 may be housed in a more distant location so as not to interfere during the procedure.

[0017] To support the robotic system described above, the tower 30 may include components of a computer-based control system that store computer program instructions in a non-temporary computer-readable storage medium, such as a persistent magnetic memory drive or a solid-state drive. The execution of these instructions may control the entire system or its subsystems, whether the execution takes place within the tower 30 or within the cart 11. For example, when executed by the processor of the computer system, the instructions may cause components of the robotic system to actuate the associated carriage and arm mount, actuate the robotic arm, and control medical devices. For example, in response to receiving a control signal, motors in the joints of the robotic arm may position the arm in a particular posture.

[0018] The tower 30 may also include a pump, flow meter, valve control, and / or fluid access to provide controlled irrigation and suction functions to a system that can be deployed through the endoscope 13. These components may also be controlled using the computer system of the tower 30. In some embodiments, the irrigation and suction capabilities may be delivered directly to the endoscope 13 via separate cables.

[0019] The tower 30 may include voltage and surge protectors designed to supply filtered and protected power to the cart 11, thereby avoiding the need to place power transformers and other auxiliary power components within the cart 11, resulting in the cart 11 being smaller and more portable.

[0020] Tower 30 may also include support equipment for sensors placed throughout the robot system 10. For example, Tower 30 may include optoelectronic equipment for detecting, receiving, and processing data received from optical sensors or cameras at any location in the robot system 10. In combination with a control system, such optoelectronic equipment may be used to generate real-time images for display in any number of consoles placed throughout the system, including within Tower 30. Similarly, Tower 30 may also include electronic subsystems for receiving and processing signals from placed electromagnetic (EM) sensors. Tower 30 may also be used to house and position EM field generators for detection by EM sensors within or on medical devices.

[0021] Tower 30 may also include console 31, in addition to other consoles available in the rest of the system, such as a console mounted on top of a cart. Console 31 may include a user interface and display screen, such as a touchscreen, for the operator, a physician. Consoles within System 10 are generally designed to provide both pre-operative and real-time information of the procedure, as well as robot control, and navigation and positioning information for the endoscope 13. If Console 31 is not the only console available to the physician, it may be used by a second operator, such as a nurse, to monitor the patient's health or vital signs and the operation of System 10, as well as to provide procedure-specific data such as navigation and positioning information. In other embodiments, Console 31 is housed in a separate body from Tower 30.

[0022] The tower 30 may be coupled to the cart 11 and the endoscope 13 via one or more cables or connectors (not shown). In some embodiments, the support functions from the tower 30 are provided to the cart 11 through a single cable, which simplifies and tidies up the operating room. In other embodiments, certain functions may be coupled with separate wiring and connectors. For example, power may be supplied to the cart 11 through a single power cable, while support for control, optics, fluid mechanics, and / or navigation may be provided through separate cables.

[0023] Figure 2 provides a detailed diagram of one embodiment of a cart 11 from the cart-based robot-enabled system shown in Figure 1. The cart 11 generally includes an elongated support structure 14 (often referred to as the “column”), a cart base 15, and a console 16 located at the top of the column 14. The column 14 may include one or more carriages, such as carriages 17 (alternatively referred to as “arm supports”), for supporting the deployment of one or more robot arms 12 (three are shown in Figure 2). The carriage 17 may include individually configurable arm mounts that rotate along a vertical axis to adjust the base of the robot arm 12 for better positioning relative to the patient. The carriage 17 also includes a carriage interface 19 that allows the carriage 17 to translate vertically along the column 14.

[0024] The carriage interface 19 is connected to the column 14 through slots such as slots 20 positioned on both sides of the column 14 to guide the vertical translation of the carriage 17. The slots 20 include a vertical translation interface for positioning and holding the carriage 17 at various vertical heights relative to the cart base 15. The vertical translation of the carriage 17 allows the cart 11 to adjust the reach of the robotic arm 12 to meet various table heights, patient sizes, and physician preferences. Similarly, individually configurable arm mounts on the carriage 17 allow the robotic arm base 21 of the robotic arm 12 to be angled in various configurations.

[0025] In some embodiments, a slot cover may be added to the slot 20, which is coplanar and parallel to the slot surface, to prevent dirt and fluid from entering the internal chamber of the column 14 and the vertical translation interface as the carriage 17 translates vertically. The slot cover may be deployed through a pair of spring spools positioned near the vertical top and bottom of the slot 20. The cover is coiled within the spools until it unfolds to expand and contract from a coiled state as the carriage 17 translates vertically up and down. The spring load of the spools provides a force to retract the cover into the spool as the carriage 17 translates toward the spools, while also maintaining a seal when the carriage 17 translates toward the spools. The cover may be connected to the carriage 17 using, for example, a bracket within the carriage interface 19, to ensure that the cover expands and contracts appropriately as the carriage 17 translates.

[0026] Column 14 may include internal mechanisms such as gears and motors, designed to use vertically aligned main screws to mechanically translate the carriage 17 in response to control signals generated in response to user input, such as input from a console 16.

[0027] The robotic arm 12 may generally include a robotic arm base 21 and end effectors 22, separated by a series of links 23 connected by a series of joints 24, each joint including an independent actuator, and each actuator including an independently controllable motor. Each independently controllable joint represents an independent degree of freedom available to the robotic arm 12. Each of the robotic arms 12 may have seven joints and thus be able to provide seven degrees of freedom. A large number of joints results in a large number of degrees of freedom, enabling "redundant" degrees of freedom. By having redundant degrees of freedom, the robotic arm 12 can position its respective end effectors 22 in a specific position, orientation, and trajectory in space using different link positions and joint angles. This allows the system to position and orient medical instruments from a desired point in space, while simultaneously allowing the physician to move the arm joints to clinically advantageous positions away from the patient, creating better access while avoiding arm collisions.

[0028] The cart base 15 balances the weight of the column 14 on the floor, the carriage 17, and the robotic arm 12. Therefore, the cart base 15 accommodates heavier components such as electronics, motors, power supplies, and components that enable either movement and / or fixation of the cart 11. For example, the cart base 15 includes casters 25 with rotatable wheels, allowing the cart 11 to be easily moved around the room before treatment. Once in a suitable position, the casters 25 may be prevented from moving using wheel locks to hold the cart 11 in place during treatment.

[0029] Positioned at the vertical end of column 14, the console 16 provides both a user interface and a display screen (or a dual-purpose device such as a touchscreen 26) for receiving user input, and provides the user, a physician, with both preoperative and intraoperative data. Potential preoperative data on the touchscreen 26 may include preoperative planning, navigation and mapping data derived from preoperative computed tomography (CT scan), and / or notes from preoperative patient interviews. Intraoperative data on the display may include optical information provided by tools, sensor and coordinate information from sensors, and essential patient statistics such as respiration, heart rate, and / or pulse. The console 16 may be positioned and tilted to allow the physician to access the console 16 from the opposite side of the carriage 17, on the column 14. From this position, the physician can see the console 16, the robotic arm 12, and the patient while operating the console 16 from behind the cart 11. As shown, the console 16 also includes a handle 27 to assist in the operation and stabilization of the cart 11.

[0030] Figure 3 shows one embodiment of a robot-enabled system 10 configured for ureteroscopy. In a ureteroscopy procedure, the cart 11 may be positioned to deliver a ureteroscope 32, a procedure-specific endoscope designed to traverse the patient's urethra and ureters, to the patient's lower abdominal region. In ureteroscopy, it is sometimes desirable that the ureteroscope 32 be directly aligned with the patient's urethra to reduce friction and force on sensitive anatomical structures in the region. As shown, the cart 11 may be aligned with the legs of a table to allow a robotic arm 12 to position the ureteroscope 32 for direct, linear access to the patient's urethra. From the legs of the table, the robotic arm 12 may insert the ureteroscope 32 along a virtual rail 33 directly into the patient's lower abdomen through the urethra.

[0031] After being inserted into the urethra using control techniques similar to those used in bronchoscopy, the ureteroscope 32 may be navigated to the bladder, ureters, and / or kidneys for diagnostic and / or therapeutic purposes. For example, the ureteroscope 32 may be directed towards the ureters and kidneys, and a laser or ultrasonic lithotripsy device deployed below the working channel of the ureteroscope 32 may be used to break up any formed kidney stones. After the lithotripsy is complete, the resulting stone fragments may be removed using a basket deployed below the ureteroscope 32.

[0032] Figure 4 shows one embodiment of the robot-enabled system 10, similarly positioned for vascular procedures. In vascular procedures, the system 10 may be configured such that a cart 11 can deliver a medical instrument 34, such as a maneuverable catheter, to an access point in the femoral artery within the patient's leg. The femoral artery presents both a larger diameter for navigation and a relatively less detourable, winding path to the patient's heart, thereby simplifying navigation. As in ureteroscopy procedures, the cart 11 may be positioned toward the patient's leg and lower abdomen to allow a robotic arm 12 to provide a virtual rail 35 with direct linear access to the femoral artery access point in the patient's thigh / hip region. After insertion into the artery, the medical instrument 34 may be directed and inserted by translating the instrument driver 28. Alternatively, the cart may be positioned around the patient's upper abdomen to reach alternative vascular access points, such as the carotid and brachial arteries near the shoulder and wrist.

[0033] B. Robot system - Table. Embodiments of robot-enabled medical systems may also incorporate a patient table. Incorporating a table reduces the amount of capital equipment in the operating room by eliminating carts, thereby allowing for better access to the patient. Figure 5 shows one embodiment of such a robot-enabled system arranged for a bronchoscopy procedure. System 36 includes a support structure or column 37 for supporting a platform 38 (illustrated as “table” or “bed”) above the floor. Similar to cart-based systems, the end effector of the robotic arm 39 of system 36 includes instrument drivers 42 designed to manipulate elongated medical instruments, such as the bronchoscope 40 in Figure 5, through or along a virtual rail 41 formed from a linear array of instrument drivers 42. In practice, a C-arm for providing fluoroscopic imaging may be positioned above the upper abdominal region of the patient by placing the radiator and detector around the table 38.

[0034] Figure 6 provides an alternative diagram of the system 36 without a patient and medical equipment for illustrative purposes. As shown, the column 37 may include one or more carriages 43, illustrated as a ring shape within the system 36, which may serve as the base for one or more robotic arms 39. The carriages 43 may translate along a vertical column interface 44 along the length of the column 37 to provide different bandage points from which the robotic arms 39 can be positioned to reach a patient. The carriages 43 may rotate around the column 37 using a mechanical motor positioned within the column 37 to allow the robotic arms 39 to have access to multiple sides of a table 38, such as both sides of a patient. In embodiments with multiple carriages, the carriages may be individually positioned on the column and may translate and / or rotate independently of each other. The carriages 43 do not need to surround the column 37, nor do they need to be circular, but the illustrated ring shape facilitates the rotation of the carriages 43 around the column 37 while maintaining structural balance. The rotation and translation of the carriage 43 allows the system 36 to align medical instruments such as endoscopes and laparoscopes to different access points on the patient. In other embodiments (not shown), the system 36 may include a patient table or bed having adjustable arm supports in the form of parallel-extending bars or rails. One or more robotic arms 39 can be mounted on adjustable arm supports that can be adjusted vertically (e.g., via shoulders having elbow joints). By providing vertical adjustment, the robotic arms 39 can advantageously be compactly housed under the patient table or bed and then raised during treatment.

[0035] The robot arm 39 may be mounted on the carriage 43 via a set of arm mounts 45, which include a series of joints that can rotate individually and / or extend in a nesting manner, to provide additional configurability for the robot arm 39. Additionally, the arm mounts 45 may be positioned on the carriage 43 such that, when the carriage 43 is rotated appropriately, the arm mounts 45 can be positioned on the same side of the table 38 (as shown in Figure 6), on both sides of the table 38 (as shown in Figure 9), or on adjacent sides of the table 38 (not shown).

[0036] Column 37 structurally provides support for the table 38 and a path for the vertical translation of the carriage 43. Internally, column 37 may include a main screw for guiding the vertical translation of the carriage and a motor for mechanizing the translation of the carriage 43 based on the main screw. Column 37 may also transmit power and control signals to the carriage 43 and the robotic arm 39 mounted on it.

[0037] The table base 46 performs a similar function to the cart base 15 of the cart 11 shown in Figure 2, and accommodates heavier components to balance the table / bed 38, column 37, carriage 43, and robot arm 39. The table base 46 may also incorporate rigid casters to provide stability during treatment. Casters extending from the bottom of the table base 46 extend in opposite directions on both sides of the base 46 and may retract when it is necessary to move the system 36.

[0038] Continuing to refer to Figure 6, the system 36 may also include a tower (not shown) that divides the functions of the system 36 between the table and the tower to reduce the form factor and bulk of the table. Similar to the embodiments disclosed previously, the tower may provide the table with various support functions such as processing, computing, and control capabilities, power, fluid mechanics, and / or optical and sensor processing. The tower may also be movable to be positioned away from the patient to improve physician access and keep the operating room tidy. Additionally, by arranging components within the tower, it is possible to increase the storage space within the table base 46 for the potential accommodation of a robotic arm 39. The tower may also include a master controller or console that provides both a user interface for user input such as a keyboard and / or pendant, and a display screen (or touchscreen) for preoperative and intraoperative information such as real-time imaging, navigation, and tracking information. In some embodiments, the tower may also include a holder for a gas tank used for air delivery.

[0039] In some embodiments, the table base may house and store a robot arm when not in use. Figure 7 shows a system 47 for housing a robot arm in one embodiment of a table base system. In system 47, the carriage 48 may be translated vertically into the base 49 to house the robot arm 50, arm mount 51, and carriage 48 within the base 49. The base cover 52 may be translated and retracted to open, allowing the carriage 48, arm mount 51, and robot arm 50 to unfold around the column 53, and to close to house and protect them when not in use. The base cover 52 may be sealed with a membrane 54 along the edge of its opening to prevent the ingress of dirt and fluid when closed.

[0040] Figure 8 shows one embodiment of a robot-enabled table-based system configured for a ureteroscopy procedure. In a ureteroscopy, the table 38 may include a swivel portion 55 for positioning the patient off-angle from the column 37 and the table base 46. The swivel portion 55 may rotate or pivot around a pivot point (e.g., located below the patient's head) to position the bottom of the swivel portion 55 away from the column 37. For example, pivoting the swivel portion 55 allows a C-arm (not shown) to be positioned above the patient's lower abdomen without competing for space with the column (not shown) below the table 38. By rotating a carriage (not shown) around the column 37, the robotic arm 39 may insert the ureteroscope 56 directly into the patient's inguinal region along a virtual rail 57 to reach the urethra. In a ureteroscopy, stirrups 58 may also be fixed to the swivel portion 55 of the table 38 to support the patient's leg position during the procedure and to allow clear access to the patient's inguinal region.

[0041] In laparoscopic procedures, minimally invasive instruments may be inserted into the patient's anatomical structures through small incisions in the patient's abdominal wall. In some embodiments, the minimally invasive instruments include elongated rigid members, such as shafts, used to access anatomical structures within the patient. After the patient's abdominal cavity is expanded, the instruments may be directed to perform surgical or medical tasks such as grasping, cutting, ablation, and suturing. In some embodiments, the instruments may include scopes, such as laparoscopes. Figure 9 shows one embodiment of a robot-enabled table-based system configured for laparoscopic procedures. As illustrated in Figure 9, the carriage 43 of the system 36 may rotate and be vertically adjusted to position a pair of robotic arms 39 on either side of the table 38 so that instruments 59 can be positioned using arm mounts 45 to pass through minimal incisions on either side of the patient and reach the patient's abdominal cavity.

[0042] To accommodate laparoscopic procedures, the robot-enabled table system may also tilt the platform to a desired angle. Figure 10 shows one embodiment of a robot-enabled medical system having pitch or tilt adjustment. As shown in Figure 10, the system 36 can adapt to the tilt of the table 38 to position one part of the table at a greater distance from the floor than the other part. Additionally, the arm mount 45 may rotate to match the tilt so that the robot arm 39 maintains the same planar relationship as the table 38. To adapt to steeper angles, the column 37 may also include a nested section 60 that allows the column 37 to extend vertically to prevent the table 38 from contacting the floor or colliding with the table base 46.

[0043] Figure 11 provides a detailed diagram of the interface between the table 38 and the column 37. The pitch rotation mechanism 61 may be configured to change the pitch angle of the table 38 relative to the column 37 in multiple degrees of freedom. The pitch rotation mechanism 61 may also be enabled by positioning orthogonal axes 1 and 2 at the column-table interface, each axis being actuated by separate motors 3 and 4 in response to an electric pitch angle command. Rotation along one screw 5 allows for tilt adjustment along one axis 1, and rotation along the other screw 6 allows for tilt adjustment along the other axis 2. In some embodiments, ball joints may be used to change the pitch angle of the table 38 relative to the column 37 in multiple degrees of freedom.

[0044] For example, pitch adjustment is particularly useful when positioning the table in the Trendelenburg position, that is, when positioning the patient's lower abdomen higher off the floor than the patient's upper abdomen for lower abdominal surgery. In the Trendelenburg position, the patient's internal organs slide towards the patient's upper abdomen due to gravity, emptying the abdominal cavity so that minimally invasive tools can be inserted for lower abdominal surgical or medical procedures such as laparoscopic prostatectomy.

[0045] Figures 12 and 13 show isometric and end views of alternative embodiments of the table-based surgical robot system 100. The surgical robot system 100 includes one or more adjustable arm supports 105 (see, for example, Figure 14) which can be configured to support one or more robot arms relative to the table 101. In the illustrated embodiment, a single adjustable arm support 105 is shown, but additional arm supports 105 can be provided on the opposite side of the table 101. The adjustable arm supports 105 can be moved relative to the table 101 to adjust and / or change the position of the adjustable arm support 105 and / or any robot arm mounted thereon relative to the table 101. For example, the adjustable arm support 105 can be adjusted with one or more degrees of freedom relative to the table 101. The adjustable arm supports 105 provide the system 100 with high versatility, including the ability to easily accommodate one or more adjustable arm supports 105 and any robot arms mounted thereon under the table 101. The adjustable arm support 105 can be raised from its retracted position to a position below the upper surface of the table 101. In other embodiments, the adjustable arm support 105 can be raised from its retracted position to a position above the upper surface of the table 101.

[0046] The adjustable arm support 105 can provide multiple degrees of freedom, including lift, lateral translation, and tilt. In the illustrated embodiments of Figures 12 and 13, the arm support 105 comprises four degrees of freedom, which are indicated by arrows in Figure 12. The first degree of freedom allows for adjustment of the adjustable arm support 105 in the z-direction ("Z-lift"). For example, the adjustable arm support 105 may include a carriage 109 configured to move up and down along or relative to a column 102 supporting the table 101. The second degree of freedom allows the adjustable arm support 105 to tilt. For example, the adjustable arm support 105 may include a rotational joint, which may allow the adjustable arm support 105 to be aligned with a Trendelenburg position bed. The third degree of freedom may allow the adjustable arm support 105 to "pivot upward," which can be used to adjust the distance between the side of the table 101 and the adjustable arm support 105. The fourth degree of freedom allows the adjustable arm support 105 to translate along the longitudinal length of the table.

[0047] The surgical robot system 100 shown in Figures 12 and 13 may include a table supported by a column 102 mounted on a base 103. The base 103 and column 102 support the table 101 with respect to a support surface. The floor axis 131 and support axis 133 are shown in Figure 13.

[0048] The adjustable arm support 105 can be mounted on the column 102. In other embodiments, the arm support 105 can be mounted on the table 101 or the base 103. The adjustable arm support 105 may include a carriage 109, a bar or rail connector 111, and a bar or rail 107. In some embodiments, one or more robot arms mounted on the rail 107 can translate and move relative to each other.

[0049] The carriage 109 may be attached to the column 102 by a first joint 113, thereby allowing the carriage 109 to move relative to the column 102 (e.g., up and down on the first or vertical axis 123). The first joint 113 can provide a first degree of freedom ("Z-lift") to the adjustable arm support 105. The adjustable arm support 105 may include a second joint 115 that provides a second degree of freedom (tilt) to the adjustable arm support 105. The adjustable arm support 105 may include a third joint 117 that can provide a third degree of freedom ("upward pivot") to the adjustable arm support 105. An additional joint 119 (shown in Figure 13) may be provided to mechanically restrain the third joint 117 to maintain the orientation of the rail 107 when the rail connector 111 is rotated around a third axis 127. The adjustable arm support 105 may include a fourth joint 121 that can provide the adjustable arm support 105 with a fourth degree of freedom (translation) along a fourth axis 129.

[0050] Figure 14 shows an end view of a surgical robotic system 140A having two adjustable arm supports 105A and 105B mounted on either side of a table 101. A first robotic arm 142A is attached to a bar or rail 107A of the first adjustable arm support 105B. The first robotic arm 142A includes a base 144A attached to the rail 107A. The distal end of the first robotic arm 142A includes an instrument drive mechanism 146A that can be coupled to one or more robotic medical instruments or tools. Similarly, a second robotic arm 142B includes a base 144B attached to the rail 107B. The distal end of the second robotic arm 142B includes an instrument drive mechanism 146B. The instrument drive mechanism 146B may be configured to be coupled to one or more robotic medical instruments or tools.

[0051] In some embodiments, one or more of the robot arms 142A, 142B include an arm having seven or more degrees of freedom. In some embodiments, one or more of the robot arms 142A, 142B may include eight degrees of freedom, including an insertion axis (one degree of freedom including insertion), a wrist (three degrees of freedom including wrist pitch, yaw, and roll), an elbow (one degree of freedom including elbow pitch), a shoulder (two degrees of freedom including shoulder pitch and yaw), and a base 144A, 144B (one degree of freedom including translation). In some embodiments, the insertion degrees of freedom can be provided by the robot arms 142A, 142B, but in other embodiments, the instrument itself provides insertion via an instrument-based insertion architecture.

[0052] C. Equipment drivers and interfaces. The end effector of the system's robotic arm may include (i) an instrument driver (alternatively called an “instrument drive mechanism” or “instrument device manipulator”) incorporating electromechanical means for operating a medical instrument, and (ii) a removable or detachable medical instrument, which may lack any electromechanical components such as a motor. This dichotomy may arise from the need to sterilize medical instruments used in medical procedures and the inability to properly sterilize expensive capital equipment due to the complexity of the mechanical assembly of medical instruments and the sensitivity of electronic components. Therefore, medical instruments may be designed to be removed, detached, and replaced from the instrument driver (and thus its system) for individual sterilization or disposal by the physician or physician's staff. In contrast, the instrument driver does not need to be replaced or sterilized and may be covered for protection.

[0053] Figure 15 shows an exemplary instrument driver. Positioned at the distal end of a robotic arm, the instrument driver 62 includes one or more drive units 63 arranged with parallel axes to provide controlled torque to a medical instrument via a drive shaft 64. Each drive unit 63 comprises an individual drive shaft 64 for interacting with the instrument, a gearhead 65 for converting the rotation of the motor shaft into a desired torque, a motor 66 for generating the drive torque, an encoder 67 for measuring the speed of the motor shaft and providing feedback to the control circuit, and a control circuit 68 for receiving a control signal and operating the drive unit. Each drive unit 63 is independently controlled and motorized, and the instrument driver 62 can provide multiple (e.g., four as shown in Figure 15) independent drive outputs to the medical instrument. During operation, the control circuit 68 receives a control signal, transmits a motor signal to the motor 66, compares the resulting motor speed measured by the encoder 67 with a desired speed, modulates the motor signal to generate the desired torque.

[0054] For procedures requiring a sterile environment, the robotic system may incorporate a drive interface, such as a sterilization adapter connected to a sterilization drape, positioned between the instrument driver and the medical instrument. The primary purpose of the sterilization adapter is to transmit angular motion from the instrument driver's drive shaft to the instrument's drive input while maintaining physical separation between the drive shaft and the drive input, and thus sterility. Thus, an exemplary sterilization adapter may include a series of rotary inputs and outputs intended to be mated with the instrument driver's drive shaft, and a drive input to the instrument. The sterilization drape connected to the sterilization adapter is made of a thin, flexible material, such as transparent or translucent plastic, and is designed to cover the instrument driver, the robotic arm, and capital equipment such as a cart (in a cart-based system) or a table (in a table-based system). The use of the drape allows the capital equipment to be positioned close to the patient while still being located in an area where sterilization is not required (i.e., a non-sterilized field). On the other side of the sterilization drape, the medical instrument may interface with the patient in an area where sterilization is required (i.e., a sterile field).

[0055] D. Medical devices. Figure 16 shows an exemplary medical instrument with a pair of instrument drivers. Like other instruments designed for use with robotic systems, the medical instrument 70 includes an elongated shaft 71 (or elongated body) and an instrument base 72. The instrument base 72, also referred to as an “instrument handle” due to its design intended for manual interaction by a physician, may include a rotatable drive input 73, such as a receptacle, pulley, or spool, which is generally designed to mate with a drive output 74 extending through a drive interface on the instrument driver 75 at the distal end of a robotic arm 76. When physically connected, latched, and / or coupled, the mated drive input 73 of the instrument base 72 may share a rotation axis with the drive output 74 in the instrument driver 75, allowing for the transmission of torque from the drive output 74 to the drive input 73. In some embodiments, the drive output 74 may include a spline designed to mate with a receptacle on the drive input 73.

[0056] The elongated shaft 71 is designed to be delivered through either an anatomical opening or lumen, such as in an endoscope, or a minimally invasive incision, such as in a laparoscopy. The elongated shaft 71 may be either flexible (e.g., having properties similar to an endoscope) or rigid (e.g., having properties similar to a laparoscope), or may include a customized combination of both flexible and rigid portions. When designed for laparoscopy, the distal end of a rigid elongated shaft may be connected to an end effector extending from a joined list formed from a clevis having at least one degree of freedom, and to a surgical tool or medical instrument, such as a gripper or scissors, which can be actuated based on force from a tendon as the drive input rotates in response to torque received from the drive output 74 of the instrument driver 75. When designed for endoscopy, the distal end of a flexible elongated shaft may include a maneuverable or controllable bend that can articulate and flex based on torque received from the drive output 74 of the instrument driver 75.

[0057] Torque from the instrument driver 75 is transmitted downstream of the elongated shaft 71 via tendons along the elongated shaft 71. These individual tendons, such as pull wires, may be individually fixed to individual drive inputs 73 within the instrument handle 72. From the instrument handle 72, the tendons are directed downward along the elongated shaft 71 to one or more pull lumens and fixed to the distal portion of the elongated shaft 71 or to the wrist of the distal portion of the elongated shaft. During surgical procedures such as laparoscopy, endoscopy, or hybrid procedures, these tendons may be coupled to distally attached end effectors such as wrists, grippers, or scissors. Under such configurations, torque applied to the drive inputs 73 transmits tension to the tendons, thereby acting on the end effectors in some way. In some embodiments, during surgical procedures, the tendons can rotate the joint around an axis, thereby moving the end effectors in one direction or another. Alternatively, the tendon may be connected to one or more jaws of the gripping device at the distal end of the elongated shaft 71, and the gripping device will be closed by tension from the tendon.

[0058] In endoscopy, tendons may be coupled to a flexure or articulation portion positioned along an elongated shaft 71 (e.g., at the distal end) via adhesive, a control ring, or other mechanical fixation. When fixedly attached to the distal end of a flexure, the torque applied to the drive input 73 is transmitted downstream of the tendon, causing the softer flexure portion (sometimes called the articulation portion or articulation region) to flex or articulate. Along the non-flexure portion, it may be advantageous to equilibrium the radial forces resulting from the tension in the pull wire by making the individual pull lumens that orient the individual tendons along (or inward) the wall of the endoscope shaft spiral or spiral. The angle of the spiral and / or spacing between these can be modified or designed for a particular purpose, with tighter spirals resulting in less shaft compression under load, while smaller amounts of spirals result in greater shaft compression under load but limit flexion. At the other end of the spectrum, the lumen may be oriented parallel to the longitudinal axis of the elongated shaft 71 to enable controlled articulation at the desired flexed or articulated portion.

[0059] In endoscopic procedures, the elongated shaft 71 houses several components that support robotic procedures. The shaft 71 may include working channels for deploying, irrigating, and / or aspirating surgical tools (or medical instruments) to the surgical area at the distal end of the shaft 71. The shaft 71 may also house wires and / or optical fibers for sending and receiving signals to and from an optical assembly at the distal tip, which may include an optical camera. The shaft 71 may also house optical fibers for transporting light from a proximal light source, such as a light-emitting diode, to the distal end of the shaft 71.

[0060] At the distal end of the instrument 70, the distal tip may include an opening for a working channel for delivering the tool to the surgical site for diagnosis and / or treatment, irrigation, and aspiration. The distal tip may also include a port for a camera, such as a fiberscope or digital camera, to capture images of the internal anatomical space. In connection with this, the distal tip may also include a port for a light source to illuminate the anatomical space when a camera is used.

[0061] In the example shown in Figure 16, the drive shaft axis, i.e., the drive input axis, is perpendicular to the axis of the elongated shaft 71. However, this configuration complicates the rolling capability of the elongated shaft 71. Rolling the elongated shaft 71 along its axis while the drive input 73 is stationary results in undesirable entanglement of the tendon as it extends from the drive input 73 and enters the lumen within the elongated shaft 71. Such resulting entanglement of the tendon can interfere with control algorithms intended to predict the movement of the flexible elongated shaft 71 during endoscopic procedures.

[0062] Figure 17 shows an alternative design for an instrument driver and instrument in which the axis of the drive unit is parallel to the axis of the instrument's elongated shaft. As shown, the circular instrument driver 80 includes four drive units, each having a drive output unit 81 aligned in parallel at the end of a robot arm 82. The drive units and their respective drive output units 81 are housed in a rotary assembly 83 of the instrument driver 80, which is driven by one of the drive units in the assembly 83. In response to the torque provided by the rotary drive unit, the rotary assembly 83 rotates along a circular bearing that connects the rotary assembly 83 to the non-rotating portion 84 of the instrument driver 80. Power and control signals may be transmitted from the non-rotating portion 84 of the instrument driver 80 to the rotary assembly 83 via electrical contacts that can be maintained through rotation by brushed slip ring connections (not shown). In other embodiments, the rotary assembly 83 may be integrated into the non-rotatable portion 84 and therefore respond to a separate drive unit that is not parallel to the other drive units. The rotation mechanism 83 enables the instrument driver 80 to rotate the drive unit and their respective drive output units 81 as a single unit around the instrument driver shaft 85.

[0063] Similar to the embodiments disclosed previously, the instrument 86 may include an elongated shaft portion 88 and an instrument base 87 (shown by a transparent outer skin for illustrative purposes) which includes a plurality of drive inputs 89 (such as receptacles, pulleys, and spools) configured to receive a drive output portion 81 in the instrument driver 80. Unlike the embodiments disclosed previously, the instrument shaft 88 extends from the center of the instrument base 87, and its axis is substantially parallel to the axis of the drive inputs 89, rather than orthogonal as shown in the design of Figure 16.

[0064] When coupled to the rotary assembly 83 of the instrument driver 80, the medical instrument 86, including the instrument base 87 and instrument shaft 88, rotates together with the rotary assembly 83 around the instrument driver axis 85. Since the instrument shaft 88 is positioned at the center of the instrument base 87, the instrument shaft 88 is coaxial with the instrument driver axis 85 when mounted. Therefore, the rotation of the rotary assembly 83 causes the instrument shaft 88 to rotate around its own longitudinal axis. Furthermore, as the instrument base 87 rotates together with the instrument shaft 88, none of the tendons connected to the drive input 89 within the instrument base 87 become entangled during rotation. Thus, the parallelism of the axes of the drive output 81, the drive input 89, and the instrument shaft 88 allows the shaft to rotate without any control tendons becoming entangled.

[0065] Figure 18 shows a device 150 having an insertable device base architecture according to one embodiment. The device 150 may be coupled to one of the device drivers described above. The device 150 includes an elongated shaft 152, an end effector 162 connected to the shaft 152, and a handle 170 coupled to the shaft 152. The elongated shaft 152 includes a tubular member having a proximal portion 154 and a distal portion 156. The elongated shaft 152 includes one or more channels or grooves 158 along its outer surface. The grooves 158 are configured to receive one or more wires or cables 180 through them. Thus, one or more cables 180 extend along the outer surface of the elongated shaft 152. In other embodiments, the cables 180 may also extend through the elongated shaft 152. Operation of one or more of the cables 180 (e.g., via a device driver) results in the operation of the end effector 162.

[0066] The fixture handle 170, which may also be referred to as the fixture base, may generally include one or more mechanical input parts 174, such as a mounting interface 172 having a receptacle, pulley, or spool, designed to reciprocately engage with one or more torque couplers on the mounting surface of the fixture driver.

[0067] In some embodiments, the instrument 150 includes a series of pulleys or cables that allow an elongated shaft 152 to translate relative to the handle 170. In other words, the instrument 150 itself includes an instrument-based insertion architecture that adapts to the insertion of the instrument, thereby minimizing reliance on a robotic arm to provide insertion of the instrument 150. In other embodiments, a robotic arm may be significantly involved in the insertion of the instrument.

[0068] E. Controller. Any of the robotic systems described herein may include an input device or controller for operating an instrument attached to a robotic arm. In some embodiments, the controller may be coupled to the instrument (e.g., communicatively, electronically, electrically, wirelessly, and / or mechanically) so that operation of the controller triggers a corresponding operation of the instrument, for example, via master-slave control.

[0069] Figure 19 is a perspective view of one embodiment of the controller 182. In this embodiment, the controller 182 includes a hybrid controller that can have both impedance control and admittance control. In other embodiments, the controller 182 can utilize only impedance or passive control. In other embodiments, the controller 182 can utilize only admittance control. Being a hybrid controller, the controller 182 can advantageously have lower perceived inertia during use.

[0070] In the illustrated embodiment, the controller 182 is configured to enable the operation of two medical devices and includes two handles 184. Each of the handles 184 is connected to a gimbal 186. Each gimbal 186 is connected to a positioning platform 188.

[0071] As shown in Figure 19, each positioning platform 188 includes a selective compliance assembly robot arm (SCARA) 198 coupled to a column 194 by a prism-shaped joint 196. The prism-shaped joint 196 is configured to translate along the column 194 (e.g., along a rail 197) so that each of the handles 184 is translated in the z-direction, providing a first degree of freedom. The SCARA 198 is configured to allow the movement of the handles 184 in the xy-plane, providing two additional degrees of freedom.

[0072] In some embodiments, one or more load cells are positioned within the controller. For example, in some embodiments, load cells (not shown) are positioned within each body of the gimbal 186. By providing load cells, a portion of the controller 182 can operate under admittance control, thereby advantageously reducing the perceived inertia of the controller during use. In some embodiments, the positioning platform 188 is configured for admittance control and the gimbal 186 is configured for impedance control. In other embodiments, the gimbal 186 is configured for admittance control and the positioning platform 188 is configured for impedance control. Thus, in some embodiments, the translational or positional degrees of freedom of the positioning platform 188 may depend on admittance control, while the rotational degrees of freedom of the gimbal 186 may depend on impedance control.

[0073] F. Navigation and control. Conventional endoscopy may involve the use of fluoroscopy (e.g., delivered via a C-arm) and other forms of radiation-based imaging modalities to provide intracavitary guidance to the physician operator. In contrast, the robotic systems envisioned by this disclosure can provide non-radiation-based navigation and localization means to reduce physician exposure to radiation and reduce the amount of equipment in the operating room. As used herein, the term “localization” may mean determining and / or monitoring the position of an object within a reference coordinate system. Techniques such as preoperative mapping, computer vision, real-time EM tracking, and robot command data may be used individually or in combination to achieve a radiation-free operating environment. In other cases where radiation-based imaging modalities are still used, preoperative mapping, computer vision, real-time EM tracking, and robot command data may be used individually or in combination to enhance information obtained solely by radiation-based imaging modalities.

[0074] Figure 20 is a block diagram illustrating a positioning system 90 for estimating the location of one or more elements of a robotic system, such as the location of an instrument, according to an exemplary embodiment. The positioning system 90 may be a set of one or more computer devices configured to execute one or more instructions. The computer devices may be embodied by a processor (or more processors) and computer-readable memory within one or more components considered above. For example, but not limited to, the computer devices may be located in the tower 30 shown in Figure 1, in the cart 11 shown in Figures 1 to 4, or in the bed shown in Figures 5 to 14.

[0075] As shown in Figure 20, the positioning system 90 may include a positioning module 95 that processes input data 91-94 to generate location data 96 for the distal end of a medical device. The location data 96 may be data or logic representing the location and / or orientation of the distal end of the device relative to a reference system. The reference system may be the anatomical structure of a patient or a reference system to a known object such as an EM field generator (see the following description of an EM field generator).

[0076] Here, various input data 91-94 are described in more detail. Preoperative mapping can be used by the localization module 95 to generate model data 91. Preoperative mapping can be achieved using the acquisition of low-dose CT scans. Preoperative CT scans are reconstructed into three-dimensional images that are visualized, for example, as "slices" of notched diagrams of the patient's internal anatomical structures. When analyzed as a whole, image-based models can be generated for the anatomical cavities, spaces, and structures of the patient's anatomical structures, such as the patient's lung network. Techniques such as center-line geometry can be determined and approximated from the CT images to create three-dimensional volumes of the patient's anatomical structures, referred to as model data 91 (also referred to as "preoperative model data" if generated using only preoperative CT scans). The use of center-line geometry is discussed in U.S. Patent Application No. 14 / 523,760, the contents of which are incorporated in their entirety herein. Network phase models may also be derived from CT images and are particularly suitable for bronchoscopy.

[0077] In some embodiments, the instrument may be equipped with a camera to provide visual data (or image data) 92 to a localization module 95. The localization module 95 may process the visual data 92 to enable one or more visual-based (or image-based) location tracking modules or functions. For example, preoperative model data 91 may be used in conjunction with the visual data 92 to enable computer vision-based tracking of a medical instrument (e.g., an endoscope or an instrument that advances through the working channel of an endoscope). For example, using the preoperative model data 91, a robotic system may generate a library of predicted endoscopic images from the model based on the expected movement path of the endoscope, with each image linked to a location in the model. During surgery, this library may be referenced by the robotic system to assist in localization by comparing real-time images captured by a camera (e.g., a camera at the distal end of the endoscope) with those in the image library.

[0078] Other computer vision-based tracking techniques use feature tracking to determine the movement of the camera, and therefore the endoscope. Some mechanisms in the localization module 95 can identify circular geometric shapes in preoperative model data 91 corresponding to anatomical lumens and track changes in those geometric shapes to determine which anatomical lumen was selected, as well as the relative rotation and / or translational movement of the camera. The use of phase maps may further improve vision-based algorithms or techniques.

[0079] Optical flow, another computer vision-based technique, may analyze the displacement and translation of image pixels in a video sequence within visual data 92 to infer camera movement. Examples of optical flow techniques include motion detection, object segmentation calculation, luminance, motion compensation coding, and stereoscopic parallax measurement. By comparing multiple frames across multiple iterations, the movement and location of the camera (and therefore the endoscope) can be determined.

[0080] The positioning module 95 can generate the real-time location of the endoscope in a global coordinate system that can be aligned to the patient's anatomical structure represented by a preoperative model, using real-time EM tracking and EM data 93. In EM tracking, an EM sensor (or tracker) containing one or more sensor coils embedded in one or more locations and orientations within the medical instrument (e.g., the endoscopic instrument) measures fluctuations in the EM field generated by one or more static EM field generators positioned at known locations. The location information detected by the EM sensor is stored as EM data 93. The EM field generator (or transmitter) can be positioned close to the patient to generate a low-intensity magnetic field that the embedded sensor can detect. The magnetic field induces a small current in the sensor coil of the EM sensor, and this current can be analyzed to determine the distance and angle between the EM sensor and the EM field generator. These distances and orientations can be intraoperatively "aligned" to the patient's anatomical structure (e.g., preoperative model) to determine the geometric transformation that aligns the location in the preoperative model of the patient's anatomical structure with a single location in the coordinate system. Once aligned, the EM tracker, embedded in one or more locations on the medical instrument (e.g., the distal tip of an endoscope), can provide a real-time display of the medical instrument's progression through the patient's anatomical structure.

[0081] Robot command and kinematic data 94 may also be used by a localization module 95 to provide location data 96 for the robotic system. Device pitch and yaw resulting from joint motion commands can be determined during preoperative calibration. Intraoperatively, these calibration measurements can be used in combination with known insertion depth information to estimate the position of the instrument. Alternatively, these calculations can be analyzed in combination with EM, vision, and / or phase modeling to estimate the position of the medical instrument in the network.

[0082] As shown in Figure 20, several other input data can be used by the positioning module 95. For example, although not shown in Figure 20, a device utilizing a shape-detection fiber can provide shape data that the positioning module 95 can use to determine the location and shape of the device.

[0083] The localization module 95 can use a combination of input data 91-94. In some cases, such a combination may employ a probabilistic approach in which the localization module 95 assigns confidence weights to locations determined from each of the input data 91-94. Therefore, if the EM data is unreliable (for example, if EM interference is present), the reliability of the location determined by the EM data 93 may be reduced, and the localization module 95 may become more heavily reliant on the visual data 92 and / or the robot command and kinematic data 94.

[0084] As discussed above, the robotic systems considered herein can be designed to incorporate one or more combinations of the technologies described above. A computer-based control system for a robotic system based on a tower, bed, and / or cart may store computer program instructions in a non-temporary computer-readable storage medium, such as a persistent magnetic memory drive or a solid-state drive, which, when executed, cause the system to receive and analyze sensor data and user commands, generate control signals for the entire system, and display navigation and localization data such as the position of instruments in a global coordinate system and an anatomical map.

[0085] 2. Robot-controllable electromagnetic field generator Embodiments of robot-controllable EM field generators that can be configured for use with robotic medical systems (e.g., those described above and others) are described herein. The robot-controllable EM field generator may be configured to be coupled to (or otherwise mounted or integrated with) the robotic arm of the robotic medical system. Such a robot-controllable EM field generator can offer advantages over other EM field generators commonly used with robotic medical systems, which typically require more complex alignment steps to relate the EM field coordinate frame to the robotic system's coordinate frame or global coordinate frame, and generally cannot be repositioned during treatment without having to repeat the complex alignment steps.

[0086] By coupling the EM field generator to a robotic arm, the kinematics of the robotic arm can be used to align the EM field coordinate frame to a robotic coordinate frame or a global coordinate frame. Such alignment based on the kinematics of the robotic arm can be performed automatically or without requiring an operator to perform separate alignment steps (such as those described below with reference to Figures 22B and 22C). This eliminates the need for more complex or user-involved alignment, which is often found with other types of EM field generators used with robotic medical systems. Furthermore, with the EM field generator coupled to the robotic arm, the EM field generator becomes robot-controllable or positionable. The robotic arm can be commanded to translate, rotate, or a combination of both to physically move the EM field generator to various positions during robotic medical procedures.

[0087] As will be described in more detail below, such robot-controllable EM field generators can offer several advantages and are useful in a wide range of applications. For example, using such a robot-controllable EM field generator eliminates the need to perform a separate alignment step to align the EM field coordinate frame with the robot coordinate frame or global coordinate frame, provides a wider field of view through the ability to move the EM field generator using a robotic arm, enables the use of smaller EM field generators (e.g., compact electromagnetic field generators), simplifies the setup of robotic medical systems, and improves the accuracy of the detection position of EM sensors. Applications of such robot-controllable EM field generators may include, for example, automated tool tracking, automated or extended electromagnetic field generator setup, EM focusing, anatomical mapping, and others.

[0088] These and other features of robot-controllable electromagnetic field generators will be described in more detail below, after first providing an overview of the use of EM sensors and EM field generators in robotic medical systems.

[0089] A. Overview of the use of EM sensors and EM field generators Robotic medical systems can use various types of position sensors to facilitate the tracking and navigation of various tools and instruments. As described above, one type of position sensor that may be used in robotic medical systems is an EM position sensor (also referred to herein as an EM sensor, beacon, or tracker). One or more EM sensors may be provided, for example, on part of a tool or instrument used during a medical procedure. The position of the EM sensor may be determined and used to track the position of the corresponding tool or instrument. One or more EM sensors may also be provided on the patient, for example, to track the patient's movements during the procedure (such as movements due to breathing or other types of movement). As described above with respect to Figure 20, the positioning system 90 may use EM data 93, along with various other types of data, to provide navigation and guidance information to the system operator.

[0090] An EM sensor is used with one or more EM field generators configured to generate a low-intensity magnetic field. The position of the EM sensor relative to the EM field generator can be determined once the EM sensor is positioned within the magnetic field generated by the EM field generator (also referred to as the working volume of the EM field generator). For example, as described above with reference to Figure 20, the EM sensor may comprise one or more sensor coils that measure fluctuations in the magnetic field generated by the EM field generator. The magnetic field can induce small currents in the sensor coils of the EM sensor, which can be analyzed, for example, to determine the distance and angle between the EM sensor and the EM field generator.

[0091] In some embodiments, the position of the EM sensor relative to the EM field generator (e.g., three-dimensional position) can be determined. In some embodiments, the orientation of the EM sensor (e.g., pitch, yaw, and / or roll) can also be determined. A 5-degree-of-freedom (DoF) EM sensor can provide the three-dimensional position of the EM sensor, as well as its pitch and yaw. A 6DoF EM sensor can provide the three-dimensional position of the EM sensor, as well as its pitch, yaw, and roll.

[0092] Figure 21 shows an example of an EM field generator 202 and an EM sensor 204. The EM field generator 202 is configured to generate a magnetic field. In the illustrated embodiment, the magnetic field of the EM field generator 202 has a working volume 206. When the EM sensor 204 is positioned within the working volume 206 of the magnetic field generated by the EM field generator 202, the position (or position and orientation) of the EM sensor 204 can be determined with respect to an EM coordinate frame 208 associated with the EM field generator 202. For example, the distance and angle between the EM sensor 204 and the origin of the EM coordinate frame 208 may be determined to determine the position (e.g., x, y, and z positions) and / or orientation (e.g., pitch, yaw, and / or roll) of the EM sensor 204 within the working volume 206.

[0093] In the illustrated embodiment, the working volume 206 of the EM field generator 202 is represented as having the shape of a rectangular prism protruding from one side of the EM field generator 202. However, this represents only one type of working volume 206, and EM field generators 202 can be provided that generate working volumes 206 of various sizes, shapes, and positions relative to the EM field generator 202.

[0094] Figure 22A shows an exemplary robotic medical system 200 using the EM field generator 202 of Figure 21. The robotic medical system 200 may be similar to the robotic medical systems described above with reference to Figures 1 to 20. In the illustrated embodiment, the robotic medical system 200 comprises a cart 210 including two robotic arms 212. Although two robotic arms 212 are illustrated, other numbers of robotic arms 212 are also possible. For example, Figure 2 (mentioned above) shows an exemplary cart 11 including three robotic arms 12. In addition, in some embodiments, the robotic arms 212 do not need to be attached to the cart 210. In some embodiments, the robotic arms may be coupled to a bed or patient platform 215, for example, as shown in Figures 5 to 10 and Figure 14 (mentioned above). In some embodiments, the robotic medical system 200 may comprise robotic arms 212 coupled to the cart 210, patient platform, and / or other components of the system.

[0095] The robot arm 212 can be associated with a robot coordinate frame 216. Due to the known kinematics of the robot arm 212, the position of the robot arm 212 (e.g., the position of the distal end of the robot arm 212 and / or the position of any device 214 attached thereto) can be determined by referring to the robot coordinate frame 216. For example, since the lengths of the various links constituting the robot arm 212 are known and the angles between the links of the robot arm 212 can be determined, the position of the robot arm is kinematically defined within the robot coordinate frame 216.

[0096] In some embodiments, the position and orientation of the patient platform 215 and the cart 210 relative to the patient may be determined, set, or controlled such that the robot coordinate frame 216 can be considered a global coordinate frame that can be aligned with the patient or a portion of the patient's anatomical structure. For example, in some embodiments, up, down, right, left, etc., within the robot coordinate frame 216 may correspond to up, down, right, left, etc., within a global coordinate frame that includes the patient, patient platform 215, and / or other components.

[0097] As shown in Figure 22A, a tool or instrument 214 (e.g., a laparoscope or endoscopic instrument) can be coupled to one of the robotic arms 212. The instrument 214 can be inserted into a patient to perform a medical procedure. Although only a single instrument 214 is illustrated, it should be understood that in other embodiments, other numbers of instruments 214 (e.g., two or more instruments) can be used. One or more robotic arms 212 can be used to position and control the instrument 214. An operator (not shown) can control the robotic arms 212 and the instrument 214 using a controller.

[0098] In the illustrated embodiment, the device 214 includes an EM sensor 204 positioned at its distal end. Furthermore, the illustrated embodiment includes a second EM sensor 204 attached to the patient's chest, which may be used to track patient movement or motion, such as movement caused by the patient's breathing.

[0099] Figure 22A shows that in some embodiments, the EM field generator 202 can be positioned relative to a patient such that the working volume 206 of the EM field generator 202 overlaps with a portion of the anatomical structure of the patient on which a medical procedure is being performed. As used herein, a portion of the anatomical structure of a patient on which a medical procedure is being performed may be referred to as a medical site, which may include any site on which a medical procedure, including biopsy, endoscopy, surgery, treatment, etc., can be performed. In some embodiments, the EM field generator 202 may be supported by a stand or other support structure (not shown) such that the working volume 206 is positioned relative to the patient. In other embodiments, the EM field generator 202 may be supported by or attached to a patient platform 215, or integrated into or supported by another component in the operating environment (such as the cart 30 described above with reference to Figures 1, 3, and 4).

[0100] As described above with reference to Figure 21, once the EM sensor 204 is positioned within the working volume 206 of the EM field generator 202, the position of the EM sensor 204 within the EM coordinate frame 208 can be determined. However, determining the position of the EM sensor 204 within the EM coordinate frame 208 may not be particularly useful unless the EM coordinate frame 208 is aligned with the robot coordinate frame 216 or the global coordinate frame (which may be the robot coordinate frame 216, as described above). Therefore, an alignment step to associate the EM coordinate frame 208 with the robot coordinate frame 216 is often required.

[0101] In some robotic medical systems, such as the robotic medical system 200 illustrated in Figure 22A, aligning the EM coordinate frame 208 with the robot coordinate frame 216 may require the operator to perform certain steps to achieve alignment. The alignment process may add steps to the procedure and may increase the procedure time. Figures 22B and 22C show exemplary alignment steps or procedures that may be used to align the robot coordinate frame 216 associated with the robotic medical system 200 with the EM coordinate system 208 associated with the EM field generator 202. As will be described in more detail below, the need to perform these types of alignment steps or procedures can be eliminated or reduced by providing a robotic medical system having an EM field generator mounted on its robotic arm, for example, as shown in Figure 23.

[0102] Figure 22B shows an exemplary alignment procedure that may be used to align a robot coordinate frame 216 associated with a robotic medical system 200 with an EM coordinate system 208 associated with an EM field generator 202. To achieve alignment, several points are identified within both the robot coordinate frame 216 and the EM coordinate frame 208. Once the points are identified within each of the robot coordinate frame 216 and the EM coordinate frame 208, the alignment algorithm can determine the transformation between the two coordinate frames. The alignment procedure in Figure 22B uses one of the robotic arms 212 of the robotic system, an EM field generator 202, an EM probe 203, and an alignment fixture 205. In the illustrated embodiment, the alignment fixture 205 comprises a cube having references or markers located at the vertices of the cube. Thus, in the illustrated embodiment, the alignment fixture 205 includes eight references. Other numbers of references and other shapes may also be used for the alignment fixture 205. With the alignment fixture 205 positioned in a stationary location, the operator can command the robot arm 212 to make contact with each of the references of the alignment fixture 205. Thus, the position of each reference in the robot coordinate frame 216 can be determined. The operator can then touch each of the references with the EM probe 203. The EM probe 203 may include a handheld EM sensor 204 having a tip position clearly defined relative to the electromagnetic field generator 202. By making contact with each of the references with the EM probe 203, the position of each reference in the EM coordinate frame 208 can be determined. Using the same points now known in the robot coordinate frame 216 and the EM coordinate frame 208 (the references of the alignment fixture 205), the alignment algorithm can determine the transformation between the two coordinate frames.

[0103] Figure 22C shows another exemplary alignment procedure that may be used to align a robot coordinate frame 216 associated with a robotic medical system 200 with an EM coordinate system 208 associated with an EM field generator 202. This alignment procedure uses one of the robotic arms 212 of the robotic system, an EM field generator 202, and an EM sensor 204 mounted on the end effector of the robotic arm 212. Using the EM sensor 204 mounted on the end effector of the robotic arm 212, the operator can then command the robotic arm 212 to move along or trace an alignment trajectory 207. In the illustrated embodiment, the alignment trajectory 207 includes a cubic shape, but other alignment trajectories including different shapes can be used. At various points along the alignment trajectory 207 (e.g., vertices of the cubic shape), the position of the end effector in the robot coordinate frame 216 and the position of the EM sensor 204 in the EM coordinate frame 208 can be recorded. Once a sufficient number of points have been identified in both the robot coordinate frame 216 and the EM coordinate frame 208, the alignment algorithm can determine the transformation between the two coordinate frames.

[0104] In robot systems such as the system 200 shown in Figure 22A, alignment procedures such as any of the procedures described above with reference to Figures 22B and 22C may be performed prior to initiating the procedure for aligning the EM coordinate frame 208 and the robot coordinate frame 216.

[0105] These types of alignments have various drawbacks. For example, such alignments can be tedious and time-consuming. They are generally not considered automated because they may require user input and / or must be performed before starting a robotic medical procedure. Furthermore, such alignments can introduce inaccuracies into the system, for example, if the operator does not navigate precisely to the reference point. In addition, such alignments require the position of the EM field generator 202 to remain fixed during the procedure. If the EM field generator 202 moves (e.g., accidentally bumped by someone in the operating room, or moved to allow access to the patient), the alignment must be performed again to re-establish the relationship. This can be problematic, for example, because the EM field generator 202 may need to be moved to allow access to the fluoroscopy C-arm. In this case, it is necessary to remove the EM field generator 202, move the C-arm to a position suitable for capturing one or more images, then remove the C-arm, return the EM field generator 202 to its original position, and have the operator repeat the alignment step between the EM coordinate frame and the robot or global coordinate frame.

[0106] As will be explained in more detail below, the need for these types of alignment can be reduced or eliminated by using a robot-controllable electromagnetic field generator that can be directly coupled to the robot arm, for example, as shown in Figures 23 to 25. By coupling the robot-controllable electromagnetic field generator to the robot arm, the relationship or alignment between the robot-controllable electromagnetic field generator and the robot arm can be kinematically determined, and thus the EM coordinate frame 208 can be positioned within the robot coordinate frame 216.

[0107] In addition, in earlier systems that use alignment procedures such as those described with reference to Figures 22B and 22C, the position of the EM field generator 202 must remain fixed during the procedure, which may necessitate the use of a physically larger and sometimes more cumbersome EM field generator 202 to generate a working volume 206 large enough to cover the relevant medical site. In some situations or applications, this may increase the difficulty of setting up and positioning the EM field generator 202 relative to the patient. In some situations or applications, this may limit the available space around the patient during the procedure and / or require more setup time. For example, the use of a robot-controllable electromagnetic field generator that can be directly coupled to a robotic arm, as shown in Figures 22-25, can reduce or eliminate one or more of these limitations associated with the EM field generator 202 as shown in Figure 22A.

[0108] B. Overview of a robot-controllable electromagnetic field generator Figure 23 shows one embodiment of a robotic medical system 300, which includes one embodiment of a robot-controllable EM field generator 302 coupled to a robotic arm 212. As will be described in more detail below, coupling the EM field generator 302 to the robotic arm 212 can provide several advantages, including, for example, (1) enabling more accurate and / or simpler alignment between the EM coordinate frame 308 and the robotic coordinate frame 216 of the EM field generator 302 based on the kinematics of the robotic arm 212, and / or (2) enabling the position of the EM field generator 302 to be moved or readjusted using the robotic arm 212.

[0109] Similar to the robotic medical system 200 in Figure 22A, in the robotic medical system 300 in Figure 23, the robotic arm 212 can be associated with a robot coordinate frame 216. Due to the known kinematics of the robotic arm 212, the position of the robotic arm 212 can be determined by referring to the robot coordinate frame 216. For example, since the lengths of the various links constituting the robotic arm 212 are known and the angles between the links of the robotic arm 212 can be determined, the position of the robotic arm 212 is kinematically defined within the robot coordinate frame 216. In addition, with respect to some embodiments of the robotic medical system 300, the position and orientation of the patient platform 215 and the cart 210 relative to the patient can be determined, set, or controlled such that the robot coordinate frame 216 can be considered a global coordinate frame that can be aligned with the patient or a part of the patient's anatomical structure. For example, in some embodiments, up, down, right, left, etc., within the robot coordinate frame 216 can correspond to up, down, right, left, etc., within a global coordinate frame that includes the patient, patient platform 215, and / or other components. This facilitates the operator's ability to navigate within the patient's anatomical structure. As a more specific example, during a bronchoscopy or ureteroscopy procedure, the robotic system 300, which includes a cart from which a robotic arm 212 extends, can position the cart parallel to the bed. The bed and cart share a common gravity vector (i.e., an upward point toward the ceiling for both of them). Using this information, the system can provide spatial context by indicating to the user which direction is up (forward) or down (backward) when the user is driving the bronchoscope or ureteroscope. In urological applications, the user may need to determine whether the ureteroscope is in the posterior or anterior calyx during target selection for percutaneous access. Since the system's orientation is known, the system can provide anatomical context to facilitate this navigation.

[0110] As shown in Figure 23, the EM field generator 302 can be coupled to one of the robot arms 212. In some embodiments, the EM field generator 302 is coupled to or attached to the distal end of the robot arm 212, but the EM field generator 302 may be coupled to or attached to the robot arm 212 at other locations (e.g., between the distal and proximal ends of the arm). Figures 24 and 25, described below, show more detailed embodiments of the EM field generator 302 and, in some embodiments, how it can be coupled to the robot arm 212. Other methods and mechanisms for coupling the EM field generator 302 to the robot arm 212 are also possible. Furthermore, in some embodiments, the EM field generator 302 can be integrated into the robot arm 212 itself (e.g., it may be a component on or inside the robot arm 212). For example, the EM field generator 302 may be integrated into one of the links of the robot arm 212 (such as the furthest link), or into an instrument driver or instrument drive mechanism positioned at the distal end of the robot arm 212, for example. Exemplary instrument drivers 62, 75, 83, and 146 into which the EM field generator 302 can be incorporated are shown in Figures 14-17 above, but these examples are not limiting.

[0111] By coupling the EM field generator 302 to the robot arm 212, the kinematics of the robot arm 212 can be used to provide alignment between the EM coordinate frame 308 of the EM field generator 302 and the robot coordinate frame 216 and / or the global coordinate frame. That is, the kinematics of the robot arm 212 (which is known as described above) can be used to associate the EM coordinate frame 308 of the EM field generator 302 with the robot coordinate frame 216 or the global coordinate frame. This can provide one or more advantages compared to a robotic medical system that includes a separate EM field generator 202, such as the robotic system 200 in Figure 22A.

[0112] One advantage is that by coupling the EM field generator 302 to the robot arm 212, the need for a separate alignment step (which may require user input, as described above) may be eliminated. For example, in the robotic medical system 300, it may not be necessary to manually perform a separate alignment step, such as the alignment step described with reference to Figures 22B and 22C, to align the EM coordinate frame 308 to the robot coordinate frame 216 or the global coordinate frame. In the robotic medical system 300, such alignment can be performed automatically (for example, by the system) simply by attaching the EM field generator 302 to the robot arm 212. The robotic medical system 300 can then use the known kinematics of the robot arm 212 to associate the EM coordinate frame 308 with the robot coordinate frame 216 or the global coordinate frame.

[0113] Another advantage that can be achieved by coupling the EM field generator 302 to the robot arm 212 is that the accuracy of alignment between the EM coordinate frame 308 and the robot coordinate frame 216 can be improved. As mentioned above, in systems such as the robot system 200 in Figure 22A, the alignment step can result in the system being inaccurate, for example, due to inaccuracies in navigation during the manual alignment step. In the robotic medical system 300, the accuracy of alignment between the EM coordinate frame 308 and the robot coordinate frame 216 is determined primarily by the accuracy of the kinematics of the robot arm 212. If these are known and clearly defined, the alignment between the EM coordinate frame 308 and the robot coordinate frame 216 can be very accurate.

[0114] A further advantage provided by some embodiments, including coupling the EM field generator 302 to the robotic arm 212, may be that the position of the EM field generator 302 can be adjusted using the robotic arm 212. This can, by itself, provide one or more advantages. As an example, the robotic arm 212 can be used to move or reposition the EM field generator 302. This may, for example, allow an operator to adjust the position of the working volume 306 of the magnetic field of the EM field generator 302. As described below, for example, the robotic arm 212 can be used to move the EM field generator 302 so that it tracks the movement of an instrument 214 as the instrument moves through the body. In addition, for example, the EM field generator 302 may be moved or repositioned by the robotic arm 212 to facilitate access to the patient, for example, to allow a fluoroscopy C-arm to access the patient. For example, the robotic arm 212 can be used to move the EM field generator 302 out of the way. Furthermore, when the robot arm 212 moves the EM field generator 302, the relationship between the EM coordinate frame 308 and the robot coordinate frame 216, determined by the kinematics of the robot arm 212, remains known. Therefore, it may not be necessary to realign the EM coordinate frame 308 with respect to the robot coordinate frame 216 after the EM field generator 302 has moved.

[0115] Another advantage achievable by coupling the EM field generator 302 to the robot arm 212 may include improved accuracy of the determined position of the EM sensor 204, as the orientation and distance between the EM field generator 302 and the EM sensor 204 can be adjusted to improve accuracy. For example, the accuracy of the determined position of the EM sensor 204 may decrease as it approaches the edge of the working volume 306. If the EM sensor 204 is determined to be closer to the edge of the working volume 306, the robot arm 212 can move the EM field generator 302 so that the EM sensor 204 is positioned closer to the center of the working volume 306 (or further away from the edge of the working volume 306), thereby improving the accuracy of the determined position. As another example, the robot arm 212 can move the EM field generator 302 closer to the EM sensor 204 to focus on the EM sensor 204 and track its position more precisely (referred to herein as EM focusing).

[0116] In addition, since the robot arm 212 can easily adjust the position of the EM field generator 302 without requiring a separate alignment step, a smaller EM field generator can generally be used compared to other robot systems (such as robot system 200) that use stationary EM field generators. When stationary EM field generators are used, they must be large enough to provide a sufficiently large working volume to cover the medical site. Even then, they may not cover all parts of the patient's anatomical structure, such as parts of the patient's anatomical structure that are navigated through to access the medical site. By coupling the EM field generator 302 to the robot arm 212, the position of the working volume 306 can be adjusted by moving the EM field generator 302 using the robot arm 202, so a smaller EM field generator, which may have a smaller working volume in some cases, can be used. In some embodiments, this may allow the use of a compact field generator (cFG). However, the use of a compact field generator is not required, and in some embodiments, a larger field generator can be coupled to the robot arm.

[0117] When coupled to the robotic arm 212, the EM field generator 302 can function similarly to the EM field generator 202 described above. For example, in the illustrated embodiment of the system 300 in Figure 23, one or more of the instruments 214 may include one or more EM sensors 204 positioned on their distal end and / or on other parts of the instrument 214. In addition, the illustrated embodiment includes a second EM sensor 204 attached to the patient's chest, which may be used to track movements caused by the patient's breathing. Once the EM sensor 204 is positioned within the working volume 206 of the EM field generator 302, the position of the EM sensor 204 in the EM coordinate frame 308 can be determined.

[0118] Furthermore, by using the kinematics of the robot arm 212, the EM coordinate frame 308 can be aligned with the robot coordinate frame 216, so that the position of the EM sensor 204 can be determined within the robot coordinate frame 216 or the global coordinate frame. By physically connecting the EM field generator 302 to the robot arm 212, the EM coordinate frame 308 and the robot coordinate frame 216 or the global coordinate frame can be linked so that the position of the EM sensor 204 can be determined within the robot coordinate frame 216 or the global coordinate frame.

[0119] For example, in the illustrated embodiment of Figure 23, the robotic medical system 300 may include an EM field generator 302 configured to generate an EM field. The robotic medical system 300 may include a robotic arm 212. The first robotic arm 212 may be coupled to the EM field generator 302 and configured to articulate to move the EM field generator 302. The robotic medical system 300 may also include one or more processors configured to determine the position of an EM sensor 204 in the EM field in an EM coordinate frame 302 associated with the EM field generator 302. The processors may also be configured to determine the alignment between the EM coordinate frame 308 and the robot coordinate frame 216 associated with the first robotic arm 212 based on determining the position of the EM field generator 302 in the robot coordinate frame. This can be achieved by using the kinematics of the robotic arm 212, as described above. The processors may further be configured to determine the position of the EM sensor 204 in the robot coordinate frame 216 based on the alignment.

[0120] As another example, in the illustrated embodiment of Figure 23, the robotic medical system 300 may include a controller or control circuit having one or more processors configured to control the movement of one or more robotic arms 212. The robotic arms 212 may be configured to perform articulated movements in response to receiving commands from the control circuit. An EM field generator 302 may be coupled to the robotic arms 212 so that the EM field generator also moves in response to one or more commands from the control circuit.

[0121] In addition to the robot-controllable EM field generator 302 coupled to the robotic arm 212, the illustrated embodiment of the robotic system 300 in Figure 23 may in many respects be similar to the robotic system 200 described above with reference to Figure 22A, which includes a separate EM field generator 202, or to other robotic medical systems described throughout this application or elsewhere. For example, in the illustrated embodiment, the robotic medical system 300 comprises a cart 210 including two robotic arms 212. Although shown with two robotic arms 212, the robotic system 300 may use any other number of robotic arms 212. For example, Figure 2 (mentioned above) shows an exemplary cart 11 including three robotic arms 12. In addition, in some embodiments, the robotic arms 212 do not need to be attached to the cart 210. In some embodiments, the robotic arms may be coupled to a bed or patient platform 215, for example, as shown in Figures 5-10 and Figure 14 (mentioned above). In some embodiments, the robotic medical system 300 may include a cart 210, a patient platform, and / or a plurality of robotic arms 212 (e.g., two, three, four, five, six, or more robotic arms 212) coupled to other components of the system.

[0122] Furthermore, while the robotic system 300 is shown in Figure 23 with a single tool or instrument 214 (e.g., a laparoscope or endoscopic instrument) attached to one of the robotic arms 212, it should be understood that in other embodiments, other numbers of instruments 214 (e.g., two, three, four, five, six, or more instruments, including a laparoscope, endoscope, and camera) may be used.

[0123] Figure 24 shows one embodiment of the EM field generator 302. In the illustrated embodiment, the EM field generator 302 is configured to be coupled to an instrument drive mechanism 402 that can be positioned on the robot arm so that the EM field generator 302 can be mounted on the robot arm 212. As described above with reference to Figure 16, the instrument drive mechanism, such as the instrument drive mechanism 402, may be configured to have an interface 404 adapted for attachment to various tools or instruments for use by a robotic medical system. The EM field generator 302 may be configured to have a corresponding interface 320 configured to be coupled to the interface 404 of the instrument drive mechanism 402. In this way, the EM field generator 302 can be coupled to the robot arm 212 in the same manner as other robotic tools or instruments of the robotic system.

[0124] For example, the interface 320 of the EM field generator 302 includes one or more connectors 322 configured to connect to one or more corresponding connectors 406 on the interface 404 of the equipment drive mechanism 402. Such a configuration may allow an operator to easily and detachably connect the EM field generator 302 to the equipment drive mechanism 402.

[0125] In the illustrated embodiment, the EM field generator 302 includes a housing 324. The proximal end 326 of the housing 234 may be configured to be coupled to a device drive mechanism 402. For example, the proximal end 326 may include an interface 320 and a connector 322. The housing 324 may extend from the proximal end 326 to the distal end 328. The distal end 328 may include an EM field generator unit 330 configured to generate a magnetic field. In the illustrated embodiment, the distal end 328 includes a clip 332 configured to secure the EM field generator unit 330. Other mechanisms or methods for securing the EM field generator unit 330 are possible. In some embodiments, for example, as shown, the EM field generator unit 330 may be detachable from the housing 324. In some embodiments, the EM field generator unit 330 is integrated into the housing 324. In the illustrated embodiment, the EM field generator unit 330 includes a cFG, but other types of electromagnetic field generators may also be used.

[0126] The size and shape of the housing 326 may be determined or selected such that the kinematic relationship between the EM field generator unit 330 and the robot arm to which the EM field generator 302 is attached is known, thereby enabling the determination of the kinematic alignment between the EM coordinate frame associated with the EM field generator and the robot coordinate frame associated with the robot arm.

[0127] In the illustrated embodiment, the EM field generator unit 330 includes one or more connectors 334. The connectors 334 can electrically connect the EM field generator unit 330 to a robot system so that the robot system can communicate with the EM field generator unit. In other embodiments, an electrical connection may be made between the interface 320 of the EM field generator 302 and the interface 404 of the equipment drive mechanism 402.

[0128] Figure 25 shows how the EM field generator 302 can be attached to the instrument drive mechanism 402 according to one embodiment. In the illustrated embodiment, the instrument drive mechanism 402 is shown covered with a sterile drape. The sterile adapter 410 is shown positioned on the interface 404 of the instrument drive mechanism 402. The interface 320 of the EM field generator 302 can be attached to the sterile adapter 410 so that the EM field generator 302 is coupled to the instrument drive mechanism 402 and the sterile adapter 410 is positioned between them. As shown in Figure 25, the EM field generator unit 330 may also be covered with a sterile drape.

[0129] Figure 26 is a flowchart illustrating one embodiment of Method 500 for performing a robotic medical procedure using an EM field generator configured to be coupled to (or otherwise connected to or integrated with) a robotic arm. In the illustrated embodiment, the Method begins in block 502. Block 502 may include generating an EM field using an EM field generator coupled to a first robotic arm. The EM field may be associated with an EM coordinate frame. Next, Method 500 moves to block 504, where an alignment is determined between the EM coordinate frame and a robot coordinate frame associated with the first robotic arm. The alignment may be based on determining the position of the EM field generator in the robot coordinate frame based on the kinematics of the first robotic arm. In block 506, the position of an EM sensor in the EM coordinate frame is determined. In block 508, the Method includes determining the position of an EM sensor in the robot coordinate frame based on the alignment determined in block 504.

[0130] In some embodiments, Method 500 may optionally include detachably coupling the EM field generator to a first robotic arm, for example, as shown in Figure 25. Method 500 may also optionally include moving the EM field generator with the first robotic arm. Moving the EM field generator may be useful in that the working volume of the EM field can be positioned as needed to facilitate treatment and / or the EM field generator can be moved out of the way to allow access to the patient during medical treatment. Moving the EM field generator with the first robotic arm to adjust the position of the EM field generator relative to the EM sensor can also improve the accuracy of the determined position of the EM sensor in the EM field coordinate frame (and correspondingly in the robot coordinate frame through alignment).

[0131] Method 500 may also include moving a medical instrument coupled to a second robotic arm, and a robotic coordinate frame may also be associated with the second robotic arm. An EM sensor may be positioned on the medical instrument such that guidance for the navigation of the second instrument may be determined at least in part on the determined position of the EM sensor.

[0132] C. Exemplary applications of robot-controllable electromagnetic field generators In addition to providing one or more of the advantages described above, robot-controllable electromagnetic field generators coupled to or otherwise integrated with a robotic arm can be used to enable, facilitate, and / or improve various functionalities during robotic medical procedures. This section outlines several exemplary applications for robot-controllable electromagnetic field generators that may be advantageous over other robotic systems, including, for example, static or stationary electromagnetic field generators.

[0133] i. Setting up electromagnetic field generators and tracking equipment As described above, robot-controllable electromagnetic field generators coupled to or integrated into robot arms can be used to facilitate the setup and tracking of the electromagnetic field generators. This section provides several examples to illustrate these concepts without limitation.

[0134] As used herein, “setting up” the EM field generator refers to determining where the EM field generator should be positioned to facilitate the procedure. The position may be based, for example, on the determined positions of one or more EM sensors. In some cases, the position may be adjusted during the procedure, for example, when one or more of the EM sensors are moved during the procedure. As described above with reference to Figures 21 and 22A, the EM field generator 202 is positioned relative to the patient such that the working volume 206 of the electromagnetic field generator 202 overlaps with a portion of the patient's anatomical structure (e.g., the medical site) on which the medical procedure is being performed. The EM field generator 202 may be supported by a stand or other support structure, such as a patient platform 215. Once positioned, alignment steps (such as those described with reference to Figures 22B and 22C) are performed to align the EM coordinate frame 208 associated with the EM field generator 202 with respect to the robot coordinate frame 216. Furthermore, during the procedure, movement had to be done manually, and the alignment step had to be repeated to realign the EM coordinate frame 208 associated with the EM field generator 202 to the robot coordinate frame 216, so the EM field generator 202 was generally not moved. A robot-controllable EM field generator that can be repositioned using the system's robotic arm simplifies the setup procedure by allowing the system to determine where the EM field generator should be positioned and then moving the EM field generator to those locations using the system's robotic arm. In addition, if the position of tools and instruments, including EM sensors, changes during the procedure, the system can adjust the position of the EM field generator so that it remains positioned in a favorable location.

[0135] Figures 28A and 28B show an introductory example demonstrating that a robot-controlled EM field generator 302 (as described above, for example, with reference to Figures 23-25) can be used to facilitate the setup and positioning of various tools and devices used in medical procedures. Additional examples are described in further detail below. Figure 28A shows three EM position sensors 204, as well as the EM field generator 302 coupled to a robotic arm 212. The EM position sensors 204 can be positioned, for example, on a medical instrument (such as an endoscope, laparoscope, and / or other types of surgical tools) or directly on a patient (for example, an EM patch sensor configured to track patient movement or respiration). For ease of illustration, the medical instruments are not shown.

[0136] As shown in Figure 28A, the system may be configured to move the EM field generator 302 using a robotic arm 212 to locate the EM sensor 204. Such movement may include translation, rotation, and / or vibration of the EM field generator 302. An example of a circular path is shown in Figure 28A. Once the position of the EM sensor 202 is determined, the robotic arm 212 may move to reposition the EM field generator 302 to a relatively improved (referred to herein as a “workplace”) or centralized location where the EM sensor 204 is positioned within the working volume 306 of the EM field generator 302, as shown, for example, in Figure 28B. This can facilitate setup because the system can determine the workplace or central location where the EM field generator 302 should be automatically positioned. In relevant embodiments, the positions of one or more EM sensors 204 may be adjusted to achieve a relatively improved arrangement of the EM sensors 204 based on the range of movement or vibration achievable by the EM field generator, the medical procedure step, and / or the anatomical characteristics of the patient.

[0137] In some embodiments, the system determines the position and / or orientation of the EM sensor 204, and then calculates the centroid or geometric center of those positions, or the shape defined by one or more of those positions. The EM field generator 302 can then be positioned using the robotic arm 212 so that the center of the working volume 306 aligns, for example, with the determined centroid of the EM sensor 204 positions.

[0138] In addition, Figure 28A shows that the EM field generator 302 can increase the functional size of the working volume 306. For example, if the EM sensors 204 are spaced so far apart that they cannot all fit within the working volume 306, the system can move the EM field generator (in scanning motion) to generate a functional working volume 306 that is large enough to track all of the EM sensors 204. In some embodiments, the system can prioritize the EM sensors such that certain EM sensors 204 always remain positioned within the working volume 306, while others are sometimes inside and sometimes outside the working volume 306.

[0139] As used herein, “instrument tracking” generally refers to moving a robot-controllable EM field generator using a robotic arm to track or follow the movement of an EM sensor. As described above, conventional systems generally involve the use of stationary EM field generators that cannot be moved to track or follow the movement of an EM sensor, or (to avoid repositioning) cannot be moved. In such systems, if an EM sensor moves outside the working volume of the EM field generator, the EM sensor becomes undetectable until it is returned to the working volume. A robot-controllable electromagnetic field generator that can be moved with a robotic arm can facilitate instrument tracking by allowing the EM field generator to be moved or otherwise adjusted to track the movement of one or more instrument sensors. Thus, instead of moving the EM sensor outside the working volume of the EM field generator, the EM field generator can be moved with the EM sensor so that the EM sensor remains positioned within the working volume, facilitating continuous tracking of the EM sensor.

[0140] Figures 27A and 27B illustrate an embodiment of automated instrument tracking that can be facilitated using a robot-controllable electromagnetic field generator, such as the EM field generator 302 described above with reference to Figures 23-25. Specifically, Figures 27A and 27B illustrate an example of automated instrument tracking during an exemplary medical procedure, for example, during a ureteroscopy procedure in which the ureteroscope 502 is robotically navigated through the patient's orifice, through the ureter, and into the kidney. Although a ureteroscopy procedure is illustrated, automated instrument tracking can be used in other types of procedures, including endoscopic and / or laparoscopy procedures. Figure 27A shows a procedure at an earlier or first time / step (e.g., immediately after insertion of the ureteroscope 502), and Figure 27B shows a procedure at a later or second time / step (e.g., after the distal tip of the ureteroscope 502 has been navigated into the kidney).

[0141] As shown in Figures 27A and 27B, the ureteroscope 502 may be coupled to one or more robotic arms 212 configured to manipulate and insert the ureteroscope 502, as described with respect to Figure 3, for example. In Figures 27A and 27B, the EM field generator 302 is coupled to a third robotic arm 212 configured to move to adjust the position of the EM field generator 302 (and correspondingly, the position of the working volume 306 of the magnetic field generated by the EM field generator 302). As described above, since the EM field generator 302 is mounted on the robotic arm 212, the EM coordinate frame associated with the EM field generator 302 can be automatically aligned to the robotic coordinate frame associated with the robotic arm 212 through the kinematics of the robotic arm 212 to which the EM field generator 302 is mounted.

[0142] As shown in Figures 27A and 27B, the system may be configured to move the EM field generator 302 using a corresponding robotic arm 212 to automatically track the position of the ureteroscope 502. In the illustrated embodiment, the ureteroscope 502 includes an EM sensor 204 positioned at its distal tip. As described above, the position of the EM sensor 204 can be determined relative to the EM coordinate frame when the EM sensor 204 is positioned within the working volume 306 of the EM field generator 302. Furthermore, since the EM coordinate frame can be aligned with the robotic coordinate frame, the position of the EM sensor 204 within the robotic coordinate frame can also be determined.

[0143] As the ureteroscope 502 is further inserted into the patient, the robotic arm 212 to which the EM field generator is attached may be moved or adjusted to automatically reposition the EM field generator so that the position sensor 204 remains positioned within the working volume 306 of the EM field generator. In some embodiments, this automatic tracking of the ureteroscope 502 is configured such that the EM field generator 302 is moved or adjusted so that the EM sensor 204 remains positioned at the center of the working volume 306, although this is not required in all embodiments. In addition, in some embodiments, the orientation of the EM field generator 302 may be adjusted relative to the EM sensor 204 to provide optimal or improved accuracy in determining its position.

[0144] This type of automated tracking can advantageously allow the position of the ureteroscope 502 to be tracked throughout the entire procedure (e.g., from insertion into the kidney to operation). This may not be possible with robotic systems using static or stationary electromagnetic field generators. As mentioned above, static or stationary electromagnetic field generators are often configured so that their working volume is centered on a major medical site (e.g., the kidney). At this location, the working volume may not extend far enough to cover insertion and navigation through the ureter. Therefore, in systems using static or stationary electromagnetic field generators, the operator may not be able to utilize EM-based navigation until the instrument is navigated into the working volume of the electromagnetic field generator. Furthermore, moving the static or stationary electromagnetic field generator to follow the instrument during insertion and navigation into the medical site is generally not possible because it would require repeating the alignment step each time the electromagnetic field generator is moved, which is impractical and could cause delays during the medical procedure and potentially adversely affect the outcome of the procedure.

[0145] However, as shown in Figures 27A and 27B, the robot-controlled EM field generator 302 allows the robot arm 212 to reposition the EM field generator 302 as needed, making it easy to track the position of the ureteroscope 502 from insertion into the kidney to operation, and the kinematics of the robot arm 212 can be used to provide continuous alignment between the EM coordinate frame and the robot coordinate frame.

[0146] An additional benefit that can be achieved using the automated instrument tracking shown in Figures 27A and 27B is improved anatomical mapping. When an instrument is navigated through the patient's anatomical structure, the position of the EM sensor on the instrument can be used to construct an anatomical map. For example, in the case of bronchoscopy, since the bronchoscope, which includes the EM sensor, is navigated through the patient's airway, the determined position of the EM sensor can be used to generate a map of the patient's lungs. With a static or stationary EM field generator, this is only possible for areas within the working volume of the EM field generator. However, the ability to generate an anatomical map is enhanced by using a robot-controllable EM field generator 302, which allows the robotic arm 212 to reposition the EM field generator 302.

[0147] Using the examples in Figures 27A and 27B, the position of the EM sensor 204 is determined by insertion into the kidney (Figure 27B) (Figure 27A), and a nearly complete anatomical map of the patient's urinary tract can be generated. In contrast, when a static electromagnetic field generator is used with its working volume centered on the kidney, EM sensor data that could be used to map the patient's ureters may not be available. Therefore, by using a robot-controllable EM field generator 302 as described herein, the ability to generate anatomical mappings using EM sensor data can be extended, resulting in a more complete anatomical map.

[0148] Figure 29 is a block diagram representing a robotic medical system 300, such as the example shown in Figure 22A, which includes a robot-controllable EM field generator 302 mounted on a robotic arm 212. As shown in Figure 29 (and also in Figure 23), the system 300 may include one or more robotic arms 212. In the illustrated embodiment, the system 300 includes a first robotic arm 212 and a second robotic arm 212, but in other embodiments, it may include a different number of robotic arms (e.g., one, two, three, four, five, six, or more). As shown, the first robotic arm 212 is coupled to the EM field generator 302. As described above, the EM field generator 302 can be attached to or coupled to the first robotic arm 212, or it can be directly incorporated into the first robotic arm 212 itself. The EM field generator 302 is configured to generate an EM field having a working volume 306, within which the position of the EM sensor 204 can be determined relative to the EM coordinate frame 308. The first robotic arm 212 may be configured to adjust the position of the EM field generator 302. For example, the first robotic arm 212 can move (e.g., change its posture or shape) to readjust the position of the EM field generator 302 and, accordingly, the position of the working volume 306 of the EM field.

[0149] The first robotic arm 212 (and other robotic arms 212 in the system) is associated with the robotic coordinate frame 216 as described above. The position of the EM field generator 302 within the robotic coordinate frame 216 can be determined based on the kinematic position of the arm 212. This allows the alignment or mapping between the EM coordinate frame 308 and the robotic coordinate frame 216 to be determined as described above. As shown in Figure 29 (and also in Figure 23), the system 300 may include one or more EM sensors 204. The positions of the EM sensors 204 can be determined relative to the EM coordinate frame 308 and mapped to the robotic coordinate frame 216 using the alignment described above.

[0150] In Figure 29 (and Figure 23), one of the EM sensors 204 is positioned on a medical device 214. In some embodiments, the medical device 214 may be coupled to a second robotic arm 212. This may allow the second robotic arm 212 to operate and control the medical device 214. When the second robotic arm 212 operates and controls the medical device 214, the position of the medical device 214 may be determined based on the determined position of the EM sensor 204. As shown in Figure 29, the system 300 may include additional EM sensors 204 that can be tracked in the same way.

[0151] The block diagram in Figure 29 further illustrates that the system 300 may include a processor 380 and a memory 382. The memory 382 may be used to store instructions, which, when executed by the processor 380, can enable various functionalities of the system 300, such as determining and using alignment and mapping from the EM coordinate frame 308 to the robot coordinate frame 216, setting up the EM field generator 302, and tracking the medical device 214, as described throughout this section and the entire application. For example, the processor 380 may be able to communicate with the first robot arm 212 and the EM field generator 302 and be configured to determine the position of one or more of the EM sensors 204 in the EM field generated by the EM field generator 302, and to adjust the position of the EM field generator 302 by commanding the movement of the first robot arm 212 based on the determined positions of the EM sensors 204. The commanded movement of the first robot arm 212, based on the determined position of the EM sensor 204, can be used to set the position of the EM field generator 302 or to track the position of the EM sensor 204 as it moves during treatment.

[0152] In some embodiments, the processor 302 is configured to first determine the position of the EM sensor 204 relative to the EM coordinate frame 308, and then map that position to the robot coordinate frame 216. As described above, this can be achieved, for example, by (i) determining the position of the EM sensor 204 in the EM field relative to the EM coordinate frame 308 associated with the EM field generator 302, (ii) determining the alignment between the EM coordinate frame 308 and the robot coordinate frame 216 associated with the first robot arm 212 based on determining the position of the EM field generator in the robot coordinate frame 216, and (iii) determining the position of the EM sensor 204 in the robot coordinate frame 216 based on the alignment. The determination of the position of the EM field generator 302 in the robot coordinate frame 216 in step (ii) can be based on the kinematics of the first robot arm 212.

[0153] Figures 30A to 33D show various examples or approaches for moving the EM field generator 302 using the first robotic arm 204 based on the determined position of the EM sensor 204, in order to facilitate electromagnetic field generator setup and equipment tracking.

[0154] Figures 30A and 30B are perspective views showing an example of moving an EM field generator using a robotic arm so that an EM sensor is positioned in a predetermined location within the EM field. This can be done, for example, during the setup of the electromagnetic field generator and can be advantageously used to position the EM field generator 302 relative to the EM sensor 204. In some embodiments, it may be advantageous to set up or align the EM field generator 302 so that the EM sensor 204 is positioned together with the working volume 306 of the EM field. This may be useful for several reasons. For example, in some cases, this can be done so that the working volume 306 of the EM field is centered around the EM sensor 204. This can help ensure that the EM sensor 204 can still be tracked, as it can move without moving outside the working volume 306 if it moves. As another example, this can improve the accuracy of the determined position of the EM sensor 204, because in some examples, the ability to determine the position of the EM sensor 204 within the EM field may be more accurate towards the center of the working volume 306 than on the edges of the working volume 306. Therefore, in some embodiments, the predetermined position is the center of the working volume 306, and the EM field generator 302 moves so that the EM sensor 204 is positioned at the center of the working volume 306, as shown in Figures 30A and 30B.

[0155] In Figures 30A and 30B, the predetermined position 384 is the center of the working volume 306 and is represented as x in the figures. Figure 30A shows the system in a first state before moving the EM field generator 302 so that the EM sensor 204 is positioned at the predetermined position 384. As shown in Figure 30A, the EM sensor 204 is positioned within the working volume 306 of the EM field generator 302, but not at the predetermined position 384. In order to position the EM sensor 204 at the predetermined position 384, the EM field generator 302 must be moved in the direction of the arrows shown in the figure, which causes a corresponding movement of the working volume 306 and the predetermined position 384. Figure 30B shows the system in a second state after the EM sensor 204 has been positioned at the predetermined position 384 in the EM field by adjusting the position of the EM field generator 302 based on the determined position of the EM sensor 204. In this example, the EM field generator 302 can be considered "set" here because its position is adjusted relative to the EM sensor 204. Notably, the movement of the EM field generator 302 can be achieved by the robot arm 212 to which the EM field generator 302 is attached. Furthermore, so that all positions (e.g., the EM sensor position, the predetermined position 384, and the position of the EM field generator 302) can be handled within a single space (e.g., the robot coordinate frame), points in the EM coordinate frame, such as the position of the EM sensor 204 and the predetermined position 384, can be mapped to the robot coordinate frame using kinematic alignment based on the robot arm.

[0156] In Figures 30A and 30B, a predetermined location within 384 within the working volume 306 of the EM field includes the center of the working volume 306. However, this does not have to be the case in all embodiments, and other predetermined locations 384 within the working volume 306 can be used. For example, another possible predetermined location 384 may include a location at one extreme end of the working volume 306 so that the EM sensor 204 can move across the entire working volume 306 before the EM field generator 302 needs to be repositioned to continue tracking the EM sensor 204.

[0157] In some embodiments, the EM field generator 302 can be positioned such that the EM sensor 204 is positioned within a predetermined region or portion of the working volume 306 of the EM field, rather than at a specific predetermined location. Figures 31A and 31B are perspective views showing an example of moving the EM field generator 302 using a robotic arm 212 so that the EM sensor 204 is positioned within a predetermined region 386 within the working volume 306. In Figures 31A and 31B, the predetermined region 386 is shown, representing a subdivision of the working volume 386. In the figures, both the working volume 306 and the predetermined region 386 are represented as right-angle prisms. However, this is only an example, and other shapes are possible for the working volume and the predetermined region 386. Furthermore, the shape of the predetermined region 386 does not have to correspond to the shape of the working volume 306. For example, the working volume 306 may include the shape of a right-angle prism, and the predetermined region 386 may include the shape of a sphere within the working volume 306. In some embodiments, the predetermined region 386 may represent a portion of the working volume 306 in which the position of the EM sensor 204 can be determined with greater precision. Therefore, it may be desirable to position the EM field generator 302 relative to the EM sensor 204 so that the EM sensor 204 is located within the predetermined region 386. In some embodiments, the predetermined region 386 may include the entire working volume 306.

[0158] Figure 31A shows the system in a first state before the position of the EM field generator 302 is adjusted so that the EM sensor 204 is positioned within a predetermined region 386. As shown, the EM sensor 204 is positioned within the working volume 306 of the EM field, but not within the predetermined region 386. To position the working volume 306 so that the EM sensor 204 is positioned within the predetermined region 386, the EM field generator 302 may be moved in the direction of the arrows shown using the robotic arm 212 to which it is attached. This movement causes corresponding movements of the working volume 306 and the predetermined region 386. Figure 31B shows the system in a second state after the position of the EM field generator 302 has been adjusted so that the EM sensor 204 is positioned within the predetermined region 386. As shown, the EM field generator 302 has been moved using the robotic arm 212 so that the EM sensor 204 is now positioned within the predetermined region 386. As described above, the alignment described above allows points in the EM coordinate frame to be mapped to corresponding points in the robot coordinate frame based on the kinematics of the arm, so that all positions can be represented in a single space.

[0159] Examples in Figures 31A-31B and 32A-32B include adjusting the position of the EM field generator 302 relative to the EM sensor 204. In some embodiments, the system may, in addition, or alternatively, adjust at least one of the orientation and position of the EM field generator 302 by determining the orientation of the EM sensor 204 in the EM field and commanding the movement of the first robot arm 212 based on the determined orientation of the EM sensor 204. In certain examples, the accuracy to which the position of the EM sensor 204 can be determined within the working volume 306 may be affected by the orientation of the EM sensor 204 relative to the EM field generator 302. Therefore, in some embodiments, the processor 380 is configured to adjust at least one of the orientation and position of the EM field generator based on the determined orientation of the EM sensor in order to improve the accuracy of the determined position of the EM sensor 204 in the EM field. This may include, for example, using a robotic arm 212 to which the EM field generator 302 is attached to adjust the pitch, yaw, and / or roll of the EM field generator 302 relative to the EM sensor 204.

[0160] As described above, the examples in Figures 31A-31B and 32A-32B generally relate to determining the position of the EM field generator 302 relative to the determined positions of the EM sensors 204 in order to set up the EM field generator 302 for treatment. This setup is facilitated by mounting the EM field generator 302 on a robotic arm 212 such that its position can be adjusted by the robot. Robotic adjustment of the position of the EM field generator 302 also facilitates instrument tracking, which may occur when one or more of the EM sensors 204 move during treatment. As described above, one or more of the EM sensors 204 may be positioned on a medical tool or instrument that is moved during treatment. The EM sensors 204 may enable tracking of the movement of these tools. As illustrated in the following examples, when movement of the EM sensors 204 is detected, a new position of the EM field generator 302 can be determined, and the EM field generator 302 can be moved using the robotic arm 212 to which it is mounted.

[0161] Figures 32A and 32B are perspective views showing an example of moving an EM field generator 302 using a robotic arm 212 along a path 390 that tracks the path 388 of movement of an EM sensor 204. In this way, the path 390 of the electromagnetic field generator 302 tracks, follows, or mirrors the path 388 of the EM sensor 204. In the example of Figures 32A and 32B, the system is configured to maintain the position of the EM field generator 302 such that the EM sensor 204 remains positioned at a predetermined location 384 (the center of the working volume 306 in the illustrated example) as the EM sensor 204 moves along the path 388.

[0162] During a medical procedure, a medical device (such as the medical device 214 in Figures 23 and 29) including an EM sensor 204 positioned on it can be navigated through the patient's anatomical structure. This movement of the EM sensor 204 (and the corresponding device 214) is represented by a path 388. Figure 32A shows a system with the EM sensor 204 positioned at a first location along the path 388. If the EM sensor 204 continues to move along the path 388 while the EM field generator 302 remains stationary, the EM sensor 204 will no longer be positioned at a predetermined location 384 within the working volume 306. In order to maintain the EM sensor 204 at the predetermined location 384 within the working volume 306, the EM field generator 302 must move along a corresponding path 390. In this way, the EM field generator 302 tracks or follows the movement of the EM sensor 204. Figure 32B shows the system after the EM sensor 204 has moved along its path 388. Since the EM field generator 302 is also moving along the corresponding path 390, the EM sensor 204 remains positioned at a predetermined location 384 within the working volume 306. This is made possible by moving the EM field generator 302 together with the robotic arm 212 to which it is mounted.

[0163] In some embodiments, the tracking of the EM sensor 204 by the EM field generator 302 does not need to have a direct correspondence. For example, the path 390 of the EM field generator 302 does not need to directly correspond to the path 388 of the EM sensor 204. Figures 33A to 33D show an example.

[0164] Figures 33A to 33D are perspective views showing an example in which an EM field generator 302 is moved using a robotic arm 212 so that the EM sensor 204, which is moving along a path 388, remains positioned within a predetermined region 386 of the working volume 306. As shown, in this example, the EM field generator 302 tracks the movement of the EM sensor 204 but does not move along a path that directly corresponds to the path 388 of the EM sensor 204. In this example, the system is configured to adjust the position of the EM field generator 302 if the EM sensor 204 moves to or outside the boundary of the predetermined region 386 of the working volume 306. For example, the processor 380 may be configured to command the movement of the EM field generator 302 using the first robotic arm 212 so that the EM sensor 204 remains positioned within a predetermined region 386 of the working volume 306 of the EM field while the medical device 214 containing the EM sensor 204 is moving.

[0165] Figure 33A shows the system in the first state. As shown, the EM sensor 204 is positioned within a predetermined region 386 of the working volume 306 and moves along the path 388. Figure 33B shows the system in the second state. As shown, the EM sensor 204 continues to move along the path 204 but is still positioned within the predetermined region 386. Since the EM sensor 204 is still positioned within the predetermined region 386, the EM field generator 302 has not yet moved. Figure 33C shows the third state, where the EM sensor 204 has reached the edge or boundary of the predetermined region 386. If the EM sensor 204 continues to move along the path 388 while the EM field generator 302 remains stationary, the EM sensor 204 will move outside the predetermined region 386. In order to keep the EM sensor 204 within the predetermined region 386, the EM field generator 302 needs to move in the direction of the arrow shown. This movement can be determined by a processor 380, which can, for example, instruct a robotic arm 212 to move the EM field generator 302. Figure 33D shows the system in a fourth state after the EM field generator 302 has been moved. As shown, the movement allows the EM field generator 302 to be repositioned so that the EM sensor 204 is repositioned within a predetermined region 386, even if the EM sensor 204 continues to move along the path 388. In this way, the EM field generator 302 moves again to track or follow the movement of the EM sensor 204, even if the movement of the EM field generator 302 does not directly correspond to the movement of the EM sensor 204, as in the example of Figures 32A and 32B.

[0166] Figure 34 is a flowchart providing an exemplary method 400 for moving an EM field generator coupled to a robotic arm based on the determined positions of EM sensors within the EM field. Method 400 can be performed, for example, by a processor 380, and can provide configuration of the electromagnetic field generator and instrument tracking. Method 400 begins with block 401, which includes determining the positions of EM sensors of a medical instrument within a working volume of an EM field generated by an EM field generator coupled to a first robotic arm. The positions of the EM sensors can be determined relative to an EM coordinate frame associated with the EM field generator. In some embodiments, the positions of the EM sensors may further be determined relative to a robot coordinate frame associated with the first robotic arm by applying alignment determined based on the kinematics of the robotic arm as described above.

[0167] Method 400 includes a block 403 on which a medical device is moved. The movement of the medical device causes a corresponding movement of an EM sensor positioned thereon. In some embodiments, the medical device comprises a manually controlled instrument, and the movement of the medical device is achieved manually. In other embodiments, the medical device includes a robotically controllable instrument. For example, the medical device may be coupled to a second robotic arm, and moving the medical device may include moving the instrument using the second robotic arm. In some embodiments, moving the instrument using the second robotic arm includes articulating the second robotic arm. The medical device may be coupled to an instrument drive mechanism, and moving the medical device using the second robotic arm may include acting on the medical device using the instrument drive mechanism. The movement of the instrument may be detected by determining that the position of an EM sensor attached to the instrument has moved.

[0168] Method 400 includes a block 405 which includes commanding a first robotic arm to move the EM field generator in response to the movement of a medical device such that the EM sensor remains positioned within the working volume of the EM field generator. For example, if the movement of the medical device causes the EM sensor to move outside the working volume, the EM field generator can be repositioned using the first robotic arm to maintain the EM sensor within the working volume of the EM field.

[0169] In some embodiments, the EM field generator tracks the movement of a medical device by commanding a first robotic arm to move the EM field generator. For example, commanding the first robotic arm to move the EM field generator may include moving the EM field generator so that the EM sensor is positioned or remains positioned within a predetermined area of ​​the EM field working volume, as shown in the examples of Figures 31A-31B and 33A-33D. In another example, commanding the first robotic arm to move the EM field generator may include moving the EM field generator so that the EM sensor is positioned or remains positioned at a predetermined location within the EM field working volume, as shown in the examples of Figures 30A-30B and 32A-32B.

[0170] In some embodiments, method 400 may optionally include determining the orientation of the EM sensor within the EM field and adjusting at least one of the orientation and position of the EM field generator by commanding the movement of the first robot arm based on the determined orientation of the EM sensor. As described above, adjusting at least one of the orientation and position of the EM field can improve the accuracy of the determined position of the EM sensor within the working volume of the EM field.

[0171] In addition, Method 400 can be performed as a loop, as shown by the dashed line 407, to continuously track the EM sensor and adjust the position of the EM field generator accordingly. That is, the position of the EM sensor can be determined in multiple discrete time steps, and the EM field generator can be readjusted based on each newly determined position.

[0172] The examples provided above show that a robotic system, such as the system 300 shown in Figures 23 and 29, can be configured to use a robotic arm 312 to move an EM field generator 302 to set up the EM field generator 302 for one EM sensor 204 and / or track the movement of one EM sensor 204. However, the system 300 is not limited to embodiments having one EM sensor 204. For example, a processor 380 may be configured to determine the positions of multiple EM sensors 204 in the EM field, determine the generator position of the EM field generator 302 based on the determined positions of the multiple EM sensors 204, and instruct the first robotic arm 212 to move the EM field generator 302 to the generator position. That is, the system 300 can be configured to set up and track an EM field generator based on multiple EM sensors 204. Examples of use based on multiple EM sensors 204 are provided with respect to Figures 35A to 37.

[0173] Figures 35A and 35B are perspective views showing an example of moving the EM field generator 302 to the electromagnetic field generator position using a robotic arm 212 based on the determined positions of multiple EM sensors 204 within the EM field. In some cases similar to the previous example, this can be done during the setup of the electromagnetic field generator and can be advantageously used to position the EM field generator 302 relative to the multiple EM sensors 204.

[0174] Figure 35A shows the system in a first state. As shown, in the first state, the multiple EM sensors 204 are positioned within the working volume 306 of the EM field generator 302. However, as shown in the figure, the multiple EM sensors 204 are not centered within the working volume 306. To center the multiple EM sensors 204 within the working volume 306, an electromagnetic field generator position 392 can be determined. The electromagnetic field generator position 392 can represent a position to which the EM field generator 302 can be moved in order to center the multiple EM sensors 204 within the working volume. In the illustrated embodiment, the generator position 392 is represented as x. The EM field generator 302 must move in the direction of the arrow shown in the figure to reach the generator position 392, which causes a corresponding movement of the working volume 306.

[0175] The generator position 392 can be determined based on the determined positions of the multiple EM sensors 302. In one example, the processor 380 is configured to determine the generator position 392 based on determining the center of gravity of the determined positions of the multiple EM sensors 204. The generator position 392 can then be determined by finding a position to which the EM field generator 302 is moved so that the center of gravity of the positions of the multiple EM sensors 204 is positioned at a predetermined position (such as the center or other desired position) within the working volume 306. As described above, the determined positions of the multiple EM sensors can be mapped to a robot coordinate frame using the aforementioned alignment, based on the kinematics of the robot arm 212 to which the EM field generator 302 is attached.

[0176] Figure 35B shows the system in a second state after adjusting the position of the EM field generator 302 to the generator position 392. As shown, the multiple EM sensors 204 are now positioned closer to the center of the working volume 206. Thus, the system can be easily configured and positioned relative to the multiple EM sensors 204. Although this example describes "centering" the multiple EM sensors 204 within the working volume 306, the generator position 392 may be determined to position the multiple EM sensors at other locations within the working volume 306.

[0177] Figures 36A, 36B, and 36C are perspective views illustrating an example of using a robotic arm 212 to readjust the electromagnetic field generator position 392 of an EM field generator 302 based on the determined positions of multiple EM sensors 204 within a working volume 306, while at least one of the multiple EM sensors 204 has moved. For example, a processor 380 may be configured to detect the movement of at least one of the EM sensors 204, determine a new generator position 392 for the EM field generator 302 based on the detected movement, and instruct the first robotic arm 212 to move the EM field generator 302 to the new generator position 392.

[0178] As shown in Figure 36A, one of the EM sensors 204 is moving along path 388. It may be desirable to adjust the position of the electromagnetic field generator 302 based on this movement. For example, if not adjusted, the EM sensor 204 may move along the path to a location outside the working volume 306, at which point tracking of the EM sensor 204 is lost.

[0179] Therefore, as shown in Figure 36B, the new electromagnetic field generator position 392 can be determined based on the positions of the multiple EM sensors 204 and taking into account the movement of one EM sensor 204 along the path 388. The new electromagnetic field generator position 392 can be based, for example, on the current center of gravity of the positions of the multiple EM sensors 204. As shown, the EM field generator 302 needs to move in the direction of the illustrated arrow to reach the new generator position 392. Figure 36C shows the system after moving to the new electromagnetic field generator position 392. As shown, the positions of the multiple EM sensors 204 are now centered again within the working volume 306. This can advantageously allow the position of the EM field generator 302 to be continuously adjusted to optimize the position of the working volume 306 so that the positions of the multiple EM sensors 204 remain positioned within the working volume 306 for as long as possible, even as they move away from each other.

[0180] Figure 37 is a flowchart illustrating an exemplary method 410 for determining the position of an electromagnetic field generator mounted on a robotic arm based on the determined positions of multiple EM sensors within the EM field. Method 410 begins with block 411, which includes generating an EM field using an EM field generator coupled to a first robotic arm.

[0181] In block 413 of method 410, the embodiment can determine the positions of multiple EM sensors in the EM field. This can be achieved by determining the positions of multiple EM sensors in the EM field relative to an EM coordinate frame associated with an EM field generator, determining the alignment between the EM coordinate frame and the robot coordinate frame associated with the first robot arm based on determining the position of the EM field generator in the robot coordinate frame, and determining the positions of multiple EM sensors in the robot coordinate frame based on the alignment. The determination of the position of the EM field generator in the robot coordinate frame can be based on the kinematics of the first robot arm.

[0182] In block 415, method 410 includes determining the generator position of the EM field generator based on the determined positions of a plurality of EM sensors. Determining the generator position may be based on determining the centroid of the positions of the plurality of EM sensors 204. The EM field generator position may be a position to which the EM field generator can be moved, which will desirablely position the plurality of EM sensors within the working volume of the EM field.

[0183] Block 417 includes commanding the first robotic arm to move the EM field generator to the generator position. For example, the system can use the first robotic arm to move the EM field generator to the generator position. Once the EM field generator is moved, the position of the work volume is repositioned accordingly relative to the position of the EM sensor.

[0184] As indicated by the dashed line 419, method 410 can be performed as a loop for continuously adjusting the position of the EM field generator, as described above with reference to Figures 36A to 36C. For example, method 410 may also include detecting the movement of at least one of a plurality of EM sensors, determining a new generator position for the EM field generator based on the detected movement, and commanding a first robotic arm to move the EM field generator to the new generator position.

[0185] ii. Electromagnetic field generator setup and equipment tracking with expanded working volume The previous section described a robotic medical system that includes a robot-controllable electromagnetic field generator that can be moved with the system's robotic arm to facilitate the setup of the electromagnetic field generator and tracking of the device. In some of the examples, it was shown that the EM field generator may be moved with the robotic arm to track EM sensors, for example, to keep moving EM sensors within the working volume of the EM field generated by the EM field generator. By moving the EM field generator and, correspondingly, the working volume of its EM field, the system described herein can be considered to have an "extended" working volume. That is, by repositioning the EM field generator and the working volume, the position of the EM sensor can be detected and determined over an area larger than the working volume of a stationary electromagnetic field generator.

[0186] This section provides additional examples to elaborate on the concept of "expanding" the working volume of an EM field generator by moving the EM field generator using a robotic arm. In particular, the examples demonstrate how EM sensors that are too far apart to simultaneously fit within the working volume of the EM field generator can be tracked by moving the EM field generator, detecting the position of the EM sensors, and mapping the detected positions of the EM sensors to a robotic coordinate frame. This offers a significant advantage over conventional systems using stationary electromagnetic field generators, where the tracking of EM sensors may be lost if the EM sensors move outside the stationary working volume. The examples in this section refer to the exemplary system 300 in Figures 23 and 29, but may be implemented using other systems including a robot-controllable electromagnetic field generator.

[0187] Herein, a first example is described with reference to Figures 38A-38C, which show the system 300 at different stages, in order to illustrate how the system 300 expands the working volume 306 of the EM field generator 302. As shown in Figure 38A, which shows some components of the system at the first stage, the system includes an EM field generator 302 configured to generate an EM field having a working volume 306 in which the position of the EM sensor 204 can be determined relative to the EM coordinate frame 308, as previously described. The EM field generator 302 is coupled to a robotic arm 212 configured to move in order to adjust the position of the EM field generator 302. As the robotic arm 212 moves the EM field generator 302, the position of the working volume 306 of the EM field generator 302 also moves.

[0188] Figure 38A also shows two EM sensors 204A and 204B. As shown, in the first stage, neither of the two EM sensors 204A nor 204B is positioned within the working volume 306 of the electromagnetic field generator 302, and therefore the positions of the EM sensors 204A and 204B are not determinable in the first stage. Furthermore, the two EM sensors 204A and 204B are positioned far enough apart that they cannot both be positioned within the working volume 306 at the same time. Therefore, as shown in Figures 38B and 38C, in order to track the positions of the EM sensors 204A and 204B, the EM field generator 302 can be moved using the robot arm 212 to different positions where each of the EM sensors 204A and 204B can be determined and mapped to the robot coordinate frame 216 to "expand" the working volume 306 of the EM field generator.

[0189] For example, a processor 380 of a system 300 that communicates with an EM field generator 302 and a robot arm 212 to which the EM field generator 302 is attached can be configured to move the EM field generator 302 to a first position in which the EM sensor 204 is positioned within the working volume 306. With the EM field generator 302 in the first position, the processor 380 can determine the position of the first EM sensor 204A within the working volume 306 relative to the EM coordinate frame 308 and map the position of the first EM sensor 204A to the robot coordinate frame 216 based on the kinematics of the robot arm with the EM field generator 302 in the first position. The processor 380 can then cause the robot arm 212 to move the EM field generator 302 to a second position in which the second EM sensor 204B is positioned within the working volume. Furthermore, with the EM field generator 302 in the second position, the processor 380 can determine the position of the second EM sensor 204B in the working volume 306 relative to the EM coordinate frame 308, and map the position of the second EM sensor 204B to the robot coordinate frame 216 based on the kinematics of the robot arm 212 with the EM field generator 302 in the second position. In this way, the positions of both EM sensors 204A and 204B can be presented in the robot coordinate frame 216.

[0190] Figure 38B shows an exemplary second stage of the system, where the EM field generator 302, along with the robot arm 212, is moved to a first position where the first EM sensor 204A is positioned within the working volume 306. With the EM field generator 302 in this position, the position of the EM sensor 204A can be determined relative to the EM coordinate frame 308, and then its position can be mapped to the robot coordinate frame 216 using the kinematic alignment of the robot arm 212 described above. In Figure 38B, the EM sensor 204A is shaded in black to indicate that its position was initially determined within the EM coordinate frame 308 but then mapped to the robot coordinate frame 216.

[0191] Figure 38C shows an exemplary third stage of the system, where the EM field generator 302, along with the robot arm 212, has been moved to a second position where the second EM sensor 204B is positioned within the working volume 306. With the EM field generator 302 in this position, the position of the EM sensor 204B can be determined relative to the EM coordinate frame 308, and then its position can be mapped to the robot coordinate frame 216 using the kinematic alignment of the robot arm 212 described above. As before, the EM sensor 204B is shaded black to indicate that its position has been mapped to the robot coordinate frame 216. The EM sensor 204A is also still shaded black to indicate its position within the robot coordinate frame 216.

[0192] Considering Figure 38C, it can be seen that the EM sensors 204A and 204B are spaced far enough apart that they cannot both fit within the working volume 306 of the EM field generator 302 at the same time, but the positions of both EM sensors 204A and 204B can be represented simultaneously or substantially concurrently within the robot coordinate frame 216. In some embodiments, the processor 380 is further configured to move the EM field generator 302 back and forth between a first position and a second position to frequently update and track the positions of the first EM sensor 204A and the second EM sensor 204B (e.g., at about 40 Hz, or both in smaller and larger time steps).

[0193] Furthermore, the examples described with reference to Figures 38A-38C are presented to simplify the description of the EM field generator 302 in a first position (Figure 38B) and a second position (Figure 38C). In practice, the system may be configured to continuously or nearly continuously detect any EM sensors 204 positioned within the EM field and map their positions to a robot coordinate frame 216. For example, the processor 380 may be configured to move the EM field generator 302 using a robot arm 212, and when an EM sensor is detected within the working volume 306, the processor 380 may be configured to (i) determine the position of the EM sensor 204 within the working volume 306 relative to the EM coordinate frame 308, and (ii) map the position of the EM sensor 204 to the robot coordinate frame 216 based on the kinematic orientation of the first robot arm 212.

[0194] In some examples, the initial position of the EM sensor 204 may not be known. To locate the EM sensor 204, the system can be configured to use a robotic arm 212 to move the EM field generator 302 along a search path or trajectory (e.g., along a predetermined path / pattern or range of motion). The search path may be configured to sweep the working volume 306 through a treatment volume larger than the working volume 306 to locate the EM sensor within the treatment volume. By sweeping through the treatment volume, the initial position of any EM sensor 204 within the treatment volume can be determined.

[0195] Figures 39A–39D illustrate one embodiment of a robotic medical system including a robot-controllable electromagnetic field generator 302 that can be moved along a search path 510 with a robotic arm 212 to detect the position of EM sensors 204. In the example of Figure 39A, the initial positions of the two EM sensors 204A and 204B are unknown. To locate the positions of the EM sensors 204A and 204B, Figure 39A shows an exemplary search path 510 along which the EM field generator 302 moves using the robotic arm 212. In the illustrated embodiment, the search path 510 comprises a substantially sinusoidal path configured for the EM sensors 204 to sweep the working volume 306 across the expected treatment volume or treatment site. The search path 510 shown in Figure 39A is provided as an example, and any number of paths including different shapes are possible.

[0196] Figure 39B shows the system in the second stage when the EM field generator 302 moves along the search path 510. In the second stage illustrated in Figure 39B, the first EM sensor 204A is positioned and detectable within the working volume 306 of the EM field generator 302. The position of the EM sensor 204A can be determined relative to the EM coordinate frame 308, and then its position can be mapped to the robot coordinate frame 216 using the kinematic alignment of the robot arm 212 described above. In Figure 39B, the EM sensor 204A is shaded in black to indicate that its position was initially determined within the EM coordinate frame 308 but then mapped to the robot coordinate frame 216. The EM field generator 302 can move along the search path 510.

[0197] Figure 39C shows the system in a third stage when the EM field generator 302 moves further along the search path 510. In the third stage illustrated in Figure 39C, the second EM sensor 204B is positioned and detectable within the working volume 306 of the EM field generator 302. The position of the EM sensor 204B can be determined relative to the EM coordinate frame 308, and then its position can be mapped to the robot coordinate frame 216 using the kinematic alignment of the robot arm 212 described above. In Figure 39B, the EM sensor 204B is shaded in black to indicate that it has been mapped to the robot coordinate frame 216. The EM field generator 302 can move along the search path 510. The position of the EM sensor 204A is also shown shaded in black to indicate that its position has been mapped to the robot coordinate frame.

[0198] Figure 39D shows the system in the fourth stage, after the EM field generator 302 has completed the search path 510. As shown, the positions of all EM sensors 204 (EM sensors 204A, 204B in the illustrated example) have been identified and mapped to the robot coordinate frame 216. In some embodiments, the system is configured to follow the search path 510 at the start of the procedure to identify the positions of the EM sensors 204. The search path 510 can be repeated periodically to determine whether any new EM sensors 204 have been introduced.

[0199] In some embodiments, the initial position of the EM sensor 204 can be determined by other means. For example, if the EM sensor 204 is included on a robot-controllable device, the system can estimate or determine the initial position of the EM sensor 204 based on the robot's kinematics. For EM sensors 204 not coupled to the system's robotic components, in some embodiments, a user can instruct or input the initial position of the EM sensor 204 to the system (e.g., via a user input device, controller, graphical user interface, etc.). For example, in the case of an EM patch sensor placed on a patient's chest to track respiration, the user placing the EM patch sensor can indicate to the system where the patch sensor is positioned.

[0200] As described above, in some examples, the position of the EM sensor 204 mapped to the robot coordinate frame 216 does not necessarily represent the live position of the sensor. Therefore, after the initial position of the EM sensor is determined, the system can determine a new tracking path that moves the EM field generator 302 to continue tracking the position of the EM sensor 204. The tracking path may be shorter than the search path, for example, which may allow for faster or more frequent remapping of the position of the EM sensor 204. An example is illustrated with reference to Figures 40A to 40D.

[0201] Figures 40A to 40D continue the examples in Figures 39A to 39D and further illustrate exemplary tracking paths 512 for the EM field generator 302. The tracking path 512 can be determined based on the determined position of the EM sensor 204 in the robot coordinate frame 216. For example, the tracking path 512 can be determined so that the EM field generator 302 is moved back and forth between the determined positions of the EM sensor 204 in the robot coordinate frame 216, and as a result the position is frequently remapped and updated. For example, when the EM field generator 302 is moved along the tracking path using the robot arm 212, the processor 380 may be configured to (i) redetermine the position of the EM sensor 204 in the working volume 306 relative to the EM coordinate frame 216, (ii) remap the position of the EM sensor 204 to the robot coordinate frame 308 based on the kinematic orientation of the robot arm 212, and (iii) determine the updated tracking path 510.

[0202] As shown in Figure 40A, the tracking path 512 can be shorter than the search path 510 (in Figures 39A-39D). This can be advantageous because it can limit unnecessary movement of the EM field generator (which could cause collisions with other objects in space) and limit the time the EM sensors are not positioned within the working volume 306. In the illustrated embodiment, the tracking path 512 includes a line that moves the EM field generator 302 back and forth between positions where each of the EM sensors 204A and 204B can be detected within the working volume 306 and remapped to the robot coordinate frame 216. Other shapes of the tracking path 512 are also possible.

[0203] Figure 40A shows an EM field generator 302 positioned at the first end of a tracking path 512 where the first EM sensor 204A is positioned within the working volume 306. At this position, the position of the first EM sensor 204A can be detected relative to the EM coordinate frame 308 and remapped to the robot coordinate frame 216. Figure 40B shows an EM field generator 302 positioned at the second end of a tracking path 512 where the second EM sensor 204B is positioned within the working volume 306. At this position, the position of the second EM sensor 204B can be detected relative to the EM coordinate frame 308 and remapped to the robot coordinate frame 216. In Figures 40A and 40B, the EM sensors 204A and 204B are shaded black to indicate that their positions have been mapped to the robot coordinate frame 216 (for example, when they are detected as the EM field generator 302 moves along the search path 510, as described in the earlier examples in Figures 39A to 39D).

[0204] Figure 40C illustrates why the position of the EM sensor 204A mapped to the robot coordinate frame may not be live. For example, as shown in Figure 40C, while the EM field generator 302 is positioned at the second end of the tracking path 512 so that the first EM sensor 204A is not positioned within the working volume 306, the first EM sensor 204A may move to a new position indicated as EM sensor 204A'. Notably, the position mapped ahead of EM sensor 204A is still shown (shaded in black). At this position, the system has not yet determined that EM sensor 204A has moved to the new position of EM sensor 204A'. In some embodiments, each sensor 204 is connected to a dedicated port of the system so that the system can identify and distinguish the sensors. Other mechanisms and methods for distinguishing sensors are equally possible.

[0205] However, Figure 40D shows that as the EM field generator 302 returns along the tracking path 512, it can detect a new position for the EM sensor 204A'. At this position, the new position of the first EM sensor 204A' can be detected relative to the EM coordinate frame 308 and remapped to the robot coordinate frame 216. To indicate that the new position of the EM sensor 204A' has been remapped to the robot coordinate frame 216, the EM sensor 204A' is shaded black in Figure 40D.

[0206] Figure 40D also shows that a new tracking path 512' can be determined based on the new position of the EM sensor 204A'. Thus, as the EM sensor 204 moves, its new position can be remapped to the robot coordinate frame 216, and the updated tracking path 512' can be determined. This process can be repeated throughout the procedure, enabling the tracking of various EM sensors 204 within an extended working volume, which is made possible by moving the EM field generator 302 using the robot arm 212 and mapping the positions of the detected EM sensors 204 to the robot coordinate frame based on the kinematic orientation of the robot arm 212.

[0207] Figure 41A is a flowchart illustrating an exemplary method 600 for extending the working volume of a robot-controllable electromagnetic field generator. Method 600 can be used to perform the functions described above, for example, with reference to Figures 38A-38C. Method 600 begins with block 601, where the EM field generator is moved to a first position. The first position may be a position where an EM sensor is detected within the working volume of the EM field generator. The EM field generator can be moved using a robotic arm to which it is mounted. In some embodiments, moving the EM field generator to the first position includes moving the EM field generator along a search path or tracking path.

[0208] The method proceeds to block 602. In block 602, the position of the first EM sensor is determined relative to the EM coordinate frame. The EM coordinate frame is associated with the EM field of the EM field generator. Next, in block 603, the position of the EM sensor in the EM coordinate frame (determined in block 602) is mapped to the robot coordinate frame. The robot coordinate frame is associated, for example, with the robot arm to which the EM field generator is coupled, and with other robotic components of the system. Mapping the position of the EM sensor to the robot coordinate frame can be based on the kinematic orientation of the first robot arm, which establishes the relationship between the EM coordinate frame and the robot coordinate frame, as the EM field generator is coupled to the robot arm as described above.

[0209] Next, method 600 can proceed to block 604. In block 604, the EM field generator is moved to a second position where the second EM sensor is detected within the working volume. As described above, the EM field generator may be moved to the second position using the robot arm to which it is mounted. In block 605, the position of the second EM sensor may be determined relative to the EM coordinate frame. Then, in block 606, the position of the second EM sensor in the EM coordinate frame (determined in block 605) is mapped to the robot coordinate frame, again using kinematic alignment of the robot arm. Finally, line 607 shows that method 600 can be performed as a loop for successively re-detecting the positions of the first and second EM sensors and re-mapping them to the robot coordinate frame.

[0210] Figure 41B is a flowchart showing another exemplary method 610 for extending the working volume of a robot-controllable electromagnetic field generator. Method 610 begins in block 611, which involves moving the EM field generator together with the robot arm to which it is mounted. In block 612, whenever an EM sensor is detected within the working volume of the EM field generator, the position of the EM sensor can be determined relative to an EM coordinate frame. In block 613, the position of the EM sensor can be mapped to a robot coordinate frame. The mapping may be based on the kinematics of the robot arm, as described above. Finally, line 614 shows that Method 610 can be performed as a loop for successively resensing the positions of a first and second EM sensor and remapping them to a robot coordinate frame.

[0211] iii. Matching percutaneously insertable devices with EM targets The robot-controllable EM field generator 302 can also facilitate the alignment of percutaneously inserted tools with other tools inserted endoscopically into the body. For example, as shown in Figure 27B, the ureteroscope 502 can be endoscopically guided to a position within the kidney. The operator may then wish to perform percutaneous insertion of a needle (or other instrument) or percutaneous insertion using a needle (or other instrument) to rendezvous with the ureteroscope 502. In some embodiments, the EM field generator 302 may include a needle guide (e.g., a channel, tube, or other structure) through which the needle (or other instrument) can be inserted. In other embodiments, the EM field generator 302 may define a space through which the needle guide or equivalent can advance and / or be positioned. For example, the needle guide may be configured to allow only one degree of freedom (insertion and / or retraction) for the needle inserted through it. The EM field generator 302 can be harmonized with the EM sensor 204 on the ureteroscope so that the needle is guided directly toward the EM sensor 204 when inserted through the needle guide. One such use case is a percutaneous procedure in which access to an internal organ or other tissue is made via needle puncture of the skin. A needle, rigidly mounted to a robotic end-effector and having a known conversion (rigid body definition) to the robotic end-effector, can therefore be harmonized with an EM beacon (target) inside the patient's body by robotic control. The needle can then be inserted through the skin along a robotically harmonized trajectory.

[0212] Figures 42A–42C illustrate an embodiment of a robotic medical system, such as the system 300 in Figures 23 and 29, which has a robot-controllable electromagnetic field generator 302 configured to facilitate alignment between a percutaneously insertable instrument 214 and an EM target 204A. The percutaneously insertable instrument 214 may include, for example, a needle, an access sheath, a laparoscopic instrument, or other types of percutaneously insertable instruments. In the example of Figures 42A–42C, it may be desirable to precisely insert the percutaneously insertable instrument 214 into a specific location within the body. To facilitate percutaneous insertion, an EM target 204A (which may be an EM sensor) can be positioned within the body at a desired location for the percutaneously insertable instrument 214. In some embodiments, the percutaneously insertable instrument 214 may also include an EM sensor 204B positioned on it.

[0213] In the embodiment shown in Figure 42A, the percutaneously insertable device 214 may be a robot-controllable device. The percutaneously insertable device 214 is shown mounted on a second robotic arm 212B configured to position and insert the percutaneously insertable device 214. As shown, the percutaneously insertable device 214 extends along axis 502. Generally, it is desirable for the percutaneously insertable device 214 to be inserted along axis 502. However, in order to accurately insert the percutaneously insertable device 214, axis 502 needs to be aligned with the EM target 204A.

[0214] Accordingly, the system's processor 380 may be configured to determine an alignment that maps the position in the EM coordinate frame associated with the EM field generator 302 to a position in the robot coordinate frame 216, based on the kinematic orientation of the first robot arm 212A to which the EM field generator 302 is attached. Based on the alignment, the processor 380 may be further configured to determine the position of the EM target 204A in the robot coordinate frame 216, and based on the position of the EM target 204A in the robot coordinate frame 216, the processor may be configured to move the second robot arm 212B to align the axis 502 of the percutaneously insertable instrument 214 with the EM target. Finally, the processor 380 may be configured to use the second robot arm 212B to insert the percutaneously insertable instrument 214 along the axis 502 toward the EM target 204A.

[0215] As described above, the alignment can be determined based on the position of the EM field generator 302 in the robot coordinate frame 216, and the position of the EM field generator having the robot coordinate frame 216 can be determined based on the kinematic orientation of the first robot arm 212A. The processor 380 may be further configured to determine the position of the EM target 204A in the robot coordinate frame 216 by determining the position of the EM target 204A in the EM coordinate frame 308 and using the alignment to map the position of the EM target 204A in the EM coordinate frame 308 to the position of the EM target 204A in the robot coordinate frame 216.

[0216] In some embodiments, the EM target 204A may be an EM sensor positioned on another robotic medical device configured to be inserted into a patient, such as an endoscope that is navigated through the body. This robotic medical device may be coupled to a third robotic arm configured to control the robotic medical device.

[0217] Figure 42A shows the system in a first state where the axis 502 of the percutaneously insertable device 214 is not aligned with the EM target 204A. Figure 42B shows the system in a second state after the axis 502 has been aligned with the EM target 204A. Figure 42C shows the system in a third state, showing the insertion of the percutaneously insertable device 214 along the axis 502 and rendezvous with the EM target 204A.

[0218] Figures 43A and 43B illustrate an embodiment of a robotic medical system, such as the system 300 in Figures 23 and 29, in which a robot-controllable electromagnetic field generator 302 is configured to facilitate alignment between an instrument guide 520 mounted on the electromagnetic field generator 302 and an EM target 204A. The instrument guide 520 may be configured so that a percutaneously insertable instrument can be inserted through its interior along an axis 522 during insertion into the patient. The instrument guide 520 can restrict the movement of the percutaneously insertable instrument to movement along the axis 522 only, thereby guiding the percutaneously insertable instrument toward the EM target 204 when the axis 522 is aligned with the EM target 204. In some embodiments, the instrument guide 520 is detachably coupled to the EM field generator 302. In other embodiments, the instrument guide 520 may be permanently coupled to or integrated with the EM field generator 302. In related embodiments, it should be noted that the instrument guide 520 may be configured to have a different shape / curve and / or length than the examples in Figures 43A and 43B.

[0219] Similar to the example described above, the axis 522 of the instrument guide 520 can be aligned with the EM target 204 to guide a percutaneously insertable instrument toward the EM target 20A. For example, the processor 380 may be configured to determine an alignment that maps a position in the EM coordinate frame 308 associated with the EM field generator 308 to a position in the robot coordinate frame 216, based on the kinematic orientation of the first robot arm 212. The processor 380 can also determine the position of the EM target 204 in the robot coordinate frame 216 based on the alignment, and based on the position of the EM target 204 in the robot coordinate frame 216, the processor 280 can cause the first robot arm 212 to move, thereby aligning the insertion axis 522 of the instrument guide 520 with the EM target 204.

[0220] Figure 43A shows the system in a first state where the axis 522 of the instrument guide 520 is not aligned with the EM target 204. Figure 43B shows the system in a second state after the axis 522 has been aligned with the EM target 204. With the system in the second state, a percutaneously insertable instrument can be inserted through the instrument guide 520 along the axis 522 toward the EM target 204.

[0221] The system may also include percutaneously insertable instruments. Percutaneously insertable instruments may include needles, access sheaths, laparoscopic instruments, or other types of percutaneously insertable instruments. Percutaneously insertable instruments may extend along an axis. In some embodiments, the system includes a second robotic arm coupled to the percutaneously insertable instrument and configured to move it. In such cases, the processor may further be configured to use the second robotic arm to align the axis of the percutaneously insertable instrument with the insertion axis, and to use the second robotic arm to insert the percutaneously insertable instrument through the instrument guide along the insertion axis toward the EM target 204. In some embodiments, the percutaneously insertable instrument may include an instrument that is manually inserted through the instrument guide 520.

[0222] In some embodiments, the EM target 204 includes an EM sensor positioned on a medical instrument configured to be inserted into a patient. The robotic medical instrument may be an endoscope. The system may include a third robotic arm coupled to the robotic medical instrument and configured to control it.

[0223] Figures 44A and 44B show one embodiment of a robotic medical system having a robot-controllable electromagnetic field generator 302 configured to facilitate alignment between an instrument guide 530 and an EM target 204. This example is similar to the examples in Figures 43A and 43B, except that the instrument guide 530 is coupled to a second robotic arm 212B instead of the EM field generator 302. Similarly, the processor 380 may be configured to determine an alignment that maps the position in the EM coordinate frame 308 associated with the EM field generator 302 to a position in the robotic coordinate frame 216, based on the kinematic orientation of the first robotic arm 212A to which the EM field generator 302 is attached. The processor 380 can further determine the position of the EM target 204 in the robotic coordinate frame based on the alignment, and move the second robotic arm 212B based on the position of the EM target in the robotic coordinate frame so that the insertion axis 532 of the instrument guide is aligned with the EM target 204. Figure 44A shows the system before the axis 532 of the instrument guide 530 is aligned with the EM target 204, and Figure 44B shows the system after alignment. At the position shown in Figure 44B, a percutaneously insertable instrument can be inserted through the instrument guide 530 toward the EM target 204.

[0224] In some embodiments, the percutaneously insertable device is coupled to a third robotic arm configured to move the percutaneously insertable device. In such cases, the processor may be further configured to use the third robotic arm to align the axis of the percutaneously insertable device with the insertion axis 532 of the device guide 530, and to use the third robotic arm to insert the percutaneously insertable device through the device guide 530 along the insertion axis 532 toward the EM target 204. In other embodiments, the percutaneously insertable device can be inserted manually through the device guide 530.

[0225] The system may also include a robotic medical device configured to be inserted into a patient. The EM target 204 may have an EM sensor on the robotic medical device. The robotic medical device may be coupled to another robotic arm configured to control the robotic medical device.

[0226] Figure 45A is a flowchart illustrating method 700 for aligning a percutaneously insertable device with an EM target using a robot-controllable electromagnetic field generator. Method 700 begins in block 701, where alignment is performed, mapping a position in the EM coordinate frame to a position in the robot coordinate frame. Alignment may be determined based on the kinematic orientation of the robot arm to which the EM field generator is attached, as described above.

[0227] Next, in block 702, the position of the EM target may be determined in the robot coordinate frame based on alignment. As described above, this may include determining the position of the EM target in the EM field generated by the EM field generator and mapping that position to the robot coordinate frame.

[0228] Based on the determined position of the EM target, in block 703, the axis of the percutaneously insertable device is aligned with the EM target. An example is shown in Figure 42B above. Finally, in block 704, the percutaneously insertable device can be inserted axially toward the EM target. An example is shown in Figure 42C above.

[0229] Figure 45B is a flowchart of a method 710 for aligning an instrument guide for a percutaneously insertable instrument with an EM target using a robot-controllable electromagnetic field generator. In block 711, method 700 includes determining an alignment that maps a position in the EM coordinate frame to a position in the robot coordinate frame. In block 712, the position of the EM target is determined in the robot coordinate frame based on the alignment. Finally, in block 713, method 700 includes aligning the axis of the instrument guide with the EM target. Examples are described above with reference to Figures 43A and 43B showing an instrument guide on an EM field generator, and Figures 44A and 44B showing an instrument guide mounted on a second robot arm.

[0230] iv. Strain detection The robot-controlled EM field generator 302 can also be used for strain detection. For example, the robot-controlled EM field generator can be commanded to move while a static EM sensor is in a region of interest, e.g., a needle insertion site, a biopsy site, or any stationary position within the working volume of the EM field generator. The EM field generator 302 can be moved by the robot arm 212 to which it is mounted, and the commanded robot motion (in the robot coordinate frame) can be compared to the recorded EM sensor trajectory (in the EM coordinate frame). The difference between the two trajectories may be a measure or indicator of the strain of the EM signal within the working volume of the EM field generator 302. A similar principle can also be used with non-static EM sensors whose motion is either known or has limited uncertainty. The accuracy of strain detection may depend on, or be limited by, the uncertainty of the knowledge of the EM sensor's motion / location.

[0231] Figure 46A shows an embodiment of a robotic medical system in which an EM field generator 302 is moved relative to a fixed EM position sensor 204, for example, to detect EM strain. As shown in the illustrated embodiment, the EM field generator 302 may be coupled to a first robotic arm 212. The EM field generator 302 may be configured to generate an EM field and may be associated with an EM coordinate frame, within which the position of an EM sensor, such as the illustrated EM sensor 204, may be determined by reference to the EM coordinate frame. The first robotic arm 212 may be configured to move to reposition the EM field generator 302. The first robotic arm 212 may be associated with a robotic coordinate frame. As described above, the motion of the robotic arm 212 within the robotic coordinate frame is known due to the known kinematics of the arm. Therefore, since the EM field generator 302 is coupled to the first robotic arm 212, the position of the EM field generator 302 within the robotic coordinate frame 212 may also be determined based on the kinematic orientation of the arm. As explained in more detail above, this relationship allows for the establishment of an alignment that can be used to determine the position of the EM sensor 204 within the EM coordinate frame of the EM field relative to the robot coordinate frame.

[0232] In the embodiment illustrated in Figure 46A, the EM sensor 204 may be provided in a fixed and stationary position to determine or detect EM strain in the EM field. For example, the EM sensor 204 may be mounted on a non-moving object such as a patient platform or other non-moving parts of a device. In some embodiments, the EM sensor 204 is positioned on a medical device, such as a robotically controlled medical device. In this case, the position of the EM sensor 204 remains fixed and stationary, as the medical device may remain stationary during the strain detection steps described with reference to Figures 46A to 47.

[0233] To detect EM strain, a processor communicating with the first robotic arm 212 (and consequently, the EM field generator 302) may be configured to instruct the first robotic arm 212 to move the EM field generator 302 along a robotic trajectory 850. In Figure 46A, the robotic trajectory 850 is represented by a dashed line, and the system is shown approximately midway through the movement so that the EM field generator 302 is positioned approximately midway along the robotic trajectory 850. In some embodiments, the movement of the EM field generator 302 by the first robotic arm 212 along the robotic trajectory 850 can be achieved under the direction or control of an operator, such as a physician, who can command the movement using a controller or other user interface. In other embodiments, the movement of the EM field generator 302 by the first robotic arm 212 along the robotic trajectory 850 can be provided automatically, for example, as part of an automated EM strain detection process operated by the system.

[0234] In Figure 46A, it is important to recognize that in this example, the EM field generator 302 is moved along the robot trajectory 850, while the EM sensor 204 remains stationary or fixed as described above. Figure 46A also shows the EM sensor trajectory 852, represented in the figure as a dashed line. However, the EM sensor trajectory 852 is not a result of the physical movement of the EM sensor 204, which remains fixed as described above. Rather, the EM sensor trajectory 852 is generated by the movement of the EM field generator 302 relative to the stationary EM sensor 204. For example, when the first robot arm 212 moves the EM field generator 302 along the robot trajectory 850, the position of the EM sensor in the EM coordinate frame associated with the EM field generator 302 is recorded, generating the EM trajectory 852.

[0235] Thus, the robot trajectory 850 is brought about by the movement of the EM field generator 302 by the first robot arm 212. The robot trajectory 850 can be detected, determined, and / or recorded based on the known kinematic movement of the first robot arm 212 during movement. In contrast, the EM sensor trajectory 852 is brought about by the movement of the EM field generator 302 relative to the stationary EM sensor 204. The EM sensor trajectory 852 can be detected, determined, and / or recorded based on detecting the EM sensor 204 within the EM field of the EM field generator 302 when the EM field generator 302 is moved while the EM sensor 204 remains stationary.

[0236] Since the EM sensor 204 remains fixed during the movement of the EM field generator 302 by the first robot arm 212, in the absence of any EM distortion, it is expected that the robot trajectory 850 will correspond to the EM sensor trajectory 852. This is because the movement of the EM field generator 302 should match the relative movement between the moving EM field generator 302 and the static EM sensor 204. Thus, the difference between the robot trajectory 850 and the EM sensor trajectory 852 can be analyzed to determine whether EM distortion exists. In some embodiments, the analysis can also provide a measure of the degree or severity of the EM distortion.

[0237] FIG. 46B shows a comparison of the robot trajectory 850 and the EM sensor trajectory 852. In this example, the starting points of each trajectory are aligned, although this is not necessary in all embodiments. As shown in FIG. 46B, it is clear that the robot trajectory 850 and the EM sensor trajectory 852 do not exactly correspond. To detect EM distortion, the difference between the trajectories can be determined. In some embodiments, the detected EM distortion indicates the EM distortion at the location of the fixed EM sensor 204.

[0238] Not all differences between the robot trajectory 850 and the EM sensor trajectory 852 are necessarily caused by EM distortion. For example, a robot system may not be able to determine robot motion with perfect accuracy for various reasons, including manufacturing tolerances, motor control limitations, and sensor accuracy. Therefore, some of the differences between the robot trajectory 850 and the EM sensor trajectory 852 may be caused by inaccuracies in determining robot motion. However, generally speaking, the inaccuracies in determining robot motion are small, for example, in the range of less than a millimeter. Similarly, the EM field generator 302 may be subject to EM noise, which can lead to a reduced resolution of the EM sensor position determined in the field. This can lead to inaccurate recording of the EM sensor trajectory 852 and cause differences not directly attributable to EM distortion. However, again, the EM field generator may be configured or selected such that the EM sensor noise is, in some embodiments, about 1 millimeter or less. Therefore, with respect to some embodiments of the robot system, inaccuracies in determining robot motion, and also inaccuracies in recording the EM sensor position in the EM field, can lead to potential inaccuracies of about 1 millimeter or less.

[0239] Therefore, in order to determine the presence of EM distortion, it may be beneficial to analyze the difference between the robot trajectory 850 and the EM sensor trajectory 852 with respect to a threshold selected to exclude or reduce contributions from other factors. In some embodiments, the threshold may be, for example, 1 millimeter, 1.25 millimeters, 1.5 millimeters, 1.75 millimeters, 2 millimeters, 2.25 millimeters, 2.5 millimeters, 2.75 millimeters, 3 millimeters, or greater. In many examples, it is beneficial to set the threshold to be greater than one or both of the error factors associated with the motion of the first robot arm or the error factors associated with noise from the EM sensor or generator.

[0240] A comparison between the robot trajectory 850 and the EM sensor trajectory 852 can be achieved in various ways. For example, the shapes of the robot trajectory 850 and the EM sensor trajectory 852 can be directly compared, as shown in Figure 46B. The shapes can be analyzed to determine how closely they correspond, and if the deviation between the shapes exceeds a threshold, the system can determine that EM strain exists and / or the degree of EM strain. In another embodiment, analyzing the robot trajectory 850 and the EM sensor trajectory 852 to determine the difference between them may include comparing a plurality of points along the robot trajectory 850 with a plurality of corresponding points along the EM sensor trajectory 852. In some embodiments, the plurality of points are determined against time associated with the movement of the EM field generator 302 using the first robot arm 212. For example, a point on the robot trajectory 850 at the start of the movement (e.g., t=0) can be compared with a corresponding point on the EM sensor trajectory 852. Subsequent points can then be compared at subsequent discrete time steps.

[0241] In some embodiments, if EM strain is detected, the system may be configured to use the first robot arm 212 to reposition the EM field generator 302 to a location that reduces the EM strain. For example, in some embodiments, the system may position the EM field generator 302 along the robot trajectory 850 where the difference between the robot trajectory 850 and the EM sensor trajectory 852 is reduced.

[0242] In some embodiments, this can be achieved by moving the EM field generator 302 along a second robotic trajectory to identify locations where EM strain can be reduced. Thus, the EM field generator 302 can move along a second robotic trajectory different from the first robotic trajectory 850. For example, the EM field generator 302 can move along the second robotic trajectory through a space different from the space covered along the first robotic trajectory 850. As described above, this is done while the EM sensor 204 remains stationary. The second robotic trajectory can be recorded along with a second corresponding EM sensor trajectory. These trajectories can be analyzed to determine whether it is possible to determine locations where EM strain can be reduced. If so, the EM field generator 302 can be moved to that position using the robotic arm 212. If not, additional robotic trajectories can be tried until a suitable location for the EM field generator 302 that reduces EM strain can be found.

[0243] In the example described above with respect to Figures 46A and 46B, the robot trajectory 850 is defined relative to the global robot reference frame, and the EM sensor trajectory 852 is defined relative to the EM field generator coordinate frame. In this case, in order to compare the robot trajectory 850 with the EM sensor trajectory 852, the EM field generator coordinate frame needs to be aligned with the global robot reference frame. This can be achieved, for example, as described above, since the EM field generator 302 is mounted on the robot arm 212 and a kinematic relationship is established between the global robot reference frame and the EM field generator coordinate frame.

[0244] Furthermore, the examples described above with respect to Figures 46A and 46B are somewhat simplified for the sake of understanding and illustration, assuming that only the position of the EM field generator 302 relative to the EM sensor 204 is changed, and its relative orientation remains unchanged. In more complex examples, both the position and orientation of the EM field generator 302 relative to the EM sensor 204 can be changed, and the detected position (and / or orientation) of the EM sensor 204 can be analyzed to determine whether (and / or the extent of) EM strain exists. In such cases, the determination of EM strain can be made relative to a global reference frame 854 associated with a cart to which the robot arm 212 is mounted, as shown in Figure 46C, which illustrates the case where the robot arm 212 to which the EM field generator 302 is mounted moves along a robot trajectory 850 that changes the position and orientation of the EM field generator 302 relative to the EM sensor 204. Since the cart and EM sensor 204 are not moving, any changes in detected position and / or orientation measured relative to the global reference frame 854 are due to strain (and negligible kinematic errors).

[0245] EM strain can be caused by any or all components positioned within the working volume of the EM field generator. Therefore, a similar process can be performed while moving other components to find their optimal or preferred positions.

[0246] Figure 47 is a flowchart illustrating an exemplary method 800 for EM strain detection. Method 800 is similar in many respects to the process described above with respect to Figures 46A to 46C. Method 800 can be performed using a robotic medical system, such as the system described above, in which the EM field is generally robotically controllable via mounting of the system to a robotic arm.

[0247] Method 800 begins with block 801. Block 801 may include moving an EM field generator, coupled to a first robotic arm, along a robotic trajectory by the first robotic arm while the EM sensor remains stationary in a certain location. As described above, the robotic trajectory can be determined based on the kinematics of the arm to which the EM field generator is mounted. In some embodiments, the robotic trajectory may be determined relative to a global reference frame, which may be associated with, for example, a cart to which the robotic arm is mounted. In some embodiments, block 801 is executed by a processor (or more processors) of the robotic medical system. For example, the processor may be configured to send commands to the first robotic arm to cause the EM field generator to move along the robotic trajectory. In some cases, the EM sensor may be fixed in a determinable position while the EM field generator moves along the robotic trajectory.

[0248] In block 802, the EM sensor trajectory of the EM sensor in the EM field associated with the EM field generator can be detected based on sensor data generated by the EM sensor as the EM field generator moves along the robot trajectory. Thus, as described above, the EM trajectory can be the recorded position of the EM sensor in the EM field during the same period that the EM field generator was moved by the robot arm along the robot trajectory.

[0249] In block 803, the robot trajectory and the EM sensor trajectory are analyzed to determine the difference between them. The analysis can be carried out in several ways, including, among other things, comparing the shapes of the trajectories and / or comparing multiple discrete points along the trajectories. Such comparisons may take into account changes in position and / or orientation.

[0250] In block 804, EM strain can be detected at the location of the EM sensor based on a comparison of the difference between the trajectories with a threshold. The threshold may be selected or determined to reduce or exclude contributions caused by factors other than EM strain, such as errors when recording or determining the robot's motion and / or when detecting the EM sensor position in the EM field.

[0251] Figure 48 is a flowchart of another exemplary method 900 for EM strain detection. Method 900 can be performed using a robotic medical system, such as the system described above, in which the EM field is generally robotically controllable via attachment of the system to a robotic arm. Method 900 begins in block 901, to which a first robotic arm is coupled to an EM field generator. The EM field generator is configured to generate an EM field and is associated with an EM coordinate frame. The position of the coupled EM field generator may be determined within the robot coordinate frame based on the kinematic orientation of the first robotic arm. In some embodiments, the position may further be determined relative to a global coordinate frame, such as a coordinate frame associated with the cart to which the robotic arm is mounted. The position of the EM sensor within the working volume of the EM field of the EM field generator may be determined based on the arm's pose, as described above, and mapped to a robot coordinate frame (or global coordinate frame).

[0252] In block 902, with the EM field generator in a first EM field generator position relative to the EM sensor, method 900 includes determining a first alignment between the EM coordinate frame and the robot coordinate frame based on the kinematics of the first robot arm with the EM field generator in the first position, and determining a first EM sensor position of the EM sensor in the robot coordinate frame based on the first alignment.

[0253] In block 903, method 900 includes using a first robotic arm to move an EM field generator from a first EM field generator position to a second EM field generator position relative to an EM sensor, the EM sensor remaining stationary during the movement. In some embodiments, the movement of the EM field generator by the first robotic arm may be achieved under the direction or control of an operator, such as a physician, who can command the movement using a controller or other user interface. In other embodiments, the movement of the EM field generator by the first robotic arm along a robotic trajectory may be provided automatically, for example, as part of an automated EM strain detection process operated by the system.

[0254] In block 904, with the EM field generator in the second EM field generator position, a second alignment between the EM coordinate frame and the robot coordinate frame is determined based on the kinematics of the first robot arm in the second EM field generator position, and the second EM sensor position of the EM sensor in the robot coordinate frame (or global coordinate frame) is also determined based on the second alignment. The second alignment is necessary because the EM field generator has been moved, and the new kinematic orientation of the robot arm at the second position is used to establish a new relationship between the robot coordinate frame and the EM coordinate frame.

[0255] In block 905, method 900 includes determining the difference between a first EM sensor position and a second EM sensor position. Since the EM sensors are not moving, the determined positions are expected to be the same. However, EM strain caused by different positions of the robot arm and the EM field generator (or other moved components) may cause the determined positions to vary. In some embodiments, changes in orientation may also be considered. In block 906, EM strain may be detected based on a comparison between the difference and a threshold (e.g., the difference exceeds a threshold). As described above, other factors may contribute to the difference. As described above, the threshold may be selected or determined to reduce the contribution of these other non-EM strain factors.

[0256] When viewed relative to a threshold, a difference exceeding the threshold can indicate the presence of EM strain. However, testing only the positions (and / or orientations) of two EM field generators may make it difficult to determine which position provided the most accurate positional determination. Therefore, it may be advantageous to continue testing multiple other EM field generator positions (and / or orientations) by repeating the relevant steps of: moving the EM field generator to a new position using a robotic arm; determining a new EM-to-robot coordinate alignment based on the new position of the EM field generator based on the kinematics of the robotic arm; and determining the position of the EM sensor within the robotic coordinate frame based on the newly determined alignment.

[0257] In some embodiments, this process can be continued to construct an EM strain map by determining strain at multiple locations. An EM field generator location that provides the least variation from other locations for determining the EM sensor position can be determined to have low EM strain, while an EM field generator location that provides the most variation from other locations for determining the EM sensor position can be determined to have high EM strain. In this way, the system or operator can determine the best positioning of the EM field generator and other components to reduce EM strain.

[0258] The EM distortions (and generally EM distortions) detected using the methods described above can be both static or dynamic. Static EM distortions are invariant and can be determined and removed from the EM measurement values by applying the corresponding correction values. Static EM distortions can generally be caused by a plurality of factors, including objects in the operating room that do not move and thus do not change during the procedure, as well as larger scale factors such as the Earth's magnetic field. Dynamic EM distortions are caused by factors that change over the course of the procedure. These can include any of the moving robotic components of the system, as well as other components (or even people) that can move through the operating room. Distortion detection can be used both for the detection and correction of static distortions, as well as for finding the optimal positions for moving components to reduce dynamic distortions.

[0259] v. Multimodal Sensor Fusion The robot-controlled EM field generator 302 may also be useful in enabling multimodal sensor fusion. As used herein, multimodal sensor fusion may refer to the synergistic simultaneous or parallel use of different sensor types during a single procedure. As an example, using the EM field generator 302 rigidly mounted on the robot arm 212, EM detection technology can be aligned with other imaging and detection modalities mounted on the robot arm 212 or within the robotic system. Aligning the EM detection modality with other imaging and detection modalities mounted on or part of the robot arm, cart, or base can facilitate the provision of, for example, simplified displays (data from multiple detection modalities can be displayed together in a unified manner), augmented reality, etc. This section describes an example in which EM detection technology may be used in conjunction with ultrasound technology to provide multimodal functionality. This may, for example, enable precise identification of the needle tip (the needle tip contains an EM sensor) in the ultrasound image. Generally, determining the needle tip in an ultrasound image can be difficult because artifacts can distort the image. Aligning the ultrasound and EM field imaging planes to a single space, such as a robot coordinate frame, can facilitate the determination of the needle tip in the ultrasound image. To achieve this, the ultrasound probe can be rigidly mounted on a robot arm, as described in more detail below. If the ultrasound imaging plane is kinematically known with respect to the coordinate frame of the EM field generator 302, an EM sensor inside the needle can be tracked in real time and overlaid on the ultrasound image.

[0260] Attaching an ultrasound probe to a robotic arm facilitates the alignment or calibration of the ultrasound probe's imaging plane. Calibrating the imaging plane of an ultrasound probe generally requires capturing three different ultrasound images from three different known positions. These three ultrasound images, along with their corresponding known positions, can be used to calibrate the imaging plane. Traditionally, additional equipment has been required to calibrate ultrasound probes used in surgical robotic systems. For example, an external position tracking system needs to be set up to determine the positions from which the three ultrasound images are captured. As a specific example, an optical sensor (such as an infrared LED) can be attached to the ultrasound probe. Position tracking systems such as Optitrack can detect the position of the optical sensor attached to the ultrasound probe and used to determine its position. This can be a disadvantage in surgical robot settings, as it requires additional equipment.

[0261] By attaching the ultrasonic probe to a robotic arm, the position of the ultrasonic probe can be easily determined relative to the robotic coordinate frame associated with the arm, based on the kinematics of the arm, in the same manner as described above for an EM field generator attached to the robotic arm. A reference ultrasonic image can be acquired at this reference position. The robotic arm can then move the ultrasonic probe to a first position where a first ultrasonic image is acquired, and this first position can be determined based on the kinematics of the arm. The robotic arm can then move the ultrasonic probe to a second position where a second ultrasonic image is acquired, and this second position can be determined based on the kinematics of the arm at the second position. The imaging plane of the ultrasonic probe can then be calibrated using the three images (base, first, and second) and positions (base, first, and second) without requiring external or additional equipment such as conventional position tracking devices.

[0262] Figure 49A shows a robotic medical system in which an ultrasound probe 1050 is mounted on a robotic arm 212 during a procedure for calibrating the imaging plane of the ultrasound probe. In the illustrated embodiment, the robotic arm 212 is coupled to the ultrasound probe 1050. For example, the ultrasound probe 1050 may be mounted on the distal end of the robotic arm 212 or an instrument device manipulator (IDM) positioned on the arm. The robotic arm 212 is configured to move to adjust the position of the ultrasound probe 1050. The robotic arm 212 is also associated with a robotic coordinate frame, as described above. In the illustrated embodiment of Figure 49A, three positions Bi, Bj, and Bk are shown. The robotic arm 212 moves to move the ultrasound probe 212 between these three positions. At each position, the kinematics of the arm can be used to determine the position of the ultrasound probe in the robotic coordinate frame.

[0263] At a first position Bi, the ultrasound probe 1050 can capture a first image. The ultrasound probe 1050 is then moved to a second position Bj using the robot arm 212. At the second position Bj, the ultrasound probe 1050 acquires a second ultrasound image. The ultrasound probe 1050 is then moved to a third position Bj using the robot arm 212. At the third position Bj, the ultrasound probe 1050 acquires a third ultrasound image. The system can calibrate the imaging plane of the ultrasound probe 1050 with respect to the robot coordinate frame based on the first ultrasound image, the first kinematics of the first robot arm with the ultrasound probe in the first ultrasound probe position, the second ultrasound image, the second kinematics of the first robot arm with the ultrasound probe in the second ultrasound probe position, and the third ultrasound image, the third kinematics of the first robot arm with the ultrasound probe in the second ultrasound probe position.

[0264] Once the imaging plane is calibrated and aligned to the robot coordinate frame, the robot system can utilize multiple sensing modalities simultaneously. Figure 49B shows an exemplary multimodal robot system. As shown in Figure 49B, the robot system may include a first robot arm 212A, a second robot arm 212B, and a third robot arm 212C. The robot arms can be mounted on a cart, patient platform, or other common structure so that each is associated with a common robot coordinate frame. In the illustrated embodiment, the ultrasound probe 1050 with a calibrated imaging plane is mounted on the first robot arm 212A. The EM field generator 302 is mounted on the second robot arm 212B. The EM field generator 302 is configured to generate an EM field in which the position of an EM sensor, such as an EM sensor 204, can be determined. The position of the EM sensor 204 can be determined within the robot coordinate frame using the EM-to-robot coordinate frame alignment procedure described above. Since the imaging plane of the ultrasonic probe is also aligned with the robot coordinate frame, the imaging planes of the EM sensor 204 are both determined within a common space (robot coordinate frame), and therefore, as shown in Figure 49B, both can be displayed on a common display.

[0265] In Figure 49B, the EM sensor 204 can be positioned at the distal end of a scope inserted into the patient's treatment area. The scope may be a robot-controlled medical instrument, mounted on and controlled by a different robotic arm. Alternatively, the scope may be a manually controlled scope. Figure 49B also shows that a medical instrument such as a needle can be mounted on a third robotic arm 212C. The position of the needle, which can be firmly mounted on the third robotic arm 212C, can also be determined relative to the robotic coordinate frame. Thus, the position of the needle can also be displayed along with the imaging plane and EM sensor data. In this way, various sensor technologies can be integrated into a single robotic system and used simultaneously during a single procedure. In this example, the rendezvous of the needle and the scope is facilitated by robotic data, ultrasound data, and EM data, improving the accuracy of the rendezvous and providing a comprehensive and improved surgical experience.

[0266] Figure 50 is a flowchart illustrating an exemplary method 1000 for calibrating the imaging plane of an ultrasound probe for use with a robotic medical system. Method 1000 begins in block 1001, where the ultrasound probe is coupled to a first robotic arm. The first robotic arm is configured to move to adjust the position of the ultrasound probe, and the first robotic arm is associated with a robotic coordinate frame. In block 1002, the ultrasound probe is moved to a first ultrasound probe position using the first robotic arm. The first ultrasound probe position may be a position from which a first ultrasound image used to calibrate the imaging plane of the ultrasound probe is captured. The first position may be determined based on the kinematics of the first robotic arm. In block 1003, a first ultrasound image is acquired with the ultrasound probe positioned at the first ultrasound probe position. In block 1004, the ultrasound probe is moved to a second ultrasound probe position using the first robotic arm. The second ultrasound probe position may be a position from which a second image can be captured, which is done in block 1005. The second image may also be used to calibrate the imaging plane of the ultrasound probe. The second position can be determined based on the kinematics of the arm. As indicated by arrow 1006, blocks 1004 and 1005 may be repeated to capture a third image at a third position, for example, as shown in Figure 49A. In some embodiments, these steps may be repeated a further number of times to capture further images at further positions.

[0267] In block 1007, the imaging plane of the ultrasonic probe relative to the robot coordinate frame can be calibrated based on a first ultrasonic image and the first kinematics of the first robot arm with the ultrasonic probe in the first ultrasonic probe position, a second ultrasonic image and the second kinematics of the first robot arm with the ultrasonic probe in the second ultrasonic probe position, and a third ultrasonic image and the third kinematics of the first robot arm with the ultrasonic probe in the third ultrasonic probe position. Once calibrated, the imaging plane is determined relative to the robot coordinate frame and can be used in conjunction with other sensing modalities (such as EM) that can similarly be determined relative to the robot coordinate frame.

[0268] In some embodiments, Method 1000 may also include coupling an EM field generator to a second robotic arm, the EM field generator being configured to generate an EM field, the EM field generator being associated with an EM coordinate frame, and the second robotic arm being configured to move to adjust the positions of the EM field generator and the second robotic arm associated with the robotic coordinate frame. Method 1000 may also include determining the alignment between the EM field coordinate frame and the robotic coordinate frame based on the kinematics of the second robotic arm. This can be achieved as described above. The first and second robotic arms may be mounted on a cart or patient platform that defines the relationship between the first and second robotic arms in the robotic coordinate frame. This may allow both the EM field generator and the ultrasound probe to be brought into the robotic coordinate frame. Method 1000 may also include positioning an EM sensor in the EM field and determining the position of the EM sensor in the robotic coordinate frame based on the alignment. In addition, method 1000 may also include displaying the imaging surface of the ultrasonic probe and overlaying an indication of the determined EM sensor position onto the displayed imaging surface of the ultrasonic probe, for example, as shown in Figure 49B.

[0269] While multimodal functionality has been described in relation to EM and ultrasound, other modalities can also be used. For example, in some embodiments, a depth sensor can be mounted on a robotic arm. The depth sensor may be used to align the position of a patient, a bed or platform, a fluoroscopy C-arm used during a procedure, and / or other items in the surgical space to a global robotic reference frame. In some embodiments, these positions may be represented as a heatmap or point cloud, as shown in Figures 51A and 51B, respectively. The depth sensor camera may be configured to measure the distance to objects within its field of view, as shown in Figure 51C, for example. Calibration for the depth sensor (for example, to align the output of the depth sensor to a robotic coordinate frame or a global coordinate frame) can be carried out in the same manner as the calibration of the ultrasound probe described above. Generally, depth sensors are calibrated by the manufacturer with respect to pixel and depth information. For use with a robotic system, it may be necessary to calibrate the orientation of the depth sensor relative to the robotic frame, as described herein, for example, which can be achieved in the same manner as the ultrasound probe described above.

[0270] In some embodiments, a robotic medical system can utilize multiple different sensor functions in combination. For example, the system may include an ultrasound probe, a camera or depth sensor, and an EM field generator, the output of which may be calibrated and aligned to a robot or global coordinate frame. In these embodiments, the ultrasound probe can provide imaging data about the patient's body. Similarly, the camera or depth sensor can provide imaging or other information about external biological structures, such as the patient's position, as well as external devices located in the operating room, such as a C-arm. This can be advantageous because knowing the patient's position relative to the robot coordinate frame may allow the system to identify, for example, a needle insertion site. In addition, knowing the position of other items in the operating room relative to the robot coordinate frame, such as a C-arm, may allow the system to know when an object that could cause distortion is approaching the electromagnetic field generator.

[0271] 3. System implementation and terminology. The implementations disclosed herein provide systems, methods, and apparatus for robot-controllable electromagnetic field generators.

[0272] When used herein, the terms “to join,” “joined,” “joined,” or other variations of the word “join” may indicate either indirect or direct connection. For example, when a first component is “joined” to a second component, the first component may be indirectly connected to the second component via another component, or directly connected to the second component.

[0273] Terms referring to processes and functions performed by specific computers as described herein may be stored as one or more instructions on processor-readable media or computer-readable media. The term “computer-readable media” means any available media that a computer or processor can access. Examples, but not limited to, such media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, compact disc read-only memory (CD-ROM), or other optical disc storage devices, magnetic disc storage devices, or other magnetic storage devices, or any other media that can be used to store desired program code in the form of instructions or data structures and that a computer can access. Note that computer-readable media may be tangible and non-temporary. As used herein, the term “code” may mean software, instructions, code, or data that is executable by a computing device or processor.

[0274] The methods disclosed herein include one or more steps or actions to achieve the described method. The method steps and / or actions may be interchangeable with one another without departing from the claims. In other words, the order and / or use of any particular steps and / or actions may be modified without departing from the claims, unless a particular order of steps or actions is required for the proper operation of the described method.

[0275] As used herein, the term “multiple” refers to two or more. For example, “multiple components” refers to two or more components. The term “decide” encompasses a wide variety of actions, and therefore “decide” may include calculating, arithmetic, processing, calculating, investigating, looking up (e.g., looking at a table, database or another data structure), confirming, etc. Also, “decide” may include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), etc. Also, “decide” may include resolving, selecting, electing, establishing, etc.

[0276] The phrase "based on" does not mean "based solely on" unless explicitly specified otherwise. In other words, the phrase "on the basis of" can mean both "based solely on" and "based at least on."

[0277] The preceding description of the disclosed implementations is provided to enable those skilled in the art to create or use the invention disclosed herein. Various modifications to these implementations will be readily apparent to those skilled in the art, and the general principles set forth herein may be applied to other implementations without departing from the scope of the invention. For example, those skilled in the art will understand that many corresponding alternative and equivalent structural details can be employed, such as equivalent methods for fastening, mounting, joining, or engaging tool components, equivalent mechanisms for producing specific operating motions, and equivalent mechanisms for delivering electrical energy. Accordingly, the disclosure herein is not limited to the explicitly described implementations, but should be given the broadest scope that corresponds to the principles and novel features disclosed herein.

[0278] [Implementation Method] (1) A robotic medical system, A first robotic arm configured to be coupled to an electromagnetic (EM) field generator and to move the EM field generator, One or more processors, Determining the EM position of the EM sensor in the EM field within the EM coordinate frame associated with the EM field generator, Determining the position of the EM field generator within the robot coordinate frame associated with the first robot arm, Based on the position of the EM field generator, the alignment between the EM coordinate frame and the robot coordinate frame is determined. A system comprising one or more processors configured to determine the position of the EM sensor within the robot coordinate frame based on the aforementioned alignment. (2) The system according to Embodiment 1, wherein one or more processors are configured to determine the position of the EM field generator based on the kinematics of the first robot arm. (3) The system according to Embodiment 1, wherein the EM field generator comprises a compact electromagnetic field generator (cFG). (4) The system according to Embodiment 1, wherein the first robotic arm is configured to be detachably coupled to the EM field generator. (5) Further comprising a second robotic arm configured to control the movement of medical instruments, The system according to Embodiment 1, wherein the robot coordinate frame is further associated with the second robot arm.

[0279] (6) The system according to Embodiment 5, wherein the first robot arm and the second robot arm are coupled to a movable cart. (7) The system according to Embodiment 5, wherein the first robotic arm and the second robotic arm are coupled to a patient platform configured to support a patient during a medical procedure. (8) The system according to embodiment 5, wherein the EM sensor is positioned on the medical device. (9) The system according to Embodiment 1, wherein one or more processors are further configured to move the EM field generator to the first robotic arm in order to enable access to the patient during a medical procedure. (10) The one or more processors The first robot arm is to move the EM field generator to a new position relative to the EM sensor, The system according to Embodiment 1, further configured to determine a new EM position of the EM sensor in the EM field within the EM coordinate frame associated with the EM field generator, wherein the new EM position has improved accuracy compared to the previous EM position.

[0280] (11) The system according to Embodiment 1, wherein one or more processors are further configured to move the first robot arm to center the EM sensor within the working volume of the EM field generator based on the position of the EM field generator and the position of the first robot arm. (12) A method for performing a robotic medical procedure, wherein the method is The method involves generating an electromagnetic (EM) field using an electromagnetic (EM) field generator coupled to a first robot arm, wherein the EM field is associated with an EM coordinate frame. The position of the EM field generator within the robot coordinate frame is determined based on the kinematics of the first robot arm, Based on the determined position of the EM field generator, the alignment between the EM coordinate frame and the robot coordinate frame associated with the first robot arm is determined. Determining the position of the EM sensor within the EM coordinate frame, A method comprising determining the position of the EM sensor within the robot coordinate frame based on the alignment described above. (13) The method according to Embodiment 12, wherein the determination of the position of the EM field generator in the robot coordinate frame is based on the kinematics of the first robot arm. (14) The method according to embodiment 12, further comprising detachably coupling the EM field generator to the first robot arm. (15) The method according to embodiment 12, further comprising moving the EM field generator with the first robot arm.

[0281] (16) The method of Embodiment 15, wherein moving the EM field generator to the first robotic arm includes moving the EM field generator to a position that allows access to a patient during a medical procedure. (17) The method of Embodiment 15, wherein moving the EM field generator on the first robotic arm includes moving the EM field generator to a new position related to improved accuracy in tracking the EM sensor within the EM field coordinate frame. (18) The method according to embodiment 15, wherein moving the EM field generator with the first robot arm includes centering the EM sensor within the working volume of the EM field generator. (19) The method of Embodiment 12, further comprising controlling a medical device coupled to a second robotic arm, wherein the robotic coordinate frame is associated with the second robotic arm. (20) The method according to embodiment 19, wherein the first robot arm and the second robot arm are coupled to a movable cart.

[0282] (21) The method according to Embodiment 20, further comprising positioning the movable cart relative to a patient platform configured to support the patient during the robotic medical procedure, such that a known spatial relationship exists between the robot coordinate frame and the patient platform. (22) The method according to embodiment 19, wherein the first robotic arm and the second robotic arm are coupled to a patient platform configured to support the patient during the robotic medical procedure. (23) The method according to embodiment 19, wherein the EM sensor is positioned on the medical device. (24) The method according to embodiment 12, wherein the EM field generator comprises a compact electromagnetic field generator (cFG). (25) A non-temporary computer-readable storage medium containing computer program instructions, wherein when the computer program instructions are executed by one or more processors, the one or more processors shall Determining the position of an electromagnetic (EM) field generator within a robot coordinate frame based on the kinematics of a first robot arm, wherein the EM field generator is coupled to the first robot arm. Based on the determined position of the EM field generator, the alignment between the EM coordinate frame and the robot coordinate frame associated with the first robot arm is determined, wherein the EM coordinate frame is associated with the EM field generated by the EM generator. Determining the position of the EM sensor within the EM coordinate frame, A non-temporary computer-readable storage medium that determines the position of the EM sensor within the robot coordinate frame based on the aforementioned alignment.

[0283] (26) The non-temporary computer-readable storage medium according to Embodiment 25, wherein the EM field generator comprises a compact electromagnetic field generator (cFG). (27) The non-temporary computer-readable storage medium according to Embodiment 25, wherein the first robotic arm is configured to be detachably coupled to the EM field generator. (28) When the computer program instruction is executed, the one or more processors further: A second robotic arm, configured to control the movement of medical instruments, is controlled. The non-temporary computer-readable storage medium according to Embodiment 25, wherein the robot coordinate frame is further associated with the second robot arm. (29) The non-temporary computer-readable storage medium according to Embodiment 28, wherein the first robot arm and the second robot arm are coupled to a movable cart. (30) The non-temporary computer-readable storage medium according to Embodiment 28, wherein the first robotic arm and the second robotic arm are coupled to a patient platform configured to support a patient during a medical procedure.

[0284] (31) A robotic medical system, A first robotic arm coupled to an EM field generator configured to generate an electromagnetic (EM) field, wherein the first robotic arm is configured to move the EM field generator, A medical device configured to be inserted into a patient, wherein the medical device is equipped with an EM sensor, One or more processors, Determining the position of the EM sensor within the EM field, A system comprising one or more processors configured to adjust the position of the EM field generator by commanding the movement of the first robot arm based on the determined position of the EM sensor. (32) The system according to Embodiment 1, wherein the EM sensor is positioned at a predetermined location within the EM field by adjusting the position of the EM field generator based on the determined position of the EM sensor. (33) The system according to embodiment 32, wherein the predetermined position within the EM field includes the center of the working volume of the EM field. (34) The system according to Embodiment 1, wherein adjusting the position of the EM field generator based on the determined position of the EM sensor includes adjusting the position of the EM field generator so that the EM sensor is positioned within a predetermined area of ​​the working volume of the EM field. (35) The one or more processors Determining the orientation of the EM sensor within the EM field, The system according to Embodiment 1, further configured to adjust at least one of the orientation and position of the EM field generator by commanding the movement of the first robot arm based on the determined orientation of the EM sensor.

[0285] (36) The system according to embodiment 35, wherein one or more processors are configured to adjust at least one of the orientation and position of the EM field generator based on the determined orientation of the EM sensor to increase the accuracy of the determined position of the EM sensor in the EM field. (37) The one or more processors The system according to Embodiment 1, further configured to instruct the first robotic arm to move the EM field generator so that the EM sensor is positioned within the working volume of the EM field while the medical device is being moved. (38) The system according to embodiment 37, wherein the movement of the EM field generator via the first robotic arm tracks the movement of the medical device. (39) The system according to embodiment 37, wherein one or more processors are configured to command the movement of the first robotic arm to move the EM field generator while the medical device is being moved, such that the EM sensor is positioned within a predetermined area of ​​the working volume of the EM field. (40) The system according to embodiment 37, wherein one or more processors are configured to command the movement of the first robotic arm to move the EM field generator while the medical device is being moved, such that the EM sensor is positioned in a predetermined location within the working volume of the EM field.

[0286] (41) The system according to embodiment 40, wherein the predetermined position includes the center of the working volume of the EM field. (42) Further equipped with a second robotic arm, The medical device is connected to the second robotic arm, The system according to embodiment 37, wherein one or more processors are configured to command the second robotic arm to move the medical device. (43) The system according to embodiment 42, wherein the first robotic arm and the second robotic arm are coupled to one of a cart and a patient platform, and the patient platform is configured to support a patient during a medical procedure. (44) The system according to embodiment 43, further comprising a third robotic arm coupled to one of the cart and the patient platform. (45) The one or more processors Determining the position of the EM sensor within the EM field relative to the EM coordinate frame associated with the EM field generator, The alignment between the EM coordinate frame and the robot coordinate frame associated with the first robot arm is determined based on determining the position of the EM field generator within the robot coordinate frame. The system according to Embodiment 1, configured to determine the position of the EM sensor within the robot coordinate frame based on the alignment described above.

[0287] (46) The system according to embodiment 45, wherein one or more processors are configured to determine the position of the EM field generator in the robot coordinate frame based on the kinematics of the first robot arm. (47) A robotic medical method, To determine the position of the EM sensor of a medical device within the working volume of an EM field generated by an electromagnetic (EM) field generator coupled to a first robotic arm, Moving the aforementioned medical device, A method comprising: instructing the first robotic arm to move the EM field generator in response to the movement of the medical device such that the EM sensor remains positioned within the working volume of the EM field generator. (48) The medical device is connected to the second robotic arm, The method according to embodiment 47, wherein moving the medical device includes moving the medical device using the second robotic arm. (49) The method of embodiment 48, wherein moving the device using the second robotic arm includes articulating the second robotic arm. (50) The medical device is coupled to a device drive mechanism, The method according to embodiment 48, wherein moving the medical device using the second robotic arm includes operating the medical device using the device drive mechanism.

[0288] (51) The method of embodiment 47, wherein the EM field generator tracks the movement of the medical device by commanding the first robotic arm to move the EM field generator. (52) The method of Embodiment 47, wherein commanding the first robot arm to move the EM field generator includes moving the EM field generator so that the EM sensor is positioned within a predetermined area of ​​the working volume of the EM field. (53) The method of Embodiment 47, wherein commanding the first robot arm to move the EM field generator includes moving the EM field generator so that the EM sensor is positioned in a predetermined location within the working volume of the EM field. (54) Determining the orientation of the EM sensor within the EM field, The method according to embodiment 47, further comprising adjusting the orientation and position of the EM field generator by commanding the movement of the first robot arm based on the determined orientation of the EM sensor. (55) The method according to embodiment 54, wherein adjusting at least one of the orientation and position of the EM field increases the accuracy of the determined position of the EM sensor within the working volume of the EM field.

[0289] (56) Determining the position of the EM sensor within the working volume of the EM field with respect to the EM coordinate frame associated with the EM field generator, The alignment between the EM coordinate frame and the robot coordinate frame associated with the first robot arm is determined based on determining the position of the EM field generator within the robot coordinate frame. The method according to embodiment 47, further comprising determining the position of the EM sensor within the robot coordinate frame based on the alignment. (57) The method according to embodiment 56, wherein determining the position of the EM field generator within the robot coordinate frame is based on the kinematics of the first robot arm. (58) A robotic medical system, An EM field generator configured to generate an electromagnetic (EM) field, A first robotic arm is coupled to the EM field generator and configured to move the EM field generator, The EM field generator and one or more processors that communicate with the first robot arm are comprising, Determining the positions of multiple EM sensors within the aforementioned EM field, Based on the determined positions of the plurality of EM sensors, the generator position of the EM field generator is determined, A system configured to command the first robot arm to move the EM field generator to the determined generator position. (59) The system according to embodiment 58, wherein one or more processors are configured to determine the generator position based on determining the centroids of the determined positions of the plurality of EM sensors. (60) Further comprising a medical instrument having an elongated shaft configured to be inserted into a patient, The system according to embodiment 58, wherein at least one of the plurality of EM sensors is positioned on the elongated shaft.

[0290] (61) Further equipped with a second robotic arm, The system according to embodiment 60, wherein the second robotic arm is coupled to the medical device so that the second robotic arm can move the medical device. (62) The system according to embodiment 61, wherein the first robotic arm and the second robotic arm are coupled to a cart or a patient platform configured to support a patient during a robotic medical procedure. (63) The system according to embodiment 60, wherein at least one of the EM sensors is a patch sensor located on the patient and configured to track the patient's respiration or other movement. (64) The one or more processors Determining the positions of the plurality of EM sensors in the EM field with respect to the EM coordinate frame associated with the EM field generator, The alignment between the EM coordinate frame and the robot coordinate frame associated with the first robot arm is determined based on determining the position of the EM field generator within the robot coordinate frame. The system according to embodiment 58, further configured to determine the position of the EM sensor within the robot coordinate frame based on the alignment. (65) The system according to embodiment 64, wherein the determination of the position of the EM field generator in the robot coordinate frame is based on the kinematics of the first robot arm.

[0291] (66) The one or more processors To detect the movement of at least one of the EM sensors, Based on the detected movement, the new generator position of the EM field generator is determined. The system according to embodiment 58, further configured to command the first robot arm to move the EM field generator to the new generator position. (67) A method for positioning an electromagnetic (EM) field generator during a robotic medical procedure, wherein the method is The method involves generating an EM field using an EM field generator attached to the first robotic arm, Determining the positions of multiple EM sensors within the aforementioned EM field, Based on the determined positions of the plurality of EM sensors, the generator position of the EM field generator is determined, A method comprising commanding the first robot arm to move the EM field generator to the generator position. (68) The method of embodiment 67, further comprising determining the generator position based on determining the centroids of the determined positions of the plurality of EM sensors. (69) The method according to embodiment 67, wherein at least one of the plurality of EM sensors is positioned on an elongated shaft of a medical device. (70) The method according to embodiment 69, wherein the medical device is coupled to a second robotic arm.

[0292] (71) The method according to embodiment 70, wherein at least one of the EM sensors is a patch sensor positioned on a patient and configured to track the patient's breathing or other movement. (72) Determining the positions of the plurality of electromagnetic (EM) sensors in the EM field with respect to the EM coordinate frame associated with the EM field generator, The alignment between the EM coordinate frame and the robot coordinate frame associated with the first robot arm is determined based on determining the generator position of the EM field generator within the robot coordinate frame. The method according to embodiment 67, further comprising determining the positions of the plurality of EM sensors within the robot coordinate frame based on the alignment. (73) The method according to embodiment 68, wherein the determination of the generator position of the EM field generator in the robot coordinate frame is based on the kinematics of the first robot arm. (74) To detect the movement of at least one of the plurality of EM sensors, Based on the detected movement, the new generator position of the EM field generator is determined. The method according to embodiment 67, further comprising commanding the first robot arm to move the EM field generator to the new generator position. (75) A robotic medical system, An EM field generator configured to generate an electromagnetic (EM) field, A first robotic arm is coupled to the EM field generator and configured to move it, A percutaneously insertable device that extends along the axis, A second robotic arm, which is attached to the percutaneously insertable device and configured to move it, An EM target configured to be positioned within the patient, The EM field generator and one or more processors that communicate with the first robot arm are comprising, Based on the kinematic orientation of the first robot arm, the alignment is determined to map the position in the EM coordinate frame associated with the EM field generator to the position in the robot coordinate frame, Based on the alignment described above, the position of the EM target within the robot coordinate frame is determined, Based on the position of the EM target within the robot coordinate frame, the second robot arm is moved to align the axis of the percutaneously insertable device with the EM target. A system configured to use the second robotic arm to induce the insertion of the percutaneously insertable instrument along the axis toward the EM target.

[0293] (76) The alignment is determined based on the position of the EM field generator within the robot coordinate frame, The system according to embodiment 75, wherein the position of the EM field generator having the robot coordinate frame is determined based on the kinematic orientation of the first robot arm. (77) The one or more processors Determining the position of the EM target within the EM coordinate frame, The system according to embodiment 76, configured to determine the position of the EM target in the robot coordinate frame by using the alignment described above to map the position of the EM target in the EM coordinate frame to the position of the EM target in the robot coordinate frame. (78) A robotic medical device configured to be inserted into the patient, The system further comprises a third robotic arm configured to be coupled to and control the robotic medical device, The system according to embodiment 75, wherein the EM target comprises an EM sensor positioned on the robotic medical device. (79) The system according to embodiment 75, wherein the percutaneously insertable instrument comprises one of a needle, an access sheath, and a laparoscopic instrument. (80) A robotic medical system, An instrument guide configured to guide a percutaneously insertable instrument along an insertion axis, wherein the instrument guide is configured to be positioned on an electromagnetic (EM) field generator, and the EM field generator is configured to generate an EM field, and A first robotic arm configured to be coupled to the EM field generator, wherein the first robotic arm is further configured to move the EM field generator and the instrument guide, One or more processors, Determining the EM target positioned within the patient, Determining alignment to map the position in the EM coordinate frame associated with the EM field to the position in the robot coordinate frame associated with the kinematic posture of the first robot arm, Based on the alignment described above, the position of the EM target within the robot coordinate frame is determined, A system comprising one or more processors configured to move the first robot arm based on the position of the EM target within the robot coordinate frame to align the insertion axis of the instrument guide with the EM target.

[0294] (81) Further comprising a second robotic arm configured to be coupled to a percutaneously insertable device extending along an axis, wherein the second robotic arm is further configured to move the percutaneously insertable device, The aforementioned one or more processors Using the second robotic arm, the axis of the percutaneously insertable device is aligned with the insertion axis, The robotic medical system according to Embodiment 80, further configured to use the second robotic arm to insert the percutaneously insertable instrument through the instrument guide along the insertion axis toward the EM target. (82) The system according to embodiment 81, further comprising a third robotic arm configured to be coupled with a robotic medical device, the third robotic arm being further configured to control the robotic medical device while the robotic medical device is inserted into the patient, and the EM target comprising an EM sensor positioned on the robotic medical device. (83) The alignment is determined based on the position of the EM field generator within the robot frame, The system according to embodiment 80, wherein the position of the EM field generator having the robot coordinate frame is determined based on the kinematic orientation of the first robot arm. (84) The one or more processors Determining the position of the EM target within the EM coordinate frame, The system according to embodiment 83, configured to determine the position of the EM target in the robot coordinate frame by mapping the position of the EM target in the EM coordinate frame to the position of the EM target in the robot coordinate frame based on the alignment described above. (85) The system according to embodiment 80, wherein the percutaneously insertable instrument comprises one of a needle, an access sheath, and a laparoscopic instrument.

[0295] (86) A robotic medical system, A first robotic arm configured to be coupled to and move an electromagnetic (EM) field generator, wherein the EM field generator is configured to generate an EM field. A second robotic arm configured to move an instrument guide, wherein the instrument guide is configured to guide a percutaneously insertable instrument along an insertion axis, One or more processors, Determining the EM target positioned within the patient, Based on the kinematic orientation of the first robot arm, the alignment is determined to map the position in the EM coordinate frame associated with the EM field generator to the position in the robot coordinate frame, Based on the alignment described above, the position of the EM target within the robot coordinate frame is determined, A system comprising one or more processors configured to move the second robot arm based on the position of the EM target within the robot coordinate frame to align the insertion axis of the instrument guide with the EM target. (87) Further comprising a third robotic arm configured to be attached to and move the percutaneously insertable device, wherein the percutaneously insertable device extends along an axis, The aforementioned one or more processors Using the third robotic arm, the axis of the percutaneously insertable device is aligned with the insertion axis, The robotic medical system according to embodiment 86, further configured to use the third robotic arm to insert the percutaneously insertable instrument through the instrument guide along the insertion axis toward the EM target. (88) further comprising a fourth robotic arm configured to be coupled to and control a robotic medical device, wherein the robotic medical device is configured to be inserted into the patient, The system according to embodiment 87, wherein the EM target comprises an EM sensor positioned on the robotic medical device. (89) The alignment is determined based on the position of the EM field generator within the robot frame, The system according to embodiment 86, wherein the position of the EM field generator having the robot coordinate frame is determined based on the kinematic orientation of the first robot arm. (90) The one or more processors Determining the position of the EM target within the EM coordinate frame, The system according to embodiment 89, configured to determine the position of the EM target in the robot coordinate frame, based on the configuration to map the position of the EM target in the EM coordinate frame to the position of the EM target in the robot coordinate frame using the alignment described above.

[0296] (91) The system according to embodiment 86, wherein the percutaneously insertable instrument comprises one of a needle, an access sheath, and a laparoscopic instrument. (92) A robotic medical method, Based on the kinematic orientation of the first robot arm, the alignment is determined to map the position in the EM coordinate frame associated with the electromagnetic (EM) field generator coupled to the first robot arm to a position in the robot coordinate frame. Based on the aforementioned alignment, the position of the EM target within the robot coordinate frame is determined, Based on the position of the EM target within the robot coordinate frame, a second robot arm connected to a percutaneously insertable device is moved to align the axis of the percutaneously insertable device with the EM target. A method comprising using the second robotic arm to cause the percutaneously insertable instrument to be inserted along the axis toward the EM target. (93) The method of Embodiment 92, wherein determining the alignment includes determining the alignment based on the position of the EM field generator in the robot frame, and the position of the EM field generator having the robot coordinate frame is determined based on the kinematic orientation of the first robot arm. (94) Determining the position of the EM target within the robot coordinate frame is Determining the position of the EM target within the EM coordinate frame, The method according to Embodiment 93, comprising using the alignment to map the position of the EM target in the EM coordinate frame to the position of the EM target in the robot coordinate frame. (95) Further comprising navigating the robotic medical device within a patient using a third robotic arm configured to be coupled to and control the robotic medical device, The method according to embodiment 92, wherein the EM target comprises an EM sensor positioned on the robotic medical device.

[0297] (96) The method according to embodiment 92, wherein the percutaneously insertable instrument comprises one of a needle, an access sheath, and a laparoscopic instrument. (97) A robotic medical method, Based on the kinematic orientation of the first robot arm, the alignment is determined to map the position in the EM coordinate frame associated with the electromagnetic (EM) field generator coupled to the first robot arm to a position in the robot coordinate frame. Based on the aforementioned alignment, the position of the EM target within the robot coordinate frame is determined, A method comprising moving the first robot arm based on the position of the EM target within the robot coordinate frame to align the insertion axis of an instrument guide positioned on the EM field generator with the EM target, wherein the instrument guide is configured to guide a percutaneously insertable instrument along the insertion axis. (98) A robotic medical method, Based on the kinematic orientation of the first robot arm, the alignment is determined to map the position in the EM coordinate frame associated with the electromagnetic (EM) field generator coupled to the first robot arm to a position in the robot coordinate frame. Based on the aforementioned alignment, the position of the EM target within the robot coordinate frame is determined, A method comprising moving a second robotic arm based on the position of the EM target within the robotic coordinate frame to align the insertion axis of an instrument guide coupled to the second robotic arm with the EM target. (99) A robotic medical system, A first robotic arm coupled to and configured to move an electromagnetic (EM) field generator, wherein the EM field generator is configured to generate an EM field having a working volume in which the position of an EM sensor can be determined relative to an EM coordinate frame, One or more processors, The EM field generator is moved using the first robotic arm, When an EM sensor is detected within the working volume, (i) Determining the position of the EM sensor within the working volume with respect to the EM coordinate frame, (ii) A system comprising one or more processors configured to map the position of the EM sensor to a robot coordinate frame associated with the first robot arm based on the kinematic orientation of the first robot arm. (100) The system according to embodiment 99, wherein one or more processors are configured to move the EM field generator along a search path using the first robotic arm, and the search path is configured to sweep the working volume through a treatment volume larger than the working volume to locate the position of an EM sensor within the treatment volume.

[0298] (101) The one or more processors Based on the determined position of the EM sensor within the robot coordinate frame, a tracking path is determined. The EM field generator is moved along the tracking path using the first robotic arm to track the position of the EM sensor, The system according to embodiment 100, further configured to use the first robotic arm to move the EM field generator along the tracking path. (102) When the EM field generator moves along the tracking path, one or more processors (i) Redetermining the position of the EM sensor within the working volume with respect to the EM coordinate frame, (ii) Remapping the position of the EM sensor to the robot coordinate frame based on the kinematic posture of the first robot arm, (iii) The system according to embodiment 101, configured to determine an updated tracking path. (103) A robotic medical system, A first robotic arm coupled to and configured to move an electromagnetic (EM) field generator, wherein the EM field generator is configured to generate an EM field having a working volume in which the position of an EM sensor can be determined relative to an EM coordinate frame, One or more processors, (i) With the EM field generator in the first position, Determining the position of the first EM sensor within the working volume relative to the EM coordinate frame, Based on the kinematics of the first robot arm when the EM field generator is in the first position, the position of the first EM sensor is mapped to a robot coordinate frame associated with the first robot arm. (ii) Moving the EM field generator to the second position, (iii) With the EM field generator in the second position, Determining the position of the second EM sensor within the working volume relative to the EM coordinate frame, A system comprising one or more processors configured to map the position of the second EM sensor to the robot coordinate frame based on the kinematics of the first robot arm when the EM field generator is in the second position. (104) The system according to embodiment 103, wherein one or more processors are further configured to move the first robotic arm such that the first EM sensor and the second EM sensor are not simultaneously located within the working volume of the EM field generator. (105) The system according to embodiment 103, wherein one or more processors are further configured to move the EM field generator between the first position and the second position in order to track the positions of the first EM sensor and the second EM sensor.

[0299] (106) A method, Using a first robotic arm, move an electromagnetic (EM) field generator positioned on the first robotic arm, When an EM sensor is detected within the working volume, (i) Determining the position of the EM sensor within the working volume of the EM field generated by the EM field generator with respect to the EM coordinate frame, (ii) A method comprising mapping the position of the EM sensor to a robot coordinate frame associated with the first robot arm based on the kinematic orientation of the first robot arm. (107) A method, (i) With the electromagnetic (EM) field generator coupled to the first robot arm in a first position, To determine the position of the first EM sensor within the working volume of the EM field generated by the EM field generator with respect to the EM coordinate frame, Based on the kinematics of the first robot arm when the EM field generator is in the first position, the position of the first EM sensor is mapped to a robot coordinate frame associated with the first robot arm. (ii) Moving the EM field generator to a second position using the first robotic arm, (iii) With the EM field generator in the second position, Determining the position of the second EM sensor within the working volume relative to the EM coordinate frame, A method comprising mapping the position of the second EM sensor to the robot coordinate frame based on the kinematics of the first robot arm when the EM field generator is in the second position. (108) A robotic medical system, A first robotic arm is coupled to an EM field generator configured to generate an electromagnetic (EM) field, EM sensor and, A processor and a processor, the processor While the EM sensor remains in place, a command is sent to the first robot arm to cause the EM field generator to move along the robot's trajectory. The EM sensor trajectory of the EM sensor within the EM field is detected, corresponding to the period during which the EM field generator moved along the robot trajectory. The robot trajectory and the EM sensor trajectory are analyzed to determine the difference between the robot trajectory and the EM sensor trajectory, A system configured to detect EM strain at the location based on a comparison between the difference and a threshold. (109) The system according to embodiment 108, wherein the processor is configured to determine the robot trajectory based on kinematic data corresponding to the first robot arm. (110) The system according to embodiment 108, wherein the processor is configured to determine the threshold based on at least one of the error factors associated with the movement of the first robot arm or the error factors associated with EM sensor noise.

[0300] (111) The system according to embodiment 108, wherein the processor is configured to determine the difference between the robot trajectory and the EM sensor trajectory based on a comparison of the shape of the robot trajectory and the shape of the EM sensor trajectory. (112) The system according to embodiment 108, wherein the processor is configured to determine the difference between the robot trajectory and the EM sensor trajectory based on a comparison of a plurality of points along the robot trajectory and a corresponding plurality of points along the EM sensor trajectory. (113) The system according to embodiment 112, wherein each of the plurality of points is determined with respect to the time associated with the movement of the EM field generator. (114) The system according to embodiment 108, wherein when the processor detects the EM strain at the location, it is further configured to use the first robotic arm to reposition the EM field generator and reduce the EM strain at the location. (115) The processor While the EM sensor remains in the location, an additional command is sent to the first robot arm to cause the EM field generator to move along the second robot trajectory. The detection of the second EM sensor trajectory of the EM sensor within the EM field, corresponding to the period during which the EM field generator moved along the second robot trajectory, The second robot trajectory and the second EM sensor trajectory are analyzed to determine the difference between the second robot trajectory and the second EM sensor trajectory, The system according to embodiment 114, configured to reposition the EM field generator using the first robot arm to reduce the EM strain at the location, by comparing the difference between the second robot trajectory and the second EM sensor trajectory with the difference between the robot trajectory and the EM sensor trajectory.

[0301] (116) The system according to embodiment 115, wherein the processor is configured to position the EM field generator at a point along the second robot trajectory when the difference between the second robot trajectory and the second EM sensor trajectory is smaller than the difference between the robot trajectory and the EM sensor trajectory. (117) A method performed by one or more sets of processors of a robotic system, wherein the method is While the electromagnetic (EM) sensor remains in place, the first robotic arm is instructed to move the EM field generator from the first location to the second location. Determining the robot trajectory associated with the EM field generator moving from the first location to the second location, The detection of the EM sensor trajectory of the EM sensor within the EM field, wherein the EM sensor trajectory is associated with the period during which the EM field generator moved along the robot trajectory. The difference between the robot trajectory and the EM sensor trajectory is determined, A method comprising detecting EM strain at the location based on a comparison of the difference with a strain threshold. (118) The method of Embodiment 117, wherein determining the robot trajectory is further based on using kinematic data corresponding to the first robot arm. (119) The method of Embodiment 117, wherein determining the robot trajectory associated with the EM field generator moving from the first location to the second location is to compensate for error factors associated with the movement of the first robot arm. (120) The method according to embodiment 117, wherein detecting the EM sensor trajectory includes compensating for error factors associated with EM sensor noise.

[0302] (121) The method according to embodiment 117, further comprising determining the difference between the robot trajectory and the EM sensor trajectory by comparing the shape of the robot trajectory with the shape of the EM sensor trajectory. (122) The method according to embodiment 117, wherein determining the difference between the robot trajectory and the EM sensor trajectory further includes comparing a plurality of points along the robot trajectory with a plurality of corresponding points along the EM sensor trajectory. (123) The method according to embodiment 122, wherein each of the plurality of points is determined with respect to the time associated with the movement of the EM field generator. (124) The method according to embodiment 117, further comprising detecting the EM strain at the location and using the first robotic arm to reposition the EM field generator to reduce the EM strain at the location. (125) While the EM sensor remains in the location, the first robot arm is commanded to cause the EM field generator to move along the second robot trajectory, The detection of a second EM sensor trajectory of the EM sensor within the EM field, wherein the second EM sensor trajectory is associated with the period during which the EM field generator moved along the second robot trajectory. The second robot trajectory and the second EM sensor trajectory are analyzed to determine the difference between the second robot trajectory and the second EM sensor trajectory, The method of embodiment 117, further comprising comparing the difference between the second robot trajectory and the second EM sensor trajectory with the difference between the robot trajectory and the EM sensor trajectory.

[0303] (126) The method of embodiment 125, further comprising positioning the EM field generator at a point along the second robot trajectory when the difference between the second robot trajectory and the second EM sensor trajectory is smaller than the difference between the robot trajectory and the EM sensor trajectory. (127) A robotic medical system, A first robotic arm coupled to an EM field generator configured to generate an electromagnetic (EM) field, wherein the first robotic arm is associated with a robot coordinate frame, and the EM field generator is associated with the EM coordinate frame, EM sensor and, The system comprises a processor that communicates with the first robot arm, and the processor Based on the kinematics of the first robot arm when the EM field generator is at the first EM field generator position relative to the location of the EM sensor, a first alignment between the EM coordinate frame and the robot coordinate frame is determined. Based on the first alignment, the first EM sensor position of the EM sensor within the robot coordinate frame is determined, Using the first robotic arm, move the EM field generator from the first EM field generator position to the second EM field generator position relative to the EM sensor, Based on the kinematics of the first robot arm when the EM field generator is at the second EM field generator position relative to the location of the EM sensor, a second alignment between the EM coordinate frame and the robot coordinate frame is determined. Based on the second alignment, the second EM sensor position of the EM sensor within the robot coordinate frame is determined, To determine the difference between the first EM sensor position and the second EM sensor position, A system configured to detect EM strain at the location based on a comparison between the difference and a threshold. (128) The system according to embodiment 127, wherein the processor is further configured to determine positional distortion. (129) The system according to embodiment 127, wherein the processor is further configured to determine orientation distortion. (130) The system according to embodiment 127, wherein the threshold is determined based on at least one of error factors associated with the motion of the first robot arm or error factors associated with EM sensor noise.

[0304] (131) A method, The electromagnetic (EM) field generator is coupled to a first robot arm, wherein the EM field generator is configured to generate an EM field associated with an EM coordinate frame, and the first robot arm is associated with the robot coordinate frame. Based on the kinematics of the first robot arm when the EM field generator is at the first EM field generator position relative to the location of the EM sensor, a first alignment between the EM coordinate frame and the robot coordinate frame is determined. Based on the first alignment, the first EM sensor position of the EM sensor within the robot coordinate frame is determined, Using the first robotic arm, the EM field generator is moved from the first EM field generator position to the second EM field generator position relative to the EM sensor, wherein the EM sensor remains stationary during the movement. Based on the kinematics of the first robot arm and the location of the EM sensor when the EM field generator is in the second EM field generator position, a second alignment between the EM coordinate frame and the robot coordinate frame is determined. Based on the second alignment, the second EM sensor position of the EM sensor within the robot coordinate frame is determined, To determine the difference between the first EM sensor position and the second EM sensor position, A method comprising detecting EM strain at the location based on the difference exceeding a threshold. (132) A robotic medical system, A first robotic arm coupled to an ultrasonic probe, wherein the first robotic arm is configured to move to adjust the position of the ultrasonic probe, and the first robotic arm is associated with a robot coordinate frame, The system comprises a first robotic arm and a processor that communicates with the ultrasonic probe, and the processor Using the first robotic arm, move the ultrasonic probe to the first ultrasonic probe position, The ultrasonic probe is positioned at the first ultrasonic probe position to capture a first ultrasonic image, Using the first robotic arm, move the ultrasonic probe to the position of the second ultrasonic probe, The ultrasound probe is positioned at the second ultrasound probe position to capture a second ultrasound image, Using the first robotic arm, move the ultrasonic probe to the third ultrasonic probe position, The ultrasonic probe is positioned at the third ultrasonic probe position to capture a third ultrasonic image, With respect to the robot coordinate frame, the imaging surface of the ultrasonic probe is set as follows: The first ultrasonic image and the first kinematics of the first robot arm when the ultrasonic probe is in the first ultrasonic probe position, The second ultrasound image and the second kinematics of the first robot arm when the ultrasound probe is in the second ultrasound probe position, A system configured to perform calibration based on the third ultrasonic image and the third kinematics of the first robot arm when the ultrasonic probe is in the third ultrasonic probe position. (133) An EM field generator configured to generate an electromagnetic (EM) field, wherein the EM field generator is associated with an EM coordinate frame, A second robotic arm configured to move to adjust the position of the EM field generator, the second robotic arm further comprises a second robotic arm associated with the robot coordinate frame, The aforementioned processor, The system according to embodiment 132, further configured to determine the alignment between the EM field coordinate frame and the robot coordinate frame based on the kinematics of the second robot arm. (134) The system according to embodiment 133, wherein the first robot arm and the second robot arm are mounted on a cart or patient platform that defines the relationship between the first robot arm and the second robot arm in the robot coordinate frame. (135) The system according to embodiment 133, further comprising an EM sensor detectable within the EM field, wherein the processor is further configured to determine the position of the EM sensor in the robot coordinate frame based on the alignment.

[0305] (136) The processor Displaying the imaging surface of the ultrasonic probe, The system according to embodiment 135, further configured to overlay an indication of the determined EM sensor position onto the displayed imaging surface of the ultrasonic probe. (137) A method, The coupling involves connecting an ultrasonic probe to a first robot arm, wherein the first robot arm is configured to move to adjust the position of the ultrasonic probe, and the first robot arm is associated with a robot coordinate frame. Using the first robotic arm, move the ultrasonic probe to the first ultrasonic probe position, The ultrasonic probe is positioned at the first ultrasonic probe position to capture a first ultrasonic image, Using the first robotic arm, move the ultrasonic probe to the position of the second ultrasonic probe, The ultrasound probe is positioned at the second ultrasound probe position to capture a second ultrasound image, Using the first robotic arm, move the ultrasonic probe to the third ultrasonic probe position, The ultrasonic probe is positioned at the third ultrasonic probe position to capture a third ultrasonic image, With respect to the robot coordinate frame, the imaging surface of the ultrasonic probe is set as follows: The first ultrasonic image and the first kinematics of the first robot arm when the ultrasonic probe is in the first ultrasonic probe position, The second ultrasound image and the second kinematics of the first robot arm when the ultrasound probe is in the second ultrasound probe position, A method comprising calibrating based on the third ultrasonic image and the third kinematics of the first robot arm when the ultrasonic probe is in the third ultrasonic probe position. (138) A coupling of an electromagnetic (EM) field generator to a second robot arm, wherein the EM field generator is configured to generate an EM field, the EM field generator is associated with an EM coordinate frame, and the second robot arm is configured to move to adjust the position of the EM field generator, and the second robot arm is associated with the robot coordinate frame. The method according to embodiment 137, further comprising determining the alignment between the EM field coordinate frame and the robot coordinate frame based on the kinematics of the second robot arm. (139) The method according to embodiment 138, wherein the first robot arm and the second robot arm are mounted on a cart or patient platform that defines the relationship between the first robot arm and the second robot arm in the robot coordinate frame. (140) Positioning the EM sensor within the EM field, The method according to embodiment 138, further comprising determining the position of the EM sensor within the robot coordinate frame based on the alignment described above.

[0306] (141) Displaying the imaging surface of the ultrasonic probe, The method according to embodiment 140, further comprising overlaying the indicated EM sensor position on the displayed imaging surface of the ultrasonic probe. (142) A robotic medical system, A first robotic arm coupled to a first imaging device, wherein the first robotic arm is configured to move to adjust the position of the first imaging device, and the first robotic arm is associated with a robot coordinate frame, The system comprises a first robotic arm and a processor that communicates with the first imaging device, and the processor Using the first robotic arm, move the first imaging device to a first position, The first imaging device captures a first image while positioned at the first position, Using the first robotic arm, move the first imaging device to a second position, The first imaging device is positioned at the second position to capture a second image, Using the first robotic arm, move the first imaging device to a third position, The first imaging device is positioned at the third location and captures a third image. The first imaging device is used with respect to the robot coordinate frame. The first image, the first kinematics of the first robot arm when the first imaging device is in the first position, The second image and the second kinematics of the first robot arm when the first imaging device is in the second position, A system configured to perform calibration based on the third image and the third kinematics of the first robot arm when the first imaging device is in the third position. (143) further comprising a second robotic arm coupled to a second imaging device, wherein the second robotic arm is configured to move to adjust the position of the second imaging device, and the second robotic arm is associated with the robotic coordinate frame, The processor communicates with the second robot arm and the second imaging device, and the processor Using the second robotic arm, move the second imaging device to the first position, The second imaging device captures a first image while positioned at the first position, Using the second robotic arm, move the second imaging device to a second position, The second imaging device captures a second image while positioned at the second location, Using the second robotic arm, move the second imaging device to a third position, The second imaging device is positioned at the third location and captures a third image. The second imaging device is used with respect to the robot coordinate frame. The first image and the first kinematics of the second robot arm when the second imaging device is in the first position, The second image and the second kinematics of the second robot arm when the second imaging device is in the second position, The system according to embodiment 142, configured to perform calibration based on the third image and the third kinematics of the second robot arm when the second imaging device is in the third position. (144) The system according to embodiment 142, wherein the first imaging device comprises one of an ultrasonic probe, a camera, or a depth sensor. (145) The system according to embodiment 144, wherein the second imaging device comprises one of an ultrasonic probe, a camera, or a depth sensor.

[0307] (146) An EM field generator configured to generate an electromagnetic (EM) field, wherein the EM field generator is associated with an EM coordinate frame, A third robotic arm configured to move in order to adjust the...

Claims

1. It is a robotic medical system, A first robotic arm coupled to an electromagnetic field generator configured to generate an electromagnetic field (EM field), wherein the first robotic arm is configured to move the electromagnetic field generator, Medical devices equipped with EM sensors, One or more processors, It is determined that the EM sensor is located at the boundary of a predetermined region within the electromagnetic field. One or more processors adjust the position of the electromagnetic field generator by commanding the movement of the first robot arm in response to the determination that the EM sensor is located at the boundary of the predetermined area, A system that includes these features.

2. The system according to claim 1, wherein adjusting the position of the electromagnetic field generator causes the EM sensor to be positioned at a predetermined location within the electromagnetic field.

3. The system according to claim 2, wherein the predetermined position within the electromagnetic field includes the center of the working volume of the electromagnetic field.

4. The system according to claim 1, wherein adjusting the position of the electromagnetic field generator based on the determination of the position of the EM sensor includes adjusting the position of the electromagnetic field generator so that the EM sensor is located within a predetermined region of the electromagnetic field.

5. The aforementioned one or more processors further Determine the direction of the EM sensor in the electromagnetic field, The direction and position of the electromagnetic field generator are adjusted by commanding the movement of the first robot arm based on the determined direction of the EM sensor. The system according to claim 1, configured as follows.

6. The system according to claim 5, wherein one or more processors are configured to adjust at least one of the orientation and position of the electromagnetic field generator based on the determined orientation of the EM sensor in order to improve the accuracy of determining the position of the EM sensor in the electromagnetic field.

7. Furthermore, including a second robotic arm, The medical device is attached to the second robot arm, The system according to claim 1, wherein one or more processors are configured to command the movement of the medical device together with the second robotic arm.

8. The system according to claim 7, wherein the first and second robotic arms are connected to one of a cart and a patient platform.

9. The system according to claim 8, further comprising a third robotic arm connected to one of the cart and the patient platform.

10. The aforementioned one or more processors The position of the EM sensor in the electromagnetic field is determined with respect to the EM coordinate system associated with the electromagnetic field generator. The registration between the EM coordinate system and the robot coordinate system associated with the first robot arm is determined based on determining the position of the electromagnetic field generator within the robot coordinate system. Based on the registration, the position of the EM sensor in the robot coordinate system is determined. The system according to claim 1, configured as described above.

11. The system according to claim 10, wherein one or more processors are configured to determine the position of the electromagnetic field generator in the robot coordinate system based on the kinematics of the first robot arm.

Images