Robotic controllable field generator

Abstract

Certain aspects relate to systems having robotically controllable field generators and applications of robotically controllable field generators. One application of a robotically controllable field generator uses a robotic arm to move a field generator coupled thereto to facilitate field generator setup and instrument tracking. Another application of such a robotically controllable field generator uses a robotic arm to move a field generator coupled thereto to facilitate alignment of an insertable instrument with a target detected within a volume of the field generator. Another application of such a robotically controllable field generator uses a robotic arm to move a field generator coupled thereto to detect distortions occurring within a medical system. Another application of such a robotically controllable field generator uses a robotic arm to facilitate multi-modal sensor fusion.

Description

Technical Field

[0001] The systems and methods disclosed herein relate to field generators for robotic medical systems, and more specifically, to robotic, positionable and / or controllable field generators for detecting distortions in medical systems, as well as related apparatus, systems and methods. Background Technology

[0002] Medical procedures such as laparoscopy or endoscopy may involve accessing and visualizing a patient's internal areas. In laparoscopic procedures, medical devices may be inserted into the internal area through a laparoscopic inlet. In endoscopic procedures, thin, flexible tubular medical devices may be inserted into the internal area through a natural patient orifice. Medical devices may include end effectors configured to perform functions during surgery.

[0003] In some procedures, robot-enabled medical systems can be used to control the insertion and / or manipulation of medical devices and end effectors. Robot-enabled medical systems may include robotic arms or other device positioning devices having manipulator components for controlling the positioning of the device during the procedure.

[0004] Robot-enabled medical systems can be configured to determine the position of a medical device based on the output of one or more position sensors that can be positioned on the device. Summary of the Invention

[0005] Robotic medical systems may include electromagnetic (EM) field generators configured to be coupled to (or otherwise integrated into) a robotic arm of the system. Such EM field generators can be considered robotically controllable or positionable because they can be controlled or repositioned using the robotic arm. The EM field generator produces a magnetic field within which the position of one or more EM sensors can be determined. Because the EM field generator is coupled to the robotic arm, the kinematics of the robotic arm can be used to establish registration between the EM coordinate system of the EM field generator and the robot coordinate system or global coordinate system of the system.

[0006] This arrangement reduces or eliminates the need for more complex registration steps that may require user input to establish the relationship between the EM coordinate system and the robot or global coordinate system. It also improves the accuracy of determining the position of EM sensors, for example by improving the setup of field generators corresponding to the robot arm, tracking medical devices, detecting sensor distortions that may affect the system, and facilitating the fusion of sensors with additional modes.

[0007] These and other features and advantages of the robotic, controllable, or positionable EM field generator will be described in more detail below. The systems, methods, and apparatuses of this disclosure each have several innovative aspects, none of which independently constitutes the desired properties disclosed herein. Attached Figure Description

[0008] The disclosed aspects will be described below in conjunction with the accompanying drawings, which are provided to illustrate and not limit the disclosed aspects, wherein similar reference numerals denote similar elements.

[0009] Figure 1 An embodiment of a cart-based robotic system arranged for diagnostic and / or therapeutic bronchoscopy is shown, according to an exemplary embodiment.

[0010] Figure 2 The following is a description of an exemplary implementation. Figure 1 Another aspect of robotic systems.

[0011] Figure 3 An example of a device arranged for ureteroscopy according to an exemplary embodiment is shown. Figure 1 The implementation plan for the robot system.

[0012] Figure 4 An example embodiment of a device arranged for vascular procedures is shown. Figure 1 The implementation plan for the robot system.

[0013] Figure 5 An embodiment of a stage-based robotic system deployed for bronchoscopy procedures, according to an exemplary embodiment, is shown.

[0014] Figure 6 Provided according to exemplary implementation Figure 5 An alternative view of the robot system.

[0015] Figure 7 An exemplary system configured to retract one or more robotic arms is shown according to an exemplary embodiment.

[0016] Figure 8 An embodiment of a table-based robotic system configured for ureteroscopy procedures is shown according to an exemplary embodiment.

[0017] Figure 9 An embodiment of a table-based robotic system configured for laparoscopic procedures, according to an exemplary implementation, is shown.

[0018] Figure 10 An example embodiment is shown with pitch or tilt adjustment. Figures 5 to 9Implementation plan for platform-based robot system.

[0019] Figure 11 Provided according to exemplary implementation Figures 5 to 10 A detailed diagram of the interface between the platform and the column in a platform-based robotic system.

[0020] Figure 12 An alternative embodiment of a stage-based robotic system according to an exemplary embodiment is shown.

[0021] Figure 13 An exemplary embodiment is shown. Figure 12 An end view of a platform-based robotic system.

[0022] Figure 14 An end view of a stage-based robotic system with a robotic arm attached thereto, according to an exemplary embodiment, is shown.

[0023] Figure 15 An exemplary device driver according to an exemplary implementation is shown.

[0024] Figure 16 An exemplary medical device with paired instrument drivers is shown according to an exemplary embodiment.

[0025] Figure 17 An alternative design of the instrument driver and instrument according to an exemplary embodiment is shown, wherein the axis of the drive unit is parallel to the axis of the slender axis of the instrument.

[0026] Figure 18 An apparatus with an apparatus-based insertion architecture according to an exemplary embodiment is shown.

[0027] Figure 19 An exemplary controller according to an exemplary implementation is shown.

[0028] Figure 20 A block diagram according to an exemplary embodiment is depicted, illustrating the estimation Figures 1 to 10 The location of one or more components of a robotic system (such as...) Figures 16 to 18 A positioning system for the location of instruments.

[0029] Figure 21 An exemplary EM field generator is shown that generates a magnetic field having a working volume. According to an exemplary embodiment, an exemplary EM sensor positioned within the working volume of the EM field generator is also shown.

[0030] Figure 22A This illustrates an embodiment including... Figure 21 A perspective view of the implementation scheme of a robotic medical system with an EM field generator.

[0031] Figure 22B and Figure 22C Exemplary registration steps according to exemplary embodiments are shown, which can be used to register with Figure 22A The robot coordinate system associated with the robotic medical system is registered with the EM coordinate system associated with the EM field generator.

[0032] Figure 23 This is a perspective view illustrating an embodiment of a robotic medical system according to an exemplary embodiment, which includes an embodiment of a robotic controllable field generator coupled to a robotic arm of the system.

[0033] Figure 24 This is a perspective view of an embodiment of a robotic controllable field generator configured to be coupled to a robotic arm, according to an exemplary embodiment.

[0034] Figure 25 An exemplary embodiment is shown. Figure 24 The robotic controllable field generator can be configured as a mechanical drive mechanism attached to the robotic arm.

[0035] Figure 26 This is a flowchart illustrating an embodiment of a method for performing robotic medical procedures using a robotic controllable field generator configured to be coupled to a robotic arm, according to an exemplary embodiment.

[0036] Figure 27A and Figure 27B An embodiment of a robotic medical system including a robotic controllable field generator is shown according to an exemplary embodiment, the robotic controllable field generator being configured for automated instrument tracking and anatomical feature mapping.

[0037] Figure 28A and Figure 28B An embodiment of a robotic medical system including a robotic controllable field generator is shown according to an exemplary embodiment, the robotic controllable field generator being configured to facilitate the setup and placement of the field generator and expand its working volume.

[0038] Figure 29 This is a block diagram illustrating an embodiment of a robotic medical system according to an exemplary embodiment, the robotic medical system including a robotic arm coupled to the system and a robotic controllable field generator.

[0039] Figure 30A and Figure 30B This is a perspective view illustrating an example of using a robotic arm to move an EM field generator according to an exemplary embodiment, such that an EM sensor is positioned at a predetermined location within the EM field.

[0040] Figure 31A and Figure 31B This is a perspective view illustrating an example of using a robotic arm to move an EM field generator according to an exemplary embodiment, such that an EM sensor is positioned within a predetermined area of ​​the EM field.

[0041] Figure 32A and Figure 32B This is a perspective view illustrating an example of moving an EM field generator along a path that tracks the movement of an EM sensor, according to an exemplary embodiment.

[0042] Figures 33A to 33D This is a perspective view illustrating an example of using a robotic arm to move an EM field generator according to an exemplary embodiment, such that an EM sensor moving along a path remains positioned within a predetermined area of ​​the EM field.

[0043] Figure 34 This is a flowchart providing an exemplary method for moving an EM field generator coupled to a robot arm based on a determined position of an EM sensor within an EM field, according to an exemplary embodiment.

[0044] Figure 35A and Figure 35B This is a perspective view illustrating an example of using a robotic arm to move an EM field generator to a field generator location based on the determined positions of multiple EM sensors within an EM field, according to an exemplary embodiment.

[0045] Figures 36A to 36C This is a perspective view illustrating an example of using a robotic arm to readjust the position of an EM field generator based on the determined positions of multiple EM sensors within an EM field, according to an exemplary embodiment, wherein at least one of the multiple EM sensors is moving.

[0046] Figure 37 This is a flowchart illustrating an exemplary method, according to an exemplary embodiment, for determining the field generator position of an EM field generator mounted on a robot arm based on the determined positions of multiple EM sensors within an EM field.

[0047] Figure 38A , Figure 38B and Figure 38C An embodiment of a robotic medical system including a robotic controllable field generator is shown according to an exemplary embodiment, the robotic controllable field generator being able to expand the working volume of the field generator by utilizing the movement of a robotic arm.

[0048] Figures 39A to 39DAn embodiment of a robotic medical system including a robotic controllable field generator is shown according to an exemplary embodiment, which can utilize the movement of a robotic arm to detect the position of an EM sensor and expand the working volume of the field generator.

[0049] Figures 40A to 40D An embodiment of a robotic medical system including a robotic controllable field generator is shown according to an exemplary embodiment, which can utilize the movement of a robotic arm to track the position of an EM sensor within the extended working volume of the field generator.

[0050] Figure 41A This is a flowchart illustrating an exemplary method for expanding the working volume of a robotic controllable field generator according to an exemplary embodiment.

[0051] Figure 41B This is a flowchart illustrating another exemplary method for expanding the working volume of a robotic controllable field generator according to an exemplary embodiment.

[0052] Figures 42A to 42C An embodiment of a robotic medical system with a robotic controllable field generator according to an exemplary embodiment is shown, the robotic controllable field generator being configured to facilitate the alignment of percutaneously insertable instruments with an EM target.

[0053] Figure 43A and Figure 43B An embodiment of a robotic medical system with a robotic controllable field generator according to an exemplary embodiment is shown, the robotic controllable field generator being configured to facilitate the alignment of an instrument guide mounted on the field generator with an EM target.

[0054] Figure 44A and Figure 44B An embodiment of a robotic medical system with a robotic controllable field generator according to an exemplary embodiment is shown, the robotic controllable field generator being configured to facilitate the alignment of the instrument guide with the EM target.

[0055] Figure 45A This is a flowchart illustrating a method for aligning a percutaneously insertable instrument with an EM target using a robotic controllable field generator, according to an exemplary embodiment.

[0056] Figure 45B This is a flowchart illustrating a method for aligning an instrument guide for a percutaneously insertable instrument with an EM target using a robotic controllable field generator, according to an exemplary embodiment.

[0057] Figure 46AAn embodiment of a robotic medical system is shown, which moves an EM field generator relative to a fixed EM position sensor to detect EM distortion, according to an exemplary embodiment.

[0058] Figure 46B An exemplary embodiment is shown. Figure 46A A comparison of the robot trajectory and the EM sensor trajectory during the movement of the EM field generator.

[0059] Figure 46C An embodiment of a robotic medical system is shown, which moves an EM field generator relative to a fixed EM position sensor to detect EM distortions that can cause changes in position and orientation, according to an exemplary embodiment.

[0060] Figure 47 This is a flowchart illustrating an exemplary method for EM distortion detection according to an exemplary embodiment.

[0061] Figure 48 This is a flowchart illustrating another exemplary method for EM distortion detection according to an exemplary embodiment.

[0062] Figure 49A The illustration shows a robotic medical system with an ultrasound probe attached thereto, according to an exemplary embodiment, during a procedure for calibrating the imaging position of the ultrasound probe.

[0063] Figure 49B A robotic medical system including an EM field generator and an ultrasound probe is illustrated. According to an exemplary embodiment, the imaging plane of the ultrasound probe is calibrated relative to the robot coordinate system and the EM coordinate system, such that the positions of the robotic tool and the EM sensor can be superimposed on the imaging plane.

[0064] Figure 50 This is a flowchart illustrating an exemplary method for calibrating the imaging plane of an ultrasound probe for use with a robotic medical system, according to an exemplary embodiment.

[0065] Figure 51A and Figure 51B Exemplary heatmaps and dot maps are shown, generated using a depth sensor attached to a robotic arm of a robotic medical system according to an exemplary embodiment.

[0066] Figure 51C An embodiment of an exemplary depth sensor according to an exemplary implementation is shown. Detailed Implementation

[0067] 1. Overview .

[0068] The aspects of this disclosure can be integrated into robot-enabled medical systems capable of performing a variety of medical procedures, including minimally invasive procedures such as laparoscopy and non-invasive procedures such as endoscopy. In endoscopic procedures, the system may be able to perform bronchoscopy, ureteroscopy, gastroscopy, etc.

[0069] In addition to executing a wide range of procedures, the system can provide additional benefits such as enhanced imaging and guidance to assist physicians. Furthermore, the system allows physicians to execute procedures from an ergonomic orientation, eliminating the need for cumbersome arm movements and positioning. Additionally, the system provides physicians with improved ease of use, enabling one or more instruments within the system to be controlled by a single user.

[0070] For illustrative purposes, various embodiments will be described below in conjunction with the accompanying drawings. It should be understood that many other specific embodiments of the disclosed concepts are possible, and various advantages can be achieved using the disclosed specific embodiments. Titles are included herein for reference and to aid in locating the various sections. These titled sections are not intended to limit the scope of the concepts described therein. Such concepts may be applicable throughout the specification.

[0071] A. Robotic System – Trolley .

[0072] Robot-enabled medical systems can be configured in a variety of ways, depending on specific procedures. Figure 1 An embodiment of a cart-based, robot-enabled system 10 arranged for diagnostic and / or therapeutic bronchoscopy is illustrated. During bronchoscopy, system 10 may include a cart 11 having one or more robotic arms 12 to deliver medical instruments, such as a manipulable endoscope 13 (which may be a procedure-specific bronchoscope for bronchoscopy), to a natural orifice entry point (i.e., the patient's mouth positioned on the table in this example) to deliver diagnostic and / or therapeutic tools. As shown, cart 11 may be positioned near the patient's upper torso to provide access to the entry point. Similarly, robotic arms 12 may be actuated to position the bronchoscope relative to the entry point. When performing GI procedures using a gastroscopy (a dedicated endoscope for gastrointestinal (GI) procedures), the same approach may be used. Figure 1 The layout within. Figure 2 An exemplary implementation of the cart is described in more detail.

[0073] Continue to refer to Figure 1Once the trolley 11 is correctly positioned, the robotic arm 12 can robotically, manually, or in combination thereof insert the maneuverable endoscope 13 into the patient. As shown, the maneuverable endoscope 13 may include at least two telescopic portions, such as an inner guide portion and an outer sheath portion, each portion being coupled to a separate instrument actuator from a set of instrument actuators 28, each instrument actuator being coupled to the distal end of a separate robotic arm. This linear arrangement of the instrument actuators 28, which facilitates coaxial alignment of the guide portion and the sheath portion, creates a “virtual track” 29, which can be repositioned in space by maneuvering one or more robotic arms 12 to different angles and / or positions. The virtual track described herein is depicted using dashed lines in the accompanying drawings, and therefore the dashed lines do not depict any physical structure of the system. Translation of the instrument actuators 28 along the virtual track 29 causes the inner guide portion to extend or retract relative to the outer sheath portion, or to advance or retract the endoscope 13 from the patient. The angle of the virtual track 29 can be adjusted, translated, and pivoted based on clinical application or physician preference. For example, in bronchoscopy, the angle and position of the virtual track 29 shown in the figure represent a trade-off between providing the physician with access to the endoscope 13 and minimizing friction caused by the endoscope 13 bending into the patient's mouth.

[0074] After insertion, endoscope 13 can be guided downwards through the patient's trachea and lungs using precise commands from the robotic system until the target destination or surgical site is reached. To enhance navigation through the patient's lung network and / or reach the desired target, endoscope 13 can be manipulated to telescopically extend the inner guide portion from the outer sheath portion to achieve enhanced joint movement and a larger radius of flexion. The use of separate instrument actuators 28 also allows the guide portion and sheath portion to be driven independently of each other.

[0075] For example, endoscope 13 can be guided to deliver a biopsy needle to a target, such as a lesion or nodule in a patient's lung. The needle can be deployed downwards along the working channel, which extends the length of the endoscope to obtain a tissue sample to be analyzed by a pathologist. Depending on the pathological findings, additional tools can be deployed downwards along the working channel of the endoscope for additional biopsies. After the nodule is identified as malignant, endoscope 13 can be used to deliver endoscopic tools to remove the potential cancerous tissue. In some cases, diagnostic and therapeutic procedures can be delivered in a separate procedure. In these cases, endoscope 13 can also be used to deliver a reference point to "mark" the location of the target nodule. In other cases, diagnostic and therapeutic procedures can be delivered during the same procedure.

[0076] System 10 may also include a movable tower 30, which can be connected to the trolley 11 via support cables to provide control, electronic, fluid, optical, sensor, and / or electrical support to the trolley 11. Placing such functionality in the tower 30 allows for a smaller form factor trolley 11 that can be more easily adjusted and / or repositioned by the operating physician and his / her staff. Additionally, the division between the trolley / table and the support tower 30 reduces operating room clutter and facilitates improved clinical workflow. While the trolley 11 can be positioned close to the patient, the tower 30 can be retracted in a remote location to avoid obstructing the path during procedures.

[0077] To support the aforementioned robotic system, tower 30 may include components of a computer-based control system that stores computer program instructions in a non-transitory computer-readable storage medium such as a permanent magnet memory drive, a solid-state drive, etc. Whether execution occurs within tower 30 or cart 11, the execution of these instructions can control the entire system or its subsystems. For example, when executed by the processor of the computer system, the instructions can cause components of the robotic system to actuate relevant brackets and arm mounts, actuate the robotic arm, and control medical devices. For instance, in response to receiving a control signal, a motor in the joint of the robotic arm can position the arm into a specific posture.

[0078] Tower 30 may also include pumps, flow meters, valve controllers, and / or fluid passages to provide controlled flushing and suction capabilities to a system that can be deployed via endoscope 13. These components may also be controlled using a computer system of tower 30. In some embodiments, flushing and suction capabilities may be delivered directly to endoscope 13 via a separate cable.

[0079] Tower 30 may include voltage and surge protectors designed to provide filtered and protected power to trolley 11, thereby avoiding the need to place power transformers and other auxiliary power components in trolley 11, resulting in a smaller and more mobile trolley 11.

[0080] Tower 30 may also include support devices for sensors deployed throughout the robotic system 10. For example, tower 30 may include optoelectronic devices for detecting, receiving, and processing data received from optical sensors or cameras throughout the robotic system 10. In conjunction with a control system, such optoelectronic devices can be used to generate real-time images for display in any number of consoles deployed throughout the system (including displays within tower 30). Similarly, tower 30 may also include electronic subsystems for receiving and processing signals received from deployed electromagnetic (EM) sensors. Tower 30 may also be used to house and position EM field generators for detection by EM sensors in or on a medical device.

[0081] In addition to other consoles available in the rest of the system (e.g., a console mounted on top of a cart), tower 30 may also include console 31. Console 31 may include a user interface and display, such as a touchscreen, for physician operators. Consoles in system 10 are generally designed to provide both robot control and preoperative and real-time information for procedures, such as navigation and positioning information for endoscope 13. When console 31 is not the only console available to the physician, it may be used by a second operator (such as a nurse) to monitor the patient’s health or vital signs and operation of system 10, as well as to provide procedure-specific data, such as navigation and positioning information. In other embodiments, console 31 is housed in a separate body from tower 30.

[0082] Tower 30 can be connected to cart 11 and endoscope 13 via one or more cables or connectors (not shown). In some embodiments, support functionality from tower 30 can be provided to cart 11 via a single cable, thereby simplifying the operating room and eliminating clutter. In other embodiments, specific functions can be coupled in separate wiring and connections. For example, while power can be provided to cart 11 via a single cable, support for controls, optics, fluid, and / or navigation can also be provided via separate cables.

[0083] Figure 2 Provided from Figure 1 The illustration shows a detailed depiction of an embodiment of a cart 11 in a cart-based robot-enabled system. The cart 11 typically includes an elongated support structure 14 (often referred to as a "post"), a cart base 15, and a console 16 at the top of the post 14. The post 14 may include one or more brackets, such as those for supporting one or more robotic arms 12. Figure 2 The bracket 17 (or alternatively, "arm support") is deployed in three configurations. The bracket 17 may include a separately configurable arm mount that rotates along a vertical axis to adjust the base of the robotic arm 12 for better positioning relative to the patient. The bracket 17 also includes a bracket interface 19 that allows the bracket 17 to translate vertically along the post 14.

[0084] The carriage interface 19 is connected to the column 14 via a slot, such as slot 20, which is positioned on the opposite side of the column 14 to guide the vertical translation of the carriage 17. Slot 20 includes a vertical translation interface to position and hold the carriage 17 relative to the trolley base 15 at various vertical heights. The vertical translation of the carriage 17 allows the trolley 11 to adjust the reach of the robotic arm 12 to accommodate various table heights, patient sizes, and physician preferences. Similarly, individually configurable arm mounts on the carriage 17 allow the robotic arm base 21 of the robotic arm 12 to be angled in various configurations.

[0085] In some embodiments, slot 20 may be supplemented with a slot cover flush and parallel to the slot surface to prevent dust and fluid from entering the internal cavity of column 14 and the vertical translation interface during the vertical translation of bracket 17. The slot cover can be deployed via a pair of spring reels positioned near the vertical top and bottom of slot 20. The cover is coiled within the reels until it is deployed to extend and retract from its coiled state during the vertical up-and-down translation of bracket 17. The spring loading of the reels provides a force to retract the cover into the reels as bracket 17 translates toward the reels, while maintaining a tight seal as bracket 17 translates away from the reels. The cover can be attached to bracket 17 using, for example, a bracket in bracket interface 19, to ensure proper extension and retraction of the cover during the translation of bracket 17.

[0086] The column 14 may internally include mechanisms such as gears and motors, which are designed to mechanically translate the bracket 17 using vertically aligned lead screws in response to control signals generated in response to user input (e.g., input from the console 16).

[0087] A robotic arm 12 typically includes a robotic arm base 21 and an end effector 22 separated by a series of links 23 connected by a series of joints 24, each joint including an independent actuator, and each actuator including an independently controllable motor. Each independently controllable joint represents an independent degree of freedom available to the robotic arm 12. Each robotic arm in the robotic arm 12 may have seven joints, and thus provide seven degrees of freedom. Multiple joints result in multiple degrees of freedom, thus allowing for “redundant” degrees of freedom. Having redundant degrees of freedom allows the robotic arm 12 to position its corresponding end effector 22 in a specific orientation, orientation, and trajectory in space using different joint positions and joint angles. This allows the system to locate and guide medical devices from desired points in space, while allowing physicians to move the arm joints to a clinically advantageous orientation away from the patient to achieve greater proximity while avoiding arm collisions.

[0088] The cart base 15 balances the weight of the counterweight 14, bracket 17, and robotic arm 12 on the floor. Therefore, the cart base 15 houses heavier components such as electronics, motors, power supplies, and components that enable the cart 11 to move and / or be secured. For example, the cart base 15 includes rollable wheel-shaped casters 25 that allow the cart 11 to easily move around the room before the procedure. Once in the correct orientation, the casters 25 can be secured using wheel locks to hold the cart 11 in the correct orientation during the procedure.

[0089] The console 16, positioned at the vertical end of column 14, allows both a user interface for receiving user input and a display screen (or dual-purpose device, such as, for example, touchscreen 26) to provide both preoperative and intraoperative data to the physician user. Potential preoperative data on touchscreen 26 may include preoperative planning, navigation, and mapping data derived from preoperative computed tomography (CT) scans and / or records from preoperative patient interviews. Intraoperative data on the display screen may include optical information from tools and sensors, coordinate information from sensors, and important patient statistics such as respiration, heart rate, and / or pulse. The console 16 can be positioned and tilted to allow the physician to access it from the side of column 14 opposite to bracket 17. From this orientation, the physician can operate the console 16 from behind cart 11 while observing the console 16, robotic arm 12, and patient. As shown, the console 16 also includes a handle 27 to aid in manipulating and stabilizing cart 11.

[0090] Figure 3 An embodiment of a robot-enabled system 10 arranged for ureteroscopy is shown. In a ureteroscopy procedure, a trolley 11 is positioned to deliver a ureteroscope 32 (a procedure-specific endoscope designed to traverse the patient's urethra and ureter) to the patient's lower abdominal region. During ureteroscopy, it is desirable to align the ureteroscope 32 directly with the patient's urethra to reduce friction and force on sensitive anatomical structures in that region. As shown, the trolley 11 can be aligned at the foot of the table to allow the robotic arm 12 to position the ureteroscope 32 for direct linear access into the patient's urethra. The robotic arm 12 can insert the ureteroscope 32 directly into the patient's lower abdomen through the urethra from the foot of the table along a virtual track 33.

[0091] After insertion into the urethra, using control techniques similar to those used in bronchoscopy, the ureteroscope 32 can be navigated to the bladder, ureter, and / or kidney for diagnostic and / or therapeutic applications. For example, the ureteroscope 32 can be guided into the ureter and kidney to break up accumulated kidney stones using a laser or ultrasonic lithotripsy device deployed downwards along the working channel of the ureteroscope 32. After lithotripsy is complete, the resulting stone fragments can be removed using a basket deployed downwards along the ureteroscope 32.

[0092] Figure 4An embodiment of a robot-enabled system 10 for vascular procedures is illustrated similarly. In vascular procedures, system 10 can be configured such that a cart 11 delivers a medical device 34 (such as a manipulable catheter) to an entry point in the femoral artery in the patient's leg. The femoral artery presents both a relatively large diameter for navigation and a relatively less circuitous and tortuous path to the patient's heart, which simplifies navigation. As in ureteroscopy procedures, the cart 11 can be positioned toward the patient's leg and lower abdomen to allow the robotic arm 12 to provide a virtual track 35 for direct linear access to the femoral artery entry point in the patient's thigh / hip region. After insertion into the artery, the medical device 34 can be guided and inserted via a translational device actuator 28. Alternatively, the cart can be positioned around the patient's upper abdomen to reach alternative vascular entry points, such as the carotid and brachial arteries near the shoulder and wrist.

[0093] B. Robot System – Unit .

[0094] Implementation plans for robot-enabled medical systems can also incorporate patient-integrated tables. Integrating tables reduces the amount of capital equipment in the operating room by removing trolleys, allowing for greater accessibility to the patient. Figure 5 An embodiment of such a robot-enabled system arranged for a bronchoscopy procedure is shown. System 36 includes a support structure or column 37 for supporting a platform 38 (shown as a "table" or "bed") on a floor. Much like a trolley-based system, the end effector of the robotic arm 39 of system 36 includes an instrument actuator 42, which is designed to manipulate elongated medical instruments, such as… Figure 5 The bronchoscope 40 is used in the bronchoscopy. In practice, the C-arm used to provide fluorescence imaging can be positioned above the patient's upper abdominal region by placing the transmitter and detector around the stage 38.

[0095] Figure 6An alternative view of system 36 without a patient and medical devices is provided for discussion purposes. As shown, column 37 may include one or more brackets 43, shown as annular in system 36, upon which one or more robotic arms 39 may be based. The brackets 43 may translate along a vertical column interface 44 extending along the length of column 37 to provide different vantage points from which the robotic arms 39 may be positioned to reach the patient. The brackets 43 may be rotated about column 37 using mechanical motors positioned within column 37 to allow the robotic arms 39 to access multiple sides of table 38, such as both sides of the patient. In embodiments with multiple brackets, the brackets may be individually positioned on the column and may translate and / or rotate independently of other brackets. While the brackets 43 need not be circular or even encircling column 37, the annular shape shown facilitates rotation of the brackets 43 about column 37 while maintaining structural balance. Rotation and translation of the brackets 43 allow system 36 to align medical devices such as endoscopes and laparoscopes to different access points on the patient. In other embodiments (not shown), system 36 may include a patient examination table or bed with an adjustable arm support, which takes the form of a rod or rail extending beside the patient examination table or bed. One or more robotic arms 39 (e.g., via a shoulder with an elbow joint) may be attached to the adjustable arm support, which can be vertically adjusted. By providing vertical adjustment, the robotic arms 39 can advantageously be compactly stored under the patient examination table or bed and subsequently raised during procedures.

[0096] The robotic arm 39 can be mounted on the bracket 43 via a set of arm mounts 45 comprising a series of joints that can be individually rotated and / or telescopically extended to provide additional configurability to the robotic arm 39. Additionally, the arm mounts 45 can be positioned on the bracket 43 such that, when the bracket 43 is properly rotated, the arm mounts 45 are positioned on the same side of the stage 38 (e.g., ...). Figure 6 As shown), on the opposite side of platform 38 (as shown) Figure 9 (as shown) or on the adjacent side of platform 38 (not shown).

[0097] Column 37 structurally supports platform 38 and provides a path for the vertical translation of bracket 43. Internally, column 37 may be equipped with a lead screw for guiding the vertical translation of the bracket, and a motor for mechanizing the lead screw-based translation of bracket 43. Column 37 may also transmit power and control signals to bracket 43 and the robotic arm 39 mounted thereon.

[0098] Platform base 46 has with Figure 2The trolley base 15 in the illustrated trolley 11 functions similarly, accommodating heavier components to balance the table / bed 38, column 37, bracket 43, and robotic arm 39. The table base 46 may also incorporate rigid casters to provide stability during operation. Casters deployed from the bottom of the table base 46 extend in opposite directions on either side of the base 46 and retract when the system 36 requires movement.

[0099] continue Figure 6 System 36 may also include a tower (not shown) that divides the functionality of system 36 between the table and the tower to reduce the form factor and volume of the table. As in previously disclosed embodiments, the tower may provide the table with a variety of support functions, such as processing, computing and control capabilities, electrical, fluid and / or optical, and sensor processing. The tower may also be movable to be positioned away from the patient, thereby improving physician access and eliminating clutter in the operating room. Additionally, placing components in the tower allows for more storage space in the table base 46 for potential retraction of the robotic arm 39. The tower may also include a main controller or console that provides both a user interface (such as a keyboard and / or widgets) for user input and a display screen (or touchscreen) for preoperative and intraoperative information (such as real-time imaging, navigation, and tracking information). In some embodiments, the tower may also include a gripper for a gas canister to be used for inflatation.

[0100] In some implementations, the base can be retracted and stored when not in use. Figure 7 A system 47 for retracting a robotic arm is illustrated in an embodiment of a platform-based system. In system 47, a bracket 48 can be vertically translated into a base 49 to retract the robotic arm 50, arm mount 51, and bracket 48 within the base 49. A base cover 52 can be translated and retracted to open to deploy the bracket 48, arm mount 51, and robotic arm 50 around a post 53, and to close to retract the bracket, arm mount, and robotic arm for protection when not in use. The base cover 52 can be sealed along the edges of its opening using a membrane 54 to prevent dust and fluid from entering when closed.

[0101] Figure 8An embodiment of a robot-enabled table-based system configured for a ureteroscopy procedure is illustrated. During ureteroscopy, table 38 may include a rotating portion 55 for positioning the patient at an angle to the column 37 and table base 46. The rotating portion 55 may rotate or pivot about a pivot point (e.g., below the patient's head) to position the lower portion of the rotating portion 55 away from the column 37. For example, pivoting of the rotating portion 55 allows a C-arm (not shown) to be positioned above the patient's lower abdomen without competing for space with the column (not shown) below table 38. By rotating a bracket (not shown) about the column 37, a robotic arm 39 may insert a ureteroscope 56 directly into the patient's groin region along a virtual track 57 to reach the urethra. During ureteroscopy, stirrups 58 may also be fixed to the rotating portion 55 of table 38 to support the orientation of the patient's legs during the procedure and allow full access to the patient's groin region.

[0102] In laparoscopic procedures, minimally invasive instruments are inserted into the patient's anatomical structures through one or more small incisions in the abdominal wall. In some embodiments, the minimally invasive instruments include elongated rigid components, such as axes, for accessing the anatomical structures within the patient. After the patient's abdominal cavity is inflated, the instruments can be guided to perform surgical or medical tasks, such as grasping, cutting, ablation, suturing, etc. In some embodiments, the instruments may include endoscopes, such as laparoscopes. Figure 9 An implementation of a table-based, robot-enabled system configured for laparoscopic procedures is shown. Figure 9 As shown, the bracket 43 of system 36 can be rotated and vertically adjusted to position the pair of robotic arms 39 on opposite sides of table 38, so that the instrument 59 can be positioned through the smallest incision on both sides of the patient to reach his / her abdominal cavity using arm mount 45.

[0103] To accommodate laparoscopic procedures, the robot-enabled platform system can also tilt the platform to the desired angle. Figure 10 An implementation scheme for a robot-enabled medical system with pitch or tilt adjustment is shown. For example... Figure 10 As shown, system 36 can adapt to the tilt of platform 38 to position one part of the platform at a greater distance from the base plate than another part. Additionally, arm mount 45 can rotate to match the tilt, ensuring that robot arm 39 maintains the same planar relationship with platform 38. To accommodate steeper angles, column 37 may also include a telescopic portion 60 that allows vertical extension of column 37 to prevent platform 38 from contacting the floor or colliding with platform base 46.

[0104] Figure 11Detailed illustrations are provided of the interface between platform 38 and column 37. The pitch-rotation mechanism 61 can be configured to change the pitch angle of platform 38 relative to column 37 in multiple degrees of freedom. The pitch-rotation mechanism 61 can be implemented by positioning orthogonal axes 1, 2 at the column interface, each axis being actuated by separate motors 3, 4 in response to electrical pitch angle commands. Rotation along one screw 5 enables tilt adjustment along axis 1, while rotation along another screw 6 enables tilt adjustment along another axis 2. In some embodiments, ball joints may be used to change the pitch angle of platform 38 relative to column 37 in multiple degrees of freedom.

[0105] For example, pitch adjustment is particularly useful when attempting to position the table in a head-down, feet-up position (i.e., positioning the patient's lower abdomen higher than their upper abdomen above the floor) for lower abdominal surgery. The head-down, feet-up position causes the patient's internal organs to slide down to their upper abdomen by gravity, clearing the abdominal cavity to allow minimally invasive instruments to enter and perform lower abdominal surgical or medical procedures, such as laparoscopic prostatectomy.

[0106] Figure 12 and Figure 13 Isometric and end views of an alternative embodiment of a stage-based surgical robot system 100 are shown. The surgical robot system 100 includes one or more robotic arms (see, for example) that can be configured to support a stage 101 relative to it. Figure 14 One or more adjustable arm supports 105 are provided. In the illustrated embodiment, a single adjustable arm support 105 is shown, but additional arm supports 105 may be positioned on opposite sides of the platform 101. The adjustable arm supports 105 may be configured such that they are movable relative to the platform 101 to adjust and / or change the position of the adjustable arm support 105 and / or any robotic arm attached to the adjustable arm support relative to the platform 101. For example, the adjustable arm support 105 may be adjusted with one or more degrees of freedom relative to the platform 101. The adjustable arm support 105 provides high flexibility to the system 100, including the ability to easily retract the one or more adjustable arm supports 105 and any robotic arms attached thereto under the platform 101. The adjustable arm support 105 may be raised from the retracted position to a position below the upper surface of the platform 101. In other embodiments, the adjustable arm support 105 may be raised from the retracted position to a position above the upper surface of the platform 101.

[0107] The adjustable arm support 105 provides several degrees of freedom, including lifting, lateral translation, and tilting. Figure 12 and Figure 13 In the exemplary embodiment, the arm support 105 is configured to have four degrees of freedom, which are in Figure 12The arrows indicate the first degree of freedom, which allows adjustment of the adjustable arm support 105 in the z-direction (“Z-lift”). For example, the adjustable arm support 105 may include a bracket 109 configured to move up or down along or relative to the column 102 of the support platform 101. The second degree of freedom allows the adjustable arm support 105 to tilt. For example, the adjustable arm support 105 may include a rotary joint that allows the adjustable arm support 105 to be aligned with the bed in a head-down, feet-up position. The third degree of freedom allows the adjustable arm support 105 to “pivot upwards”, which can be used to adjust the distance between one side of the platform 101 and the adjustable arm support 105. The fourth degree of freedom allows the adjustable arm support 105 to translate along the longitudinal length of the platform.

[0108] Figure 12 and Figure 13 The surgical robot system 100 may include a platform supported by a column 102 mounted to a base 103. The base 103 and the column 102 support the platform 101 relative to a support surface. A floor axis 131 and a support axis 133 are... Figure 13 As shown in the image.

[0109] The adjustable arm support 105 can be mounted to the column 102. In other embodiments, the arm support 105 can be mounted to the platform 101 or the base 103. The adjustable arm support 105 may include a bracket 109, a rod or rail connector 111, and a rod or rail 107. In some embodiments, one or more robotic arms mounted to the rail 107 can translate and move relative to each other.

[0110] The bracket 109 can be attached to the post 102 via a first connector 113, which allows the bracket 109 to move relative to the post 102 (e.g., such as up and down movement along a first axis or vertical axis 123). The first connector 113 can provide a first degree of freedom (“Z-lift”) to the adjustable arm support 105. The adjustable arm support 105 may include a second connector 115, which provides a second degree of freedom (tilt) to the adjustable arm support 105. The adjustable arm support 105 may include a third connector 117, which provides a third degree of freedom (“upward pivot”) to the adjustable arm support 105. An additional connector 119 may be provided (in... Figure 13 (As shown in the diagram), the additional joint mechanically constrains the third joint 117 to maintain the orientation of the guide rail 107 as the guide rail connector 111 rotates about the third axis 127. The adjustable arm support 105 may include a fourth joint 121 that can provide a fourth degree of freedom (translation) for the adjustable arm support 105 along the fourth axis 129.

[0111] Figure 14An end view of a surgical robot system 140A according to one embodiment, having two adjustable arm supports 105A, 105B mounted on opposite sides of a stage 101, is shown. A first robotic arm 142A is attached to a rod or rail 107A of the first adjustable arm support 105B. The first robotic arm 142A includes a base 144A attached to the rail 107A. The distal end of the first robotic arm 142A includes an instrument drive mechanism 146A that can be attached to one or more robotic medical instruments or tools. Similarly, a second robotic arm 142B includes a base 144B attached to the rail 107B. The distal end of the second robotic arm 142B includes an instrument drive mechanism 146B. The instrument drive mechanism 146B can be configured to be attached to one or more robotic medical instruments or tools.

[0112] In some embodiments, one or more of the robotic arms 142A and 142B include an arm with seven or more degrees of freedom. In some embodiments, one or more of the robotic arms 142A and 142B may include eight degrees of freedom, including an insertion axis (including one degree of freedom for insertion), a wrist (including three degrees of freedom for wrist pitch, yaw, and roll), an elbow (including one degree of freedom for elbow pitch), a shoulder (including two degrees of freedom for shoulder pitch and yaw), and a base 144A and 144B (including one degree of freedom for translation). In some embodiments, the insertion degree of freedom may be provided by the robotic arms 142A and 142B, while in other embodiments, the instrument itself provides insertion via an instrument-based insertion architecture.

[0113] C. Instrument drivers and interfaces .

[0114] The end effector of the system's robotic arm may include: (i) an instrument actuator (alternatively referred to as an "instrument drive mechanism" or "instrument device manipulator") incorporating electromechanical devices for actuating medical devices; and (ii) a removable or detachable medical device, which may lack any electromechanical components, such as motors. This dichotomy may be driven by the need to sterilize medical devices used in medical procedures, and the inability to adequately sterilize expensive capital equipment due to its complex mechanical components and sensitive electronics. Therefore, medical devices may be designed to be detached, removed, and interchanged from the instrument actuator (and thus from the system) for individual sterilization or disposal by a physician or physician staff. In contrast, the instrument actuator does not need to be altered or sterilized and can be covered for protection.

[0115] Figure 15An example instrument actuator is shown. The instrument actuator 62, positioned at the distal end of a robotic arm, includes one or more drive units 63 arranged parallel to an axis to provide controlled torque to a medical device via a drive shaft 64. Each drive unit 63 includes a separate drive shaft 64 for interacting with the device, a gear head 65 for converting motor shaft rotation into desired torque, a motor 66 for generating drive torque, an encoder 67 for measuring the speed of the motor shaft and providing feedback to control circuitry, and control circuitry 68 for receiving control signals and actuating the drive unit. Each drive unit 63 is independently controlled and motorized, and the instrument actuator 62 can provide multiple (e.g., as...) to the medical device. Figure 15 Four independent drive outputs are shown. In operation, the control circuit 68 receives control signals, transmits motor signals to the motor 66, compares the motor speed measured by the encoder 67 with the desired speed, and modulates the motor signals to generate the desired torque.

[0116] For procedures requiring a sterile environment, the robotic system can incorporate a drive interface, such as a sterile adapter connected to a sterile cover, positioned between the instrument actuator and the medical device. The primary purpose of the sterile adapter is to transmit angular motion from the drive shaft of the instrument actuator to the drive input of the device, while maintaining physical separation between the drive shaft and the drive input, and thus maintaining sterility. Therefore, an exemplary sterile adapter may include a series of rotary inputs and rotary outputs designed to mate with the drive shaft of the instrument actuator and the drive input on the device. The sterile cover, composed of a thin, flexible material (such as transparent or translucent plastic), is connected to the sterile adapter and designed to cover the capital device, such as an instrument actuator, robotic arm, and trolley (in trolley-based systems) or table (in table-based systems). The use of the cover allows the capital device to be positioned near the patient while still within an area that does not require sterilization (i.e., a non-sterile area). On the other side of the sterile cover, the medical device can dock with the patient in an area that requires sterilization (i.e., a sterile area).

[0117] D. Medical devices .

[0118] Figure 16An example medical device with paired instrument actuators is shown. Similar to other devices designed for use with robotic systems, the medical device 70 includes an elongated shaft 71 (or elongated body) and an instrument base 72. The instrument base 72, also referred to as the “instrument handle” due to its intended design for manual interaction by a physician, typically includes a rotatable drive input 73 (e.g., a socket, pulley, or reel) designed to mate with a drive output 74 on a drive interface extending through the distal end of the robotic arm 76. When physically connected, latched, and / or coupled, the mating drive input 73 of the instrument base 72 may share a rotational axis with the drive output 74 in the instrument driver 75 to allow torque to be transmitted from the drive output 74 to the drive input 73. In some embodiments, the drive output 74 may include a spline designed to mate with a socket on the drive input 73.

[0119] The elongated shaft 71 is designed to be delivered through an anatomical opening or cavity (e.g., as in endoscopy) or through a minimally invasive incision (e.g., as in laparoscopy). The elongated shaft 71 can be flexible (e.g., having endoscope-like properties) or rigid (e.g., having laparoscopy-like properties), or a customized combination of both flexible and rigid portions. When designed for laparoscopy, the distal end of the rigid elongated shaft can be connected to an end effector extending from a connector wrist formed by a connecting fork having at least one degree of freedom and a surgical tool or medical instrument (such as, for example, a gripper or scissors), which can be actuated based on forces from a tendon when the drive input rotates in response to torque received from the drive output 74 of the instrument actuator 75. When designed for endoscopy, the distal end of the flexible elongated shaft can include a manipulable or controllable bending segment that articulates and bends based on torque received from the drive output 74 of the instrument actuator 75.

[0120] Torque from the instrument actuator 75 is transmitted along the elongated axis 71 using tendons. These individual tendons (e.g., traction cables) can be individually anchored to individual drive inputs 73 within the instrument handle 72. From the instrument handle 72, the tendons are guided downward along one or more traction chambers of the elongated axis 71 and anchored at the distal portion of the elongated axis 71, or at the wrist at the distal portion of the elongated axis. During surgical procedures such as laparoscopy, endoscopy, or hybrid procedures, these tendons can be coupled to distally mounted end effectors, such as wrists, grippers, or scissors. In such an arrangement, torque applied to the drive input 73 transmits tension to the tendons, thereby actuating the end effector in a certain way. In some embodiments, during surgical procedures, the tendons can cause the connector to rotate about the axis, thereby causing the end effector to move in one direction or the other. Alternatively, the tendon may be connected to one or more jaws of the gripper at the distal end of the elongated shaft 71, wherein tension from the tendon causes the gripper to close.

[0121] During endoscopy, tendons can be attached via adhesives, control rings, or other mechanical fasteners to flexural or articulated segments positioned along an elongated axis 71 (e.g., at the distal end). When fixedly attached to the distal end of a flexural segment, torque applied to the drive input 73 is transmitted down the tendon, causing the softer flexural segment (sometimes referred to as an articulated segment or region) to flex or articulate. Along non-flexural segments, it can be advantageous to helve or coil individual traction cavities that guide individual tendons along the wall (or inside) of the endoscope axis to balance radial forces caused by tension in the traction lines. For specific purposes, the angle of the helices and / or the spacing between them can be varied or designed, with tighter helices exhibiting less axial compression under load, while a lower amount of helix causes greater axial compression under load but restricts flexion. Alternatively, traction cavities can be guided parallel to the longitudinal axis of the elongated axis 71 to allow controlled articulation within the desired flexural or articulated segment.

[0122] In endoscopic procedures, the elongated shaft 71 accommodates multiple components to assist in robotic procedures. The shaft 71 may include, at its distal end, a working channel for deploying surgical instruments (or medical devices), rinsing and / or aspirating the surgical area. The shaft 71 may also be adapted with wires and / or optical fibers to transmit signals to / from optical components at its distal end, which may include an optical camera. The shaft 71 may also be adapted with optical fibers to carry light from a proximal light source (such as a light-emitting diode) to the distal end of the shaft 71.

[0123] At the distal end of the instrument 70, the distal end may also include an opening for delivering tools for diagnostic and / or treatment, irrigation, and aspiration to the surgical site. The distal end may also include a port for a camera (such as a fiberoptic endoscope or digital camera) to capture images of the internal anatomical space. Relatedly, the distal end may also include a port for a light source used to illuminate the anatomical space when the camera is used.

[0124] exist Figure 16 In the example, the axis of the drive shaft, and therefore the axis of the drive input, is orthogonal to the axis of the elongated shaft 71. However, this arrangement complicates the rolling capability of the elongated shaft 71. Rolling the elongated shaft along its axis while keeping the drive input 73 stationary can cause undesirable tangling of the tendon as it extends from the drive input 73 and enters the traction cavity within the elongated shaft 71. Such tendon tangling can disrupt any control algorithms designed to predict the movement of the flexible elongated shaft 71 during endoscopic procedures.

[0125] Figure 17 An alternative design of the instrument actuator and instrument is shown, wherein the axis of the drive unit is parallel to the axis of the slender axis of the instrument. As shown, the circular instrument actuator 80 includes four drive units whose drive outputs 81 are aligned parallel to each other at the end of the robot arm 82. The drive units and their respective drive outputs 81 are housed in a rotating assembly 83 of the instrument actuator 80, driven by one of the drive units within assembly 83. In response to torque provided by the rotating drive unit, the rotating assembly 83 rotates along a circular bearing that connects the rotating assembly 83 to the non-rotating portion 84 of the instrument actuator 80. Electrical and control signals can be transmitted from the non-rotating portion 84 of the instrument actuator 80 to the rotating assembly 83 via electrical contacts, which can be maintained by rotation of a brush slip ring connection (not shown). In other embodiments, the rotating assembly 83 may be responsive to a separate drive unit integrated into the non-rotating portion 84 and therefore not parallel to the other drive units. The rotation mechanism 83 allows the instrument actuator 80 to allow the drive units and their respective drive outputs 81 to rotate as a single unit about the instrument actuator axis 85.

[0126] Similar to previously disclosed embodiments, the device 86 may include an elongated shaft portion 88 and a device base 87 (shown as having a transparent outer surface for discussion purposes), the device base including a plurality of drive inputs 89 (such as sockets, pulleys, and reels) configured to receive drive outputs 81 in the device driver 80. Unlike previously disclosed embodiments, the device shaft 88 extends from the center of the device base 87, and the axis of the device base is substantially parallel to the axes of the drive inputs 89, rather than... Figure 16 It is orthogonal as in the design.

[0127] When coupled to the rotating assembly 83 of the instrument driver 80, the medical instrument 86, including the instrument base 87 and the instrument shaft 88, rotates in combination with the rotating assembly 83 about the instrument driver axis 85. Since the instrument shaft 88 is positioned at the center of the instrument base 87, it is coaxial with the instrument driver axis 85 when attached. Therefore, rotation of the rotating assembly 83 causes the instrument shaft 88 to rotate about its own longitudinal axis. Furthermore, when the instrument base 87 rotates together with the instrument shaft 88, any tendons connected to the drive input 89 in the instrument base 87 do not become entangled during rotation. Therefore, the parallelism of the axes of the drive output 81, the drive input 89, and the instrument shaft 88 allows the shaft to rotate without causing any control tendons to become entangled.

[0128] Figure 18 A device 150 with a device-based insertion architecture according to some embodiments is shown. Device 150 is connectable to any of the device actuators described above. Device 150 includes an elongated shaft 152, an end effector 162 connected to the shaft 152, and a shank 170 connected to the shaft 152. The elongated shaft 152 includes a tubular member having a proximal portion 154 and a distal portion 156. The elongated shaft 152 includes one or more channels or grooves 158 along its outer surface. The grooves 158 are configured to receive one or more wires or cables 180 passing through the grooves. Thus, one or more cables 180 extend along the outer surface of the elongated shaft 152. In other embodiments, the cables 180 may also pass through the elongated shaft 152. Manipulation of the one or more cables 180 (e.g., via a device actuator) actuates the end effector 162.

[0129] The instrument handle 170 (also referred to as the instrument base) typically includes an attachment interface 172 having one or more mechanical inputs 174, such as jacks, pulleys, or spools, which are designed to reciprocately engage with one or more torque couplers on the attachment surface of the instrument actuator.

[0130] In some embodiments, the instrument 150 includes a series of pulleys or cables that enable the elongated shaft 152 to translate relative to the handle 170. In other words, the instrument 150 itself includes an instrument-based insertion architecture that adapts to the insertion of the instrument, thereby minimizing reliance on a robotic arm to provide the insertion of the instrument 150. In other embodiments, the robotic arm may be largely responsible for the instrument insertion.

[0131] E. Controller .

[0132] Any of the robotic systems described herein may include an input device or controller for manipulating a device attached to a robotic arm. In some embodiments, the controller may be coupled to the device (e.g., communicatively, electronically, electrically, wirelessly, and / or mechanically) such that manipulation of the controller, for example via master-slave control, causes corresponding manipulation of the device.

[0133] Figure 19 This is a perspective view of an embodiment of controller 182. In this embodiment, controller 182 includes a hybrid controller that may have both impedance and admittance control. In other embodiments, controller 182 may utilize only impedance or passive control. In other embodiments, controller 182 may utilize only admittance control. By being a hybrid controller, controller 182 advantageously has lower perceived inertia during use.

[0134] In the illustrated embodiment, controller 182 is configured to allow manipulation of two medical devices and includes two handles 184. Each handle 184 is connected to a universal joint 186. Each universal joint 186 is connected to a positioning platform 188.

[0135] like Figure 19 As shown, each positioning platform 188 includes a Selective Compliant Assembly Robotic Arm (SCARA) 198 connected to a post 194 via a prism joint 196. The prism joint 196 is configured to translate along the post 194 (e.g., along track 197) to allow each handle 184 to translate in the z-direction, thus providing a first degree of freedom. The SCARA 198 is configured to allow the handles 184 to move in the xy-plane, thus providing two additional degrees of freedom.

[0136] In some embodiments, one or more load sensors are located within the controller. For example, in some embodiments, load sensors (not shown) are located within the body of each gimbal in gimbal 186. By providing load sensors, portions of controller 182 are capable of operating under admittance control, thereby advantageously reducing the sense inertia of the controller during use. In some embodiments, positioning platform 188 is configured for admittance control, while gimbal 186 is configured for impedance control. In other embodiments, gimbal 186 is configured for admittance control, while positioning platform 188 is configured for impedance control. Thus, for some embodiments, the translational or positional degrees of freedom of positioning platform 188 may depend on admittance control, while the rotational degrees of freedom of gimbal 186 may depend on impedance control.

[0137] F. Navigation and Control .

[0138] Traditional endoscopy can involve the use of fluoroscopy (e.g., delivered via a C-arm) and other forms of radiation-based imaging modalities to provide intracavitary guidance to the operating physician. In contrast, the robotic system envisioned in this disclosure can provide radiation-free navigation and positioning, reducing physician exposure to radiation and the amount of equipment required in the operating room. As used herein, the term "positioning" can refer to determining and / or monitoring the orientation of an object in a reference coordinate system. Techniques such as preoperative mapping, computer vision, real-time EM tracking, and robot command data can be used individually or in combination to achieve a radiation-free operating environment. In other cases where radiation-based imaging modalities are still used, preoperative mapping, computer vision, real-time EM tracking, and robot command data can be used individually or in combination to improve upon information obtained solely through radiation-based imaging modalities.

[0139] Figure 20 This is a block diagram illustrating a positioning system 90 for estimating the position of one or more components of a robotic system (such as the position of a machine) according to an exemplary embodiment. The positioning system 90 may be one or more computer devices configured to execute one or more instructions. The computer devices may be embodied by a processor (or multiple processors) and computer-readable storage among the components discussed above. By way of example and not limitation, the computer device may be located in... Figure 1 Tower 30 shown Figures 1 to 4 The trolley 11 shown Figures 5 to 14 The bed, etc. shown.

[0140] like Figure 20 As shown, the positioning system 90 may include a positioning module 95 that processes input data 91-94 to generate position data 96 for the distal end of a medical device. The position data 96 may be data or logic representing the position and / or orientation of the distal end of the device relative to a reference frame. The reference frame may be relative to a patient's anatomy or a known object (such as an EM field generator) (see the discussion of EM field generators below).

[0141] The various input data 91-94 are now described in more detail. The localization module 95 can use preoperative mapping to generate model data 91. Preoperative mapping can be accomplished using a collection of low-dose CT scans. The preoperative CT scans are reconstructed into three-dimensional images that are visualized, for example, as “slices” of cross-sectional views of the patient’s internal anatomy. When analyzed in general, image-based models of the anatomical cavities, spaces, and structures of the patient’s anatomical structures, such as the patient’s lung network, can be generated. Techniques such as centerline geometry can be determined and approximated from CT images to form a three-dimensional volume of the patient’s anatomy, referred to as model data 91 (also called “preoperative model data” when generated using only preoperative CT scans). The use of centerline geometry is discussed in U.S. Patent Application 14 / 523,760, the contents of which are incorporated herein by reference in their entirety. Network topology models can also be derived from CT images and are particularly well-suited for bronchoscopy.

[0142] In some implementations, the instrument may be equipped with a camera to provide visual data (or image data) 92 to the positioning module 95. The positioning module 95 may process the visual data 92 to enable one or more vision-based (or image-based) position tracking modules or features. For example, preoperative model data 91 may be used in conjunction with the visual data 92 to enable computer vision-based tracking of a medical instrument (e.g., an endoscope or an instrument that advances through the working channel of an endoscope). For example, using the preoperative model data 91, a robotic system may generate a library of expected endoscopic images based on the model, with each image linked to a location within the model, based on the expected path of the endoscope's movement. During surgical procedures, the robotic system may refer to this library to compare real-time images captured at a camera (e.g., a camera at the distal end of the endoscope) with those images in the image library to aid in positioning.

[0143] Other computer vision-based tracking techniques use feature tracking to determine camera motion, and thus, endoscope motion. Some features of the localization module 95 can identify circular geometries corresponding to anatomical cavities in the preoperative model data 91 and track changes in those geometries to determine which anatomical cavity has been selected, as well as track the relative rotation and / or translational motion of the camera. The use of a topology map can further enhance vision-based algorithms or techniques.

[0144] Optical flow (another computer vision-based technique) can analyze the displacement and translation of image pixels in a video sequence within visual data 92 to infer camera motion. Examples of optical flow techniques can include motion detection, object segmentation computation, brightness, motion compensation coding, stereo parallax measurement, and more. Through multiple iterations and comparisons of multiple frames, the motion and position of the camera (and therefore the endoscope) can be determined.

[0145] The positioning module 95 can use real-time EM tracking and EM data 93 to generate the real-time position of the endoscope in a global coordinate system that can be registered to the patient's anatomy represented by a preoperative model. In EM tracking, an EM sensor (or tracker), including one or more sensor coils embedded in one or more locations and orientations within the medical instrument (e.g., an endoscopic tool), measures changes in the EM field generated by one or more static EM field generators positioned at known locations. The positional information detected by the EM sensor is stored as EM data 93. The EM field generator (or transmitter) can be placed close to the patient to generate a low-intensity magnetic field detectable by the embedded sensor. The magnetic field induces a small current in the sensor coil of the EM sensor, which can be analyzed to determine the distance and angle between the EM sensor and the EM field generator. These distances and orientations can be "registered" to the patient's anatomy (e.g., a preoperative model) during surgery to determine the geometric transformations that align a single location in the coordinate system with its orientation in the preoperative model of the patient's anatomy. Once registered, an embedded EM tracker in one or more orientations of the medical device (e.g., the distal end of an endoscope) can provide real-time indication of the medical device’s progress through the patient’s anatomy.

[0146] Robot commands and kinematic data 94 can also be used by the positioning module 95 to provide position data 96 for the robotic system. Device pitch and yaw from joint movement commands can be determined during preoperative calibration. During surgery, these calibration measurements can be combined with known insertion depth information to estimate the instrument's orientation. Alternatively, these calculations can be analyzed in conjunction with EM, vision, and / or topology modeling to estimate the medical device's orientation within the network.

[0147] Figure 20 As shown, the positioning module 95 can use multiple other input data. For example, although Figure 20 Not shown, but the device using shape sensing fibers can provide shape data, which the positioning module 95 can use to determine the position and shape of the device.

[0148] The localization module 95 can use the input data 91-94 in combination. In some cases, such combination can use a probabilistic method, where the localization module 95 assigns confidence weights to the location determined based on each of the input data 91-94. Therefore, in cases where the EM data may be unreliable (e.g., in the presence of EM interference), the confidence of the location determined by the EM data 93 may be reduced, and the localization module 95 may rely more heavily on the visual data 92 and / or robot commands and kinematic data 94.

[0149] As discussed above, the robotic systems discussed in this paper can be designed as a combination of one or more of the technologies mentioned above. The computer-based control system of a robotic system located in a tower, bed, and / or trolley can store computer program instructions in, for example, a non-transitory computer-readable storage medium (such as a permanent magnetic storage drive, a solid-state drive, etc.). When executed, these computer program instructions cause the system to receive and analyze sensor data and user commands, generate control signals for the entire system, and display navigation and positioning data, such as the instrument's position in a global coordinate system, anatomical diagrams, etc.

[0150] 2. Robotic Controllable Field Generator

[0151] This document describes implementations of robotic controllable EM field generators that can be configured for use with robotic medical systems, such as those described above, and other robotic medical systems. The robotic controllable EM field generator can be configured to be coupled to (or otherwise attached to or integrated into) the robotic arm of the robotic medical system. Such robotic controllable EM field generators offer advantages over other EM field generators typically used with robotic medical systems, which often require more complex registration steps to correlate the coordinate system of the EM field with the coordinate system of the robotic system or the global coordinate system, and are typically not repositionable during the procedure without repeating the complex registration steps.

[0152] By attaching an EM field generator to a robotic arm, the arm's kinematics can be used to register the EM field's coordinate system to the robot or global coordinate system. This robot arm-based kinematics-based registration can occur automatically or without requiring a separate registration step by the operator (as shown in the reference below). Figure 22B and Figure 22C (The registration steps described). This eliminates the need for the more complex or user-involved registration common with other types of EM field generators used with robotic medical systems. Furthermore, when the EM field generator is coupled to a robotic arm, the EM field generator becomes robotically controllable or positionable. During robotic medical procedures, the robotic arm can be commanded to translate, rotate, or a combination of both to physically move the EM field generator to various locations.

[0153] As will be described in more detail below, such robotic controllable EM field generators offer several advantages and are useful in a wide variety of applications. For example, using such robotic controllable EM field generators can: eliminate the need to perform a separate registration step to register the EM field's coordinate system with the robot or global coordinate system; provide a wider field of view through the ability to move the EM field generator with a robotic arm; allow the use of smaller EM field generators (e.g., compact field generators); simplify the setup of robotic medical systems; and increase the accuracy of EM sensor detection positions, etc. Applications of such robotic controllable EM field generators can include, for example, automated tool tracking, automated or enhanced field generator setup, EM focusing, anatomical mapping, and other applications.

[0154] After providing a brief overview of the use of EM sensors and EM field generators in robotic medical systems, these and other features of robotic controllable field generators will be described in more detail below.

[0155] A. A brief overview of the use of EM sensors and EM field generators

[0156] Robotic medical systems can utilize various types of position sensors to facilitate the tracking and navigation of various tools and instruments. As mentioned above, one type of position sensor that can be used in robotic medical systems is the EM position sensor (also referred to herein as an EM sensor, beacon, or tracker). One or more EM sensors can be placed on, for example, a part of a tool or instrument used during a medical procedure. The position of the EM sensor can be determined and used to track the position of the corresponding tool or instrument. One or more EM sensors can also be placed on the patient's body, for example, to track patient movement (such as movement due to breathing or other types of movement) during the procedure. As mentioned above... Figure 20 As noted, the positioning system 90 can use EM data 93 and various other types of data to provide navigation and guidance information to the operator of the system.

[0157] An EM sensor is used in conjunction with one or more EM field generators configured to generate a low-intensity magnetic field. When the EM sensor is positioned within the magnetic field generated by the EM field generator (also known as the working volume of the EM field generator), the position of the EM sensor relative to the EM field generator can be determined. For example, as referenced above... Figure 20 The EM sensor may include one or more sensor coils that measure changes in the magnetic field generated by the EM field generator. The magnetic field can induce a small current in the sensor coil of the EM sensor, and this small current can be analyzed to determine, for example, the distance and angle between the EM sensor and the EM field generator.

[0158] In some implementations, the position of the EM sensor relative to the EM field generator (e.g., three-dimensional position) can be determined. In some implementations, the orientation of the EM sensor (e.g., pitch, yaw, and / or roll) can also be determined. A five-degree-of-freedom (DoF) EM sensor can provide the three-dimensional position of the EM sensor as well as the pitch and yaw of the EM sensor. A six-DoF EM sensor can provide the three-dimensional position of the EM sensor as well as the pitch, yaw, and roll of the EM sensor.

[0159] Figure 21 An example of an EM field generator 202 and an EM sensor 204 is shown. The EM field generator 202 is configured to generate a magnetic field. In the illustrated embodiment, the magnetic field of the EM field generator 202 has an operating volume 206. When the EM sensor 204 is positioned within the operating volume 206 of the magnetic field generated by the EM field generator 202, the position (or location and orientation) of the EM sensor 204 can be determined relative to an EM coordinate system 208 associated with the EM field generator 202. For example, the distance and angle between the EM sensor 204 and the origin of the EM coordinate system 208 can be determined, such that the position (e.g., x, y, and z position) and / or orientation (e.g., pitch, yaw, and / or roll) of the EM sensor 204 within the operating volume 206 can be determined.

[0160] In the illustrated embodiment, the working volume 206 of the EM field generator 202 is represented as having the shape of a rectangular prism projected from one side of the EM field generator 202. However, this represents only one type of working volume 206, and EM field generators 202 can be provided to produce working volumes 206 of various sizes, shapes, and positions relative to the EM field generator 202.

[0161] Figure 22A It shows the use of Figure 21 An exemplary robotic medical system 200 with an EM field generator 202. The robotic medical system 200 may be similar to the one described above. Figures 1 to 20 The described robotic medical system. In the illustrated embodiment, the robotic medical system 200 includes a cart 210 with two robotic arms 212. Although shown as having two robotic arms 212, other numbers of robotic arms 212 are also possible. For example, Figure 2 The above description illustrates an exemplary trolley 11 including three robotic arms 12. Additionally, in some embodiments, the robotic arms 212 do not need to be attached to the trolley 210. In some embodiments, the robotic arms may be coupled to a bed or patient platform 215, for example, as... Figures 5 to 10 and Figure 14 As shown (as described above). In some embodiments, the robotic medical system 200 may include a robotic arm 212 connected to a trolley 210, a patient platform, and / or other components of the system.

[0162] Robotic arm 212 can be associated with robot coordinate system 216. Due to the known kinematics of robotic arm 212, the position of robotic arm 212 (e.g., the position of the distal end of robotic arm 212 and / or the position of the device 214 attached to robotic arm) can be determined with reference to robot coordinate system 216. For example, because the lengths of the various links that make up robotic arm 212 are known and the angles between the links of robotic arm 212 can be determined, the position of robotic arm is kinematically defined within robot coordinate system 216.

[0163] In some embodiments, the position and orientation of the trolley 210 relative to the patient platform 215 and the patient can be determined, set, or controlled such that the robot coordinate system 216 can be considered a global coordinate system that can be registered to the patient or a part of the patient's anatomy. For example, in some embodiments, up, down, right, left, etc., in the robot coordinate system 216 can correspond to up, down, right, left, etc., in a global coordinate system that includes the patient, the patient platform 215, and / or other components.

[0164] like Figure 22A As shown, a tool or instrument 214 (e.g., a laparoscopic or endoscopic instrument) can be coupled to one of the robotic arms 212. The instrument 214 can be inserted into a patient to perform medical procedures. Although only a single instrument 214 is shown, it should be understood that other numbers of instruments 214 (e.g., two or more instruments) can be used in other embodiments. One or more robotic arms 212 can be used to position and control the instrument 214. An operator (not shown) can use a controller to control the robotic arms 212 and the instrument 214.

[0165] In the illustrated embodiment, device 214 includes an EM sensor 204 located at its distal end. Furthermore, the illustrated embodiment includes a second EM sensor 204 attached to the patient's chest, which can be used to track patient movement or motion, such as movement caused by the patient's breathing.

[0166] Figure 22AAs shown, in some embodiments, the EM field generator 202 may be positioned relative to the patient such that the working volume 206 of the field generator 202 overlaps with a portion of the patient's anatomy used for performing medical procedures. As used herein, the portion of the patient's anatomy used for performing medical procedures may be referred to as a medical site, and this site may include a site where any medical procedure (including biopsy, endoscopy, surgery, treatment, etc.) can be performed. In some embodiments, the EM field generator 202 may be supported by a support frame or other support structure (not shown) such that the working volume 206 is positioned relative to the patient. In other embodiments, the EM field generator 202 may be supported by or attached to a patient platform 215, or integrated into another component within the operating environment (such as those referenced above). Figure 1 , Figure 3 and Figure 4 The described trolley 30) is either in or supported by this other component.

[0167] As referenced above Figure 21 As described above, when the EM sensor 204 is positioned within the working volume 206 of the EM field generator 202, the position of the EM sensor 204 within the EM coordinate system 208 can be determined. However, determining the position of the EM sensor 204 within the EM coordinate system 208 may not be particularly useful unless the EM coordinate system 208 has been registered to the robot coordinate system 216 or the global coordinate system (as noted above, the global coordinate system can be the robot coordinate system 216). Therefore, a registration step that associates the EM coordinate system 208 with the robot coordinate system 216 is typically required.

[0168] In some robotic medical systems such as Figure 22A In the robotic medical system 200 shown, registering the EM coordinate system 208 to the robot coordinate system 216 may require the operator to perform certain steps to achieve the registration. The registration process can add steps to the procedure and increase the procedure time. Figure 22B and Figure 22C Exemplary registration steps or procedures are shown for registering a robot coordinate system 216 associated with a robotic medical system 200 with an EM coordinate system 208 associated with an EM field generator 202. As will be described in more detail below, this can be achieved by providing, for example... Figure 23 The robotic medical system shown has an EM field generator attached to the robotic arm of the robotic medical system to eliminate or reduce the need to perform these types of registration steps or procedures.

[0169] Figure 22BAn exemplary registration procedure is shown that can be used to register a robot coordinate system 216 associated with a robotic medical system 200 with an EM coordinate system 208 associated with an EM field generator 202. To achieve registration, multiple points are identified within both the robot coordinate system 216 and the EM coordinate system 208. Once multiple points are identified within each of the robot coordinate system 216 and the EM coordinate system 208, the registration algorithm determines the transformation between the two coordinate systems. Figure 22B The registration procedure utilizes one of the following components of the robot system: robot arm 212, EM field generator 202, EM probe 203, and registration fixture 205. In the illustrated embodiment, registration fixture 205 comprises a cube with reference points or markers located at its vertices. Thus, in the illustrated embodiment, registration fixture 205 comprises eight reference points. Other numbers of reference points and other shapes of registration fixture 205 may also be used. With registration fixture 205 positioned in a stationary position, the operator can command robot arm 212 to touch each of the reference points of registration fixture 205. The position of each reference point within robot coordinate system 216 can thus be determined. The operator can then touch each of the reference points using EM probe 203. EM probe 203 may include a handheld EM sensor 204 having a tip position defined relative to field generator 202. By touching each of the reference points with EM probe 203, the position of each reference point within EM coordinate system 208 can be determined. Given that the same point (the reference point of the registration fixture 205) is now known in each of the robot coordinate system 216 and the EM coordinate system 208, the registration algorithm can determine the transformation between the two coordinate systems.

[0170] Figure 22C Another exemplary registration procedure is shown for aligning a robot coordinate system 216 associated with a robotic medical system 200 with an EM coordinate system 208 associated with an EM field generator 202. This registration procedure uses one of the robot arm 212 of the robotic system, the EM field generator 202, and an EM sensor 204 attached to the end effector of the robot arm 212. Using the EM sensor 204 attached to the end effector of the robot arm 212, an operator can command the robot arm 212 to move through or along a registration trajectory 207. In the illustrated embodiment, the registration trajectory 207 comprises a cube shape, although other registration trajectories including different shapes may be used. At various points along the registration trajectory 207 (e.g., vertices of the cube shape), the position of the end effector in the robot coordinate system 216 and the position of the EM sensor 204 in the EM coordinate system 208 can be recorded. Once a sufficient number of points have been identified using both the robot coordinate system 216 and the EM coordinate system 208, the registration algorithm can determine the transformation between the two coordinate systems.

[0171] In robotic systems such as Figure 22A In the system 200 shown above, a registration procedure can be performed before the procedure begins, as described in the above reference. Figure 22B and Figure 22C Any of the described procedures is used to register the EM coordinate system 208 and the robot coordinate system 216.

[0172] Various drawbacks are associated with these types of registration. For example, such registration can be tedious and time-consuming. These types of registration are generally not considered automatic because they may require user input and / or may need to be performed before starting a robotic medical procedure. Furthermore, such registration can introduce inaccuracies into the system, for example, if the operator does not navigate precisely to the reference point. Additionally, such registration requires the position of the EM field generator 202 to remain fixed during the procedure. If the EM field generator 202 is moved (e.g., accidentally bumped by someone in the operating room or moved to allow access to the patient), registration will need to be performed again to re-establish the relationship. This can be problematic because the EM field generator 202 may need to be moved, for example, to allow access to a fluorescein C-arm. In this case, the EM field generator 202 would be moved, the C-arm brought into position to capture one or more images, then the C-arm would be moved again, and the EM field generator 202 would be brought back into position, requiring the operator to redo the EM coordinate system to robot or global coordinate system registration steps.

[0173] As will be described in more detail below, this can be achieved by using, for example, Figures 23 to 25 The robotic controllable field generator shown can be directly coupled to the robot arm to reduce or eliminate the need for these types of registration. By coupling the robotic controllable field generator to the robot arm, the relationship or registration between the robotic controllable field generator and the robot arm can be kinematically determined, thereby placing the EM coordinate system 208 within the robot coordinate system 216.

[0174] Additionally, due to the use of references, etc. Figure 22B and Figure 22C In the prior systems of the registration procedures described, the position of the EM field generator 202 must remain fixed during the procedure. Therefore, it may be necessary to use a physically larger and sometimes bulkier EM field generator 202 to generate a working volume 206 large enough to cover the relevant medical site. In some cases or applications, this can increase the difficulty of setting up the EM field generator 202 and positioning it relative to the patient. In some cases or applications, this may limit the available space around the patient during the procedure and / or require additional setup time. Robotic controllable field generators that can be directly coupled to a robotic arm (e.g., as shown in Figures 22 to...) Figure 25 The use of (as shown) can reduce or eliminate the use of (as shown) Figure 22AOne or more of these limitations are associated with the EM field generator 202 shown.

[0175] B. Overview of Robotic Controllable Field Generators

[0176] Figure 23 An embodiment of a robotic medical system 300 is shown, which includes an embodiment of a robotic controllable EM field generator 302 coupled to a robotic arm 212. As will be discussed in more detail below, coupling the EM field generator 302 to the robotic arm 212 can provide several benefits, including, for example: (1) allowing for more accurate and / or simpler registration between the EM coordinate system 308 of the EM field generator 302 and the robot coordinate system 216 based on the kinematics of the robotic arm 212, and / or (2) allowing the position of the EM field generator 302 to be moved or readjusted using the robotic arm 212.

[0177] Similar to Figure 22A The robotic medical system 200, in Figure 23 In the robotic medical system 300, the robotic arm 212 can be associated with a robot coordinate system 216. Due to the known kinematics of the robotic arm 212, its position can be determined with reference to the robot coordinate system 216. For example, because the lengths of the various links constituting the robotic arm 212 are known and the angles between the links can be determined, the position of the robotic arm 212 is kinematically defined within the robot coordinate system 216. Furthermore, in some embodiments of the robotic medical system 300, the position and orientation of the trolley 210 relative to the patient platform 215 and the patient can be determined, set, or controlled such that the robot coordinate system 216 can be considered a global coordinate system that can be registered to the patient or a part of the patient's anatomy. For example, in some embodiments, up, down, right, left, etc., within the robot coordinate system 216 can correspond to up, down, right, left, etc., within a global coordinate system including the patient, the patient platform 215, and / or other components. This facilitates the operator's ability to navigate within the patient's anatomy. As a more specific example, during a bronchoscopy or ureteroscopy procedure in which the robotic system 300 includes a trolley with a robotic arm 212 extending from it, the trolley can be positioned parallel to the bed. The bed and the trolley share a common gravity vector (i.e., both pointing upwards towards the ceiling). Using this information, the system can show the user which direction is upward (anterior) or downward (posterior) as they are maneuvering the bronchoscope or ureteroscope, providing spatial context. In the case of urological applications, during the selection of a target for percutaneous access, the user may need to identify whether the ureteroscope is in the posterior or anterior renal calyx. Since the orientation of the system is known, the system can provide anatomical context to facilitate this navigation.

[0178] like Figure 23 As shown, the EM field generator 302 can be coupled to one of the robot arms 212. In some embodiments, the EM field generator 302 is coupled or attached to the distal end of the robot arm 212, although the EM field generator 302 may be coupled or attached to the robot arm 212 in other locations (e.g., between the distal and proximal ends of the arm). The following description... Figure 24 and Figure 25 A more detailed embodiment of the EM field generator 302 is shown, and the manner in which it can be coupled to the robot arm 212 in some embodiments is illustrated. Other methods and mechanisms for coupling the EM field generator 302 to the robot arm 212 are also possible. Furthermore, in some embodiments, the EM field generator 302 may be integrated into the robot arm 212 itself (e.g., it may be a component on or within the robot arm). For example, the EM field generator 302 may be integrated into one of the links of the robot arm 212 (such as the distal link) or into a machine actuator or machine drive mechanism located, for example, at the distal end of the robot arm 212. Figures 14 to 17 Exemplary instrument drivers 62, 75, 83, 146 into which the EM field generator 302 may be incorporated are shown, although these examples are not limiting.

[0179] By attaching the EM field generator 302 to the robot arm 212, the kinematics of the robot arm 212 can be used to provide registration between the EM coordinate system 308 of the EM field generator 302 and the robot coordinate system 216 and / or the global coordinate system. That is, the kinematics of the robot arm 212 (as noted above, the kinematics are known) can be used to correlate the EM coordinate system 308 of the EM field generator 302 with the robot coordinate system 216 or the global coordinate system. This can provide superior performance compared to robotic medical systems that include only a separate EM field generator 202, such as... Figure 22A The robot system 200 has one or more advantages.

[0180] One advantage is that by attaching the EM field generator 302 to the robotic arm 212, the need for a separate registration step (which, as noted above, may require user input) can be eliminated. For example, for the robotic medical system 300, it is not necessary to manually perform a separate registration step such as referencing... Figure 22B and Figure 22CThe registration steps described are for registering the EM coordinate system 308 to the robot coordinate system 216 or the global coordinate system. In the robotic medical system 300, this registration can be easily performed automatically by attaching the EM field generator 302 to the robot arm 212 (e.g., it can be performed by the system). The robotic medical system 300 can then use the known kinematics of the robot arm 212 to correlate with the EM coordinate system 308 and the robot coordinate system 216 or the global coordinate system.

[0181] Another advantage achievable by connecting the EM field generator 302 to the robot arm 212 is that it improves the accuracy of registration between the EM coordinate system 308 and the robot coordinate system 216. As noted above, in situations such as Figure 22A In the robot system 200, the registration step may introduce inaccuracies, such as due to navigation inaccuracies during the manual registration step. In the robotic medical system 300, the accuracy of the registration between the EM coordinate system 308 and the robot coordinate system 216 is primarily determined by the kinematic accuracy of the robot arm 212. When these kinematics are known and well-defined, the registration between the EM coordinate system 308 and the robot coordinate system 216 can be highly accurate.

[0182] Another advantage offered by some embodiments, including the attachment of the EM field generator 302 to the robotic arm 212, is that the position of the EM field generator 302 can be adjusted using the robotic arm 212. This in itself provides one or more advantages. For example, the robotic arm 212 can be used to move or reposition the EM field generator 302. This can, for example, allow an operator to adjust the position of the working volume 306 of the magnetic field of the EM field generator 302. As described below, the robotic arm 212 can be used to move the EM field generator 302 such that the EM field generator tracks, for example, the motion of the instrument 214 as it moves within the body. Additionally, the EM field generator 302 can be moved or repositioned by the robotic arm 212 to facilitate access to the patient, for example, allowing a fluorescein C-arm to approach the patient. For example, the robotic arm 212 can be used to remove the EM field generator 302. Furthermore, when the robotic arm 212 moves the EM field generator 302, the relationship between the EM coordinate system 308 and the robot coordinate system 216, determined by the kinematics of the robotic arm 212, remains known. Therefore, after moving the EM field generator 302, it is not necessary to re-register the EM coordinate system 308 to the robot coordinate system 216.

[0183] Another advantage achievable by coupling the EM field generator 302 to the robot arm 212 may include increased accuracy in determining the position of the EM sensor 204, as the orientation and distance between the EM field generator 302 and the EM sensor 204 can be adjusted to improve accuracy. For example, the accuracy of the determined position of the EM sensor 204 may decrease when it is near the edge of the working volume 306. If the EM sensor 204 is determined to be closer to the edge of the working volume 306, the robot arm 212 can move the EM field generator 302 such that the EM sensor 204 is positioned closer to the center of the working volume 306 (or further away from the edge of the working volume 306), where the accuracy of the determined position can be increased. As another example, the robot arm 212 can move the EM field generator 302 closer to the EM sensor 204 to focus on the EM sensor and more closely track its position (referred to herein as EM focusing).

[0184] Furthermore, because the position of the EM field generator 302 can be easily adjusted by the robotic arm 212 without a separate registration step, a smaller EM field generator can be used compared to other robotic systems that typically use stationary EM field generators (such as robotic system 200). When using stationary EM field generators, they must be large enough to provide a sufficiently large working volume to cover the medical site. Even then, they may not be able to cover all parts of the patient's anatomy, such as the parts of the patient's anatomy that are navigated through to access the medical site. By coupling the EM field generator 302 to the robotic arm 212, a smaller EM field generator, which in some cases can have a smaller working volume, can be used, since the position of the working volume 306 can be adjusted by moving the EM field generator 302 using the robotic arm 202. In some embodiments, this allows the use of a compact field generator (cFG). However, the use of a compact field generator is not required, and in some embodiments, a larger field generator can be coupled to the robotic arm.

[0185] When coupled to the robot arm 212, the EM field generator 302 can function in a similar manner to the previously described EM field generator 202. For example, in Figure 23 In an exemplary embodiment of system 300, one or more devices 214 may include one or more EM sensors 204 located at their distal end and / or other portions of the device 214. Additionally, the exemplary embodiment includes a second EM sensor 204 attached to the patient's chest, which can be used to track motion caused by the patient's breathing. When the EM sensor 204 is positioned within the working volume 206 of the EM field generator 302, its position within the EM coordinate system 308 can be determined.

[0186] Furthermore, since the kinematics of the robot arm 212 can be used to register the EM coordinate system 308 to the robot coordinate system 216, the position of the EM sensor 204 within the robot coordinate system 216 or the global coordinate system can be determined. By physically connecting the EM field generator 302 to the robot arm 212, the EM coordinate system 308 and the robot coordinate system 216 or the global coordinate system can be linked together, making it possible to determine the position of the EM sensor 204 within the robot coordinate system 216 or the global coordinate system.

[0187] For example, in Figure 23 In the illustrated embodiment, the robotic medical system 300 may include an EM field generator 302 configured to generate an EM field. The robotic medical system 300 may include a robotic arm 212. The first robotic arm 212 may be coupled to the EM field generator 302 and configured to perform articulation to move the EM field generator 302. The robotic medical system 300 may also include one or more processors configured to determine the position of an EM sensor 204 within the EM field in an EM coordinate system 302 associated with the EM field generator 302. The processors may also be configured to determine a registration between an EM coordinate system 308 and a robot coordinate system 216 based on the determined position of the EM field generator 302 in the robot coordinate system associated with the first robotic arm 212. As described above, this can be achieved using the kinematics of the robotic arm 212. Based on this registration, the processor may be further configured to determine the position of the EM sensor 204 in the robot coordinate system 216.

[0188] For example, in Figure 23 In an illustrated embodiment, the robotic medical system 300 may include a controller or control circuitry including one or more processors configured to control the movement of one or more robotic arms 212. The robotic arms 212 may be configured to perform joint movements in response to commands received from the control circuitry. An EM field generator 302 may be coupled to the robotic arm 212 such that the EM field generator also moves in response to one or more commands from the control circuitry.

[0189] In addition to the robotic controllable EM field generator 302 connected to the robotic arm 212 Figure 23 The exemplary implementation of the robot system 300 may be similar in many respects to the above references. Figure 22A The description includes a robotic system 200 with a separate EM field generator 202, or other robotic medical systems described throughout this application or elsewhere. For example, in the illustrated embodiment, the robotic medical system 300 includes a cart 210 with two robotic arms 212. Although shown as having two robotic arms 212, the robotic system 300 may use other numbers of robotic arms 212. For example, Figure 2 The above description illustrates an exemplary trolley 11 including three robotic arms 12. Additionally, in some embodiments, the robotic arms 212 do not need to be attached to the trolley 210. In some embodiments, the robotic arms may be coupled to a bed or patient platform 215, for example, as... Figures 5 to 10 and Figure 14 As shown (as described above). In some embodiments, the robotic medical system 300 may include multiple robotic arms 212 (e.g., two, three, four, five, six or more robotic arms 212) coupled to the trolley 210, the patient platform and / or other components of the system.

[0190] Furthermore, although the robot system 300 in Figure 23 The diagram shows a single tool or instrument 214 (e.g., a laparoscopic or endoscopic instrument) having one of the robotic arms attached to a robotic arm 212, but it should be understood that other numbers of instruments 214 (e.g., two, three, four, five, six or more instruments, including laparoscopic instruments, endoscopic instruments and cameras) may be used in other embodiments.

[0191] Figure 24 An embodiment of the EM field generator 302 is shown. In the illustrated embodiment, the EM field generator 302 is configured to be coupled to a machine drive mechanism 402 that can be positioned on a robot arm, such that the EM field generator 302 can be attached to the robot arm 212. (Refer to the above...) Figure 16 The instrument drive mechanism, such as instrument drive mechanism 402, may be configured to have an interface 404 adapted to be attached to various tools or instruments for use by the robotic medical system. The EM field generator 302 may be configured to have a corresponding interface 320, which is configured to connect to interface 404 of the instrument drive mechanism 402. In this way, the EM field generator 302 can be connected to the robotic arm 212 in a similar manner to other robotic tools or instruments of the robotic system.

[0192] For example, the interface 320 of the EM field generator 302 may include one or more connectors 322 configured to be coupled to one or more corresponding connectors 406 on the interface 404 of the instrument drive mechanism 402. This arrangement allows the operator to easily and removably couple the EM field generator 302 to the instrument drive mechanism 402.

[0193] In the illustrated embodiment, the EM field generator 302 includes a housing 324. A proximal end 326 of the housing 324 may be configured to be coupled to an instrument drive mechanism 402. For example, the proximal end 326 may include an interface 320 and a connector 322. The housing 324 may extend from the proximal end 326 to a distal end 328. The distal end 328 may include an EM field generator unit 330 configured to generate a magnetic field. In the illustrated embodiment, the distal end 328 includes a clamp 332 configured to secure the EM field generator unit 330. Other mechanisms or methods for securing the EM field generator unit 330 are possible. In some embodiments, for example, as shown, the EM field generator unit 330 may be removable from the housing 324. In some embodiments, the EM field generator unit 330 is integrated into the housing 324. In the illustrated embodiment, the EM field generator unit 330 includes a cFG, although other types of field generators may also be used.

[0194] The size and shape of the housing 326 can be determined or selected such that the kinematic relationship between the EM field generator unit 330 and the robot arm to which the EM field generator 302 is attached is known, so that the kinematic registration between the EM coordinate system associated with the EM field generator and the robot coordinate system associated with the robot arm can be determined.

[0195] In the illustrated embodiment, the EM field generator unit 330 includes one or more connectors 334. The connectors 334 electrically connect the EM field generator unit 330 to a robot system, enabling the robot system to communicate with the EM field generator unit. In other embodiments, an electrical connection may be made between an interface 320 of the EM field generator 302 and an interface 404 of the machine drive mechanism 402.

[0196] Figure 25 An EM field generator 302 according to one embodiment is shown attached to an instrument drive mechanism 402. In the illustrated embodiment, the instrument drive mechanism 402 is shown covered with a sterile cover. A sterile adapter 410 is shown positioned over an interface 404 of the instrument drive mechanism 402. An interface 320 of the EM field generator 302 is attached to the sterile adapter 410, such that the EM field generator 302 is coupled to the instrument drive mechanism 402, with the sterile adapter 410 positioned therebetween. Figure 25 As shown, the EM field generator unit 330 can also be covered with a sterile cover.

[0197] Figure 26This is a flowchart depicting an embodiment of a method 500 for performing robotic medical procedures using an EM field generator configured to be coupled to (or otherwise connected to or integrated into) a robotic arm. In the illustrated embodiment, the method begins at block 502. Block 502 may include generating an EM field using an EM field generator coupled to a first robotic arm. The EM field may be associated with an EM coordinate system. Next, method 500 moves to block 504, where a registration between the EM coordinate system and a robot coordinate system associated with the first robotic arm is determined. This registration may be based on determining the position of the EM field generator within the robot coordinate system according to the kinematics of the first robotic arm. At block 506, the position of an EM sensor within the EM coordinate system is determined. At block 508, the method includes determining the position of the EM sensor in the robot coordinate system based on the registration determined at block 504.

[0198] In some implementations, method 500 may optionally include removably coupling an EM field generator to a first robotic arm, for example, as Figure 25 As shown. Method 500 may optionally include moving the EM field generator using a first robotic arm. Moving the EM field generator can be useful because the working volume of the EM field can be positioned as needed to facilitate the procedure, and / or the EM field generator can be moved away to allow access to the patient during the medical procedure. Moving the EM field generator using the first robotic arm to adjust the position of the EM field generator relative to the EM sensor can also improve the accuracy of the determined position of the EM sensor in the EM field coordinate system (and correspondingly by registration in the robot coordinate system).

[0199] Method 500 may further include a medical device mobilely coupled to a second robotic arm, wherein a robot coordinate system is also associated with the second robotic arm. An EM sensor may be positioned on the medical device such that navigation guidance of the second device can be determined at least in part based on the determined position of the EM sensor.

[0200] C. Exemplary Applications of Robotic Controllable Field Generators

[0201] In addition to providing one or more of the benefits described above, robotic controllable field generators, either attached to or otherwise integrated into robotic arms, can be used to enable, facilitate, and / or improve various functions during robotic medical procedures. This section outlines several exemplary applications of robotic controllable field generators that may be superior to other robotic systems that include, for example, static or stationary field generators.

[0202] i. Field generator setup and instrument tracking

[0203] As described above, robotic controllable field generators, which are attached to or integrated into a robotic arm, can be used to facilitate field generator setup and machine tracking. Several examples are provided in this section to illustrate, rather than limit, these concepts.

[0204] As used herein, “setting up” for the EM field generator refers to determining where to position the EM field generator to facilitate the procedure. This location may be based, for example, on the determined locations of one or more EM sensors. In some cases, the location may be adjusted during the procedure, for example, when one or more EM sensors are moved during the procedure. (See above reference...) Figure 21 and Figure 22A As indicated, the EM field generator 202 will be positioned relative to the patient such that the working volume 206 of the field generator 202 overlaps with a portion of the patient's anatomy where medical procedures are performed (e.g., a medical site). The EM field generator 202 may be supported by a support frame or other support structure such as a patient platform 215. Once positioned, registration steps (such as referencing) will be performed. Figure 22B and Figure 22C The described methods are used to align the EM coordinate system 208 associated with the EM field generator 202 to the robot coordinate system 216. Furthermore, the EM field generator 202 is typically not moved during the procedure, as movement would require manual intervention and repeating the alignment steps to re-align the EM coordinate system 208 associated with the EM field generator 202 to the robot coordinate system 216. A robotically controllable EM field generator that can be repositioned using the system's robot arm facilitates procedure setup by allowing the system to determine the locations to be positioned for the EM field generator and to move the EM field generator to those locations using the system's robot arm. Additionally, during the procedure, when the positions of tools and instruments, including the EM sensor, change, the system can adjust the position of the EM field generator to maintain it in a favorable location.

[0205] Figure 28A and Figure 28B An introductory example is shown, illustrating a robotic controlled EM field generator 302 (e.g., referenced above). Figures 23 to 25 The described tools and devices can be used to facilitate the setup and placement of various instruments and devices used in medical procedures. Additional examples will be described in more detail below. Figure 28A Three EM position sensors 204 and an EM field generator 302 coupled to the robotic arm 212 are shown. The EM position sensors 204 may be positioned, for example, on a medical device (such as an endoscopic, laparoscopic, and / or other type of surgical instrument) or directly on the patient (such as an EM patch sensor configured to track patient movement or breathing). For ease of illustration, the medical device is not shown.

[0206] like Figure 28AAs shown, the system can be configured to use a robotic arm 212 to move the EM field generator 302 to identify the position of the EM sensor 204. This movement may include translation, rotation, and / or oscillation of the EM field generator 302. Figure 28A An exemplary circular path is illustrated in the diagram. Once the position of the EM sensor 202 is determined, the robot arm 212 can move to reposition the EM field generator 302 to a relatively improved position (referred to herein as the "working position") or a centered position, where the EM sensor 204 is positioned within the working volume 306 of the EM field generator 302, such as, for example... Figure 28B As shown. This can be advantageous in setup because the system can determine the operating or central location of the EM field generator 302 to be automatically positioned. In related aspects, the positions of one or more EM sensors 204 can be adjusted based on the range of movement or oscillation achievable by the EM field generator, medical procedure steps, and / or the patient's anatomical characteristics to achieve a relatively improved arrangement of the EM sensors 204.

[0207] In some implementations, the system positions the EM sensor 204 and determines its location and / or orientation, then calculates the centroid or geometric center of these locations or the shape defined by one or more of these locations. The robotic arm 212 can then be used to position the EM field generator 302 such that the center of the working volume 306 is aligned with, for example, the centroid of the determined location of the EM sensor 204.

[0208] in addition, Figure 28A The EM field generator 302 is shown to increase the functional size of the working volume 306. For example, if the EM sensors 204 are separated so that they cannot all fit within the working volume 306, the system can (in a scanning motion) move the EM field generator to generate a functional working volume 306 large enough to track all the EM sensors 204. In some embodiments, the system may prioritize certain EM sensors 204 such that they are constantly positioned within the working volume 306, while other EM sensors 204 are sometimes within and sometimes outside the working volume 306.

[0209] As used herein, “instrument tracking” broadly refers to the use of a robotic arm to move a robotically controlled EM field generator to track or follow the movement of an EM sensor. As noted above, existing systems typically involve the use of stationary EM field generators that cannot (or do not) move to track or follow the movement of an EM sensor to avoid reregistration. In such systems, if the EM sensor is moved outside the working volume of the EM field generator, the EM sensor becomes undetectable until it is moved back into the working volume. A robotically controlled field generator that can be moved using a robotic arm can facilitate instrument tracking by allowing the EM field generator to be moved or otherwise adjusted to track the movement of one or more instrument sensors. Therefore, instead of moving the EM sensor outside the working volume of the EM field generator, the EM field generator can move with the EM sensor so that the EM sensor remains positioned within the working volume, thus facilitating continuous tracking of the EM sensor.

[0210] Figure 27A and Figure 27B An introductory embodiment of automated instrument tracking is shown, which can use a robotic controllable field generator such as the one described above. Figures 23 to 25 The described EM field generator 302 is used to facilitate this. Specifically, Figure 27A and Figure 27B An example of automated instrument tracking is shown during an exemplary medical procedure such as a ureteroscopy, during which a ureteroscope 502 is robotically guided through the patient's orifice, through the ureter, and into the kidney. Although a ureteroscopy procedure is shown, automated instrument tracking can be used for other types of procedures, including endoscopic and / or laparoscopic procedures. Figure 27A The procedure is shown at an earlier or first point in time / step (e.g., exactly after insertion of ureteroscope 502), and Figure 27B The procedure is shown at a later or second time / step (e.g., after the distal end of the ureteroscope 502 has been navigated into the kidney).

[0211] like Figure 27A and Figure 27B As shown, the ureteroscope 502 can be coupled to one or more robotic arms 212, which are configured to manipulate the ureteroscope 502 and perform its insertion, such as, for example, regarding Figure 3 As described. In Figure 27A and Figure 27BIn this configuration, an EM field generator 302 is attached to a third robotic arm 212, which is configured to move to adjust the position of the EM field generator 302 (and correspondingly, adjust the position of the working volume 306 of the magnetic field generated by the EM field generator 302). As described above, because the EM field generator 302 is attached to the robotic arm 212, the EM coordinate system associated with the EM field generator 302 can be automatically registered to the robot coordinate system associated with the robotic arm 212 by means of the kinematics of the robotic arm 212 to which the EM field generator 302 is attached.

[0212] like Figure 27A and Figure 27B As shown, the system can be configured to move the EM field generator 302 using a corresponding robotic arm 212, causing the EM field generator to automatically track the position of the ureteroscope 502. In the illustrated embodiment, the ureteroscope 502 includes an EM sensor 204 positioned at its distal end. As described above, when the EM sensor 204 is positioned within the working volume 306 of the EM field generator 302, the position of the EM sensor 204 can be determined relative to the EM coordinate system. Furthermore, since the EM coordinate system can be registered to the robot coordinate system, the position of the EM sensor 204 within the robot coordinate system is also determinable.

[0213] As the ureteroscope 502 is further inserted into the patient, the robotic arm 212 attached to the EM field generator can be moved or adjusted to automatically reposition the EM field generator, ensuring that the position sensor 204 remains positioned within the working volume 306 of the EM field generator. In some embodiments, this automatic tracking of the ureteroscope 502 is configured such that the EM field generator 302 is moved or adjusted so that the EM sensor 204 remains centered within the working volume 306, although this is not required in all embodiments. Additionally, in some embodiments, the orientation of the EM field generator 302 can be adjusted relative to the EM sensor 204 to provide optimal or improved accuracy in determining the position of the EM sensor.

[0214] This type of automated tracking advantageously allows for tracking of the ureteroscope 502's position throughout the procedure (e.g., from insertion to operation within the kidney). This is not possible in robotic systems using static or stationary field generators. As noted above, static or stationary field generators are typically configured such that their working volume is centered on the primary medical site (e.g., the kidney). In this location, the working volume may not extend sufficiently to provide coverage for insertion and navigation through the ureter. Therefore, in systems using static or stationary field generators, the operator may not be able to utilize EM-based navigation until they navigate the instrument into the field generator's working volume. Moving a static or stationary field generator to follow the instrument during insertion and navigation to the medical site is also generally not feasible, as each time the field generator is moved, a registration step would need to be re-performed, which is impractical and could lead to delays during the medical procedure, potentially negatively impacting the procedure's outcome.

[0215] However, as Figure 27A and Figure 27B As shown, the robotic controlled EM field generator 302 can be easily moved from insertion to the position of the ureteroscope 502 working inside the kidney because the robotic arm 212 can reposition the EM field generator 302 as needed, and the kinematics of the robotic arm 212 can be used to provide continuous registration between the EM coordinate system and the robot coordinate system.

[0216] Usable Figure 27A and Figure 27B An additional advantage of the automated instrument tracking shown is improved anatomical mapping. As the instrument is navigated through a patient's anatomical structures, the positions of the EM sensors on the instrument can be used to construct anatomical maps. For example, in the case of bronchoscopy, when a bronchoscope including EM sensors is navigated through a patient's airway, the determined positions of the EM sensors can be used to generate a map of the patient's lungs. This is possible only for areas within the working volume of a static or stationary EM field generator. However, by using a robotically controlled EM field generator 302 that allows the robotic arm 212 to reposition the EM field generator 302, the ability to generate anatomical maps is increased.

[0217] use Figure 27A and Figure 27B Examples can be found in the insertion ( Figure 27A ) to the kidney ( Figure 27BThe location of the EM sensor 204 is determined, thereby generating a generally complete anatomical map of the patient's urinary tract. Conversely, if a static field generator is used, its working volume is centered on the kidney, and EM sensor data that can be used to map the patient's ureter may not be available. Therefore, the use of a robotic, controllable EM field generator 302, as described herein, expands the ability to generate anatomical mappings using EM sensor data, resulting in a more complete anatomical map.

[0218] Figure 29 This indicates that the robotic medical system 300, such as Figure 22A The block diagram shown illustrates an example of a robotic medical system including a robotically controlled EM field generator 302 mounted on a robotic arm 212. (As shown...) Figure 29 (and also) Figure 23 As shown, system 300 may include one or more robotic arms 212. In the illustrated embodiment, system 300 includes a first robotic arm 212 and a second robotic arm 212, although in other embodiments, other numbers of robotic arms may be included (e.g., one, two, three, four, five, six, or more robotic arms). As shown, the first robotic arm 212 is coupled to an EM field generator 302. As described above, the EM field generator 302 may be attached or coupled to the first robotic arm 212, or it may be directly integrated into the first robotic arm 212 itself. The EM field generator 302 is configured to generate an EM field having a working volume 306, the position of the EM sensor 204 within which can be determined relative to an EM coordinate system 308. The first robotic arm 212 may be configured to adjust the position of the EM field generator 302. For example, the first robotic arm 212 may be movable (e.g., change posture or shape) to readjust the position of the EM field generator 302 and correspondingly readjust the position of the working volume 306 of the EM field.

[0219] The first robot arm 212 (and the other robot arms 212 in the system) is associated with the robot coordinate system 216, as described above. Based on the kinematic position of arm 212, the position of the EM field generator 302 within the robot coordinate system 216 can be determined. This allows for the determination of registration or mapping between the EM coordinate system 308 and the robot coordinate system 216, as described above. Figure 29 (and also) Figure 23 As shown, system 300 may include one or more EM sensors 204. The position of the EM sensor 204 may be determined relative to EM coordinate system 308, and the position may be mapped to robot coordinate system 216 using the previously described registration.

[0220] exist Figure 29 (as well as Figure 23In this embodiment, one of the EM sensors 204 is positioned on the medical device 214. In some implementations, the medical device 214 may be coupled to a second robotic arm 212. This allows the second robotic arm 212 to manipulate and control the medical device 214. When the second robotic arm 212 manipulates and controls the medical device 214, the position of the medical device 214 can be determined based on the determined position of the EM sensor 204. Figure 29 As shown, system 300 may include an additional EM sensor 204 that can also be tracked.

[0221] Figure 29 The block diagram further illustrates that system 300 may include processor 380 and memory 382. Memory 382 can be used to store instructions that, when executed by processor 380, enable various functions of system 300, such as determining and using registration and mapping from EM coordinate system 308 to robot coordinate system 216, setting up EM field generator 302, and tracking medical device 214, as described in this section and throughout this application. For example, processor 380 may communicate with first robot arm 212 and EM field generator 302 and is configured to determine the position of one or more EM sensors 204 within an EM field generated by EM field generator 302, and adjust the position of EM field generator 302 by commanding movement of first robot arm 212 based on the determined position of EM sensor 204. Movement of first robot arm 212 commanded based on the determined position of EM sensor 204 can be used to set the position of EM field generator 302 or to track the position of EM sensors 204 as they move during a procedure.

[0222] In some implementations, processor 302 is configured to first determine the position of EM sensor 204 relative to EM coordinate system 308, and then map that position to robot coordinate system 216. As described above, this can be achieved, for example, by: (i) determining the position of EM sensor 204 within an EM field relative to EM coordinate system 308 associated with EM field generator 302; (ii) determining a registration between EM coordinate system 308 and robot coordinate system 216 based on the determined position of EM field generator within robot coordinate system 216 associated with first robot arm 212; and (iii) determining the position of EM sensor 204 within robot coordinate system 216 based on this registration. The determination of the position of EM field generator 302 within robot coordinate system 216 at step (ii) can be based on the kinematics of first robot arm 212.

[0223] Figures 30A to 33DVarious examples or methods are shown of using a first robotic arm 204 to move an EM field generator 302 based on the determined position of the EM sensor 204 to facilitate field generator setup and instrument tracking.

[0224] Figure 30A and Figure 30B This is a perspective view illustrating an example of using a robotic arm to move an EM field generator so that an EM sensor is positioned at a predetermined location within the EM field. This can occur, for example, during field generator setup and can be advantageously used to position the EM field generator 302 relative to the EM sensor 204. In some embodiments, it can be advantageous to set or align the EM field generator 302 such that the EM sensor 204 is located at a predetermined location within the working volume 306 of the EM field. This can be useful for several reasons. For example, in some cases, this can center the working volume 306 of the EM field around the EM sensor 204. This helps ensure that the EM sensor 204 can still be tracked even if it is moved, as it can move without moving outside the working volume 306. Furthermore, this can increase the accuracy of the determined position of the EM sensor 204, because in some cases, the ability to determine the position of the EM sensor 204 within the EM field is more accurate toward the center of the working volume 306 than at the extremes of the working volume 306. Therefore, in some embodiments, the predetermined location includes the center of the working volume 306, and the EM field generator 302 is moved such that the EM sensor 204 is positioned at the center of the working volume 306, as shown below. Figure 30A and Figure 30B As shown.

[0225] exist Figure 30A and Figure 30B In the figure, the predetermined position 384 is the center of the working volume 306, and is represented by x in the figure. Figure 30A The system is shown in a first state before the EM field generator 302 is moved so that the EM sensor 204 is positioned at a predetermined location 384. For example... Figure 30A As shown, the EM sensor 204 is positioned within the working volume 306 of the EM field generator 302, but it is not located at the predetermined position 384. In order to position the EM sensor 204 at the predetermined position 384, the EM field generator 302 must be moved in the direction of the arrow shown, thereby causing a corresponding movement of the working volume 306 and the predetermined position 384. Figure 30BA system is shown in a second state after adjusting the position of the EM field generator 302 based on the determined position of the EM sensor 204 so that the EM sensor 204 is positioned at a predetermined position 384 within the EM field. In this example, the EM field generator 302 can now be considered "set up" because the position of the EM field generator has been adjusted relative to the EM sensor 204. It should be noted that the movement of the EM field generator 302 can be accomplished using the robot arm 212 to which the EM field generator 302 is attached. Furthermore, points in the EM coordinate system, such as the position of the EM sensor 204 and the predetermined position 384, can be mapped to the robot coordinate system using kinematic registration based on the robot arm, allowing all positions (e.g., the EM sensor position, the predetermined position 384, and the position of the EM field generator 302) to be processed in a single space (e.g., the robot coordinate system).

[0226] exist Figure 30A and Figure 30B In this embodiment, the predetermined location 384 within the working volume 306 of the EM field includes the center of the working volume 306. However, this is not required in all embodiments, and other predetermined locations 384 within the working volume 306 may be used. For example, another possible predetermined location 384 may include a position located at one extreme of the working volume 306, allowing the EM sensor 204 to move across the entire working volume 306 before requiring the EM field generator 302 to be repositioned to continue tracking the EM sensor 204.

[0227] In some implementations, the EM field generator 302 may be positioned such that the EM sensor 204 is positioned within a predetermined area or portion of the working volume 306 of the EM field, rather than at a specific predetermined location. Figure 31A and Figure 31B This is a perspective view illustrating an example of using a robotic arm 212 to move an EM field generator 302 so that an EM sensor 204 is positioned within a predetermined region 386 within a working volume 306. Figure 31A and Figure 31BThe figure shows a predetermined region 386 representing a sub-part of the working volume 386. In the figures, both the working volume 306 and the predetermined region 386 are represented as rectangular prisms. However, this is merely an example, and other shapes of the working volume and the predetermined region 386 are possible. Furthermore, the shape of the predetermined region 386 does not necessarily correspond to the shape of the working volume 306. For example, the working volume 306 may include the shape of a rectangular prism, and the predetermined region 386 may include the shape of a sphere within the working volume 306. In some embodiments, the predetermined region 386 may represent a portion of the working volume 306 within which the position of the EM sensor 204 can be determined with increased accuracy, and thus it may be desirable to position the EM field generator 302 relative to the EM sensor 204 such that the EM sensor 204 is within the predetermined region 386. In some embodiments, the predetermined region 386 may include the entire working volume 306.

[0228] Figure 31A The system is shown in a first state before the position of the EM field generator 302 is adjusted so that the EM sensor 204 is positioned within a predetermined region 386. As shown, the EM sensor 204 is positioned within the working volume 306 of the EM field, but it is not positioned within the predetermined region 386. To position the working volume 306 so that the EM sensor 204 is positioned within the predetermined region 386, the EM field generator 302 can be moved in the direction of the indicated arrow using a robot arm 212 attached to it. This movement causes a corresponding movement of the working volume 306 and the predetermined region 386. Figure 31B The system is shown in a second state after the position of the EM field generator 302 has been adjusted so that the EM sensor 204 is positioned within a predetermined region 386. As shown, the EM field generator 302 has been moved using the robot arm 212 so that the EM sensor 204 is now positioned within the predetermined region 386. As previously described, the above registration maps points in the EM coordinate system to corresponding points in the robot coordinate system based on the kinematics of the arm, allowing all positions to be represented in a single space.

[0229] Figures 31A to 31B and Figures 32A to 32BExamples involve adjusting the position of the EM field generator 302 relative to the EM sensor 204. In some embodiments, the system may additionally or alternatively determine the orientation of the EM sensor 204 within the EM field and adjust at least one of the orientation and position of the EM field generator 302 by commanding movement of the first robotic arm 212 based on the determined orientation of the EM sensor 204. In some cases, the accuracy of determining the position of the EM sensor 204 within the working volume 306 may be affected or influenced by the orientation of the EM sensor 204 relative to the EM field generator 302. Therefore, in some embodiments, the processor 380 is configured to adjust at least one of the orientation and position of the EM field generator based on the determined orientation of the EM sensor to increase the accuracy of the determined position of the EM sensor 204 within the EM field. This may involve, for example, adjusting the pitch, yaw, and / or roll of the EM field generator 302 relative to the EM sensor 204 using the robotic arm 212 to which the EM field generator 302 is attached.

[0230] As mentioned above, Figures 31A to 31B and Figures 32A to 32B Examples typically relate to determining the position of the EM field generator 302 relative to the determined position of the EM sensor 204 in order to set the EM field generator 302 for use in a procedure. This setup is facilitated by mounting the EM field generator 302 on the robotic arm 212 so that the position of the EM field generator can be robotically adjusted. Robotically adjusting the position of the EM field generator 302 also facilitates instrument tracking, which can occur when one or more of the EM sensors 204 move during the procedure. As previously noted, one or more of the EM sensors 204 can be positioned on a medical tool or instrument that moves during the procedure. The EM sensors 204 can allow tracking of the movement of these tools. As described in the following example, when movement of the EM sensor 204 is detected, a new position of the EM field generator 302 can be determined, and the EM field generator 302 can be moved using the robotic arm 212 to which the EM field generator is attached.

[0231] Figure 32A and Figure 32B This is a perspective view illustrating an example of moving an EM field generator 302 along a path 390 of a movement path 388 that tracks the EM sensor 204 using a robotic arm 212. In this way, the path 390 of the field generator 302 tracks, follows, or reflects the path 388 of the EM sensor 204. Figure 32A and Figure 32B In the example, the system is configured to maintain the position of the EM field generator 302 such that the EM sensor 204 remains positioned at a predetermined location 384 (in the illustrated example, the center of the working volume 306) as the EM sensor 204 is moved along path 388.

[0232] During medical procedures, this includes medical devices (such as EM sensors 204 positioned thereon) Figure 23 and Figure 29 The medical device 214 can be navigated through the patient's anatomy. This movement of the EM sensor 204 (and the corresponding device 214) is represented by path 388. Figure 32A The system shown depicts an EM sensor 204 positioned at a first location along path 388. If the EM sensor 204 continues along path 388 while the EM field generator 302 remains stationary, the EM sensor 204 will no longer be positioned at the predetermined location 384 within the working volume 306. To maintain the EM sensor 204 at the predetermined location 384 within the working volume 306, the EM field generator 302 must be moved along the corresponding path 390. In this way, the EM field generator 302 tracks or follows the movement of the EM sensor 204. Figure 32B The system is shown after the EM sensor 204 has moved along its path 388. Because the EM field generator 302 has also moved along its corresponding path 390, the EM sensor 204 remains positioned at a predetermined location 384 within the working volume 306. This is achieved by moving the EM field generator using the robotic arm 212 to which the EM field generator 302 is attached.

[0233] In some implementations, tracking the EM sensor 204 using the EM field generator 302 does not require a direct correspondence. For example, path 390 of the EM field generator 302 does not need to directly correspond to path 388 of the EM sensor 204. Figures 33A to 33D An example is shown.

[0234] Figures 33A to 33D This is a perspective view illustrating an example of using a robotic arm 212 to move an EM field generator 302 such that an EM sensor 204, moving along path 388, remains positioned within a predetermined region 386 of the working volume 306. As will be shown, in this example, the EM field generator 302 tracks the movement of the EM sensor 204, but does not move along the path 388 directly corresponding to the EM sensor 204. In this example, the system is configured to adjust the position of the EM field generator 302 when the EM sensor 204 moves to or outside the boundary of the predetermined region 386 of the working volume 306. For example, the processor 380 may be configured to command the movement of the EM field generator 302 using the first robotic arm 212 during movement of the medical device 214 including the EM sensor 204, such that the EM sensor 204 remains positioned within the predetermined region 386 of the working volume 306 of the EM field.

[0235] Figure 33AThe system in its first state is shown. As shown, the EM sensor 204 is positioned within a predetermined area 386 of the working volume 306 and travels along path 388. Figure 33B The system in its second state is shown. As shown, the EM sensor 204 has continued along path 204 but remains positioned within the predetermined area 386. Since the EM sensor 204 is still positioned within the predetermined area 386, the EM field generator 302 has not yet moved. Figure 33C The third state is shown, where the EM sensor 204 has reached the edge or boundary of the predetermined area 386. If the EM sensor 204 continues along path 388 while the EM field generator 302 remains stationary, the EM sensor 204 will move outside the predetermined area 386. To keep the EM sensor 204 within the predetermined area 386, the EM field generator 302 must be moved in the direction of the arrow shown. This movement can be determined, for example, by a processor 380, which can command the robotic arm 212 to move the EM field generator 302. Figure 33D The system is shown in its fourth state after the EM field generator 302 has been moved. As shown, this movement repositions the EM field generator 302 so that the EM sensor 204 is repositioned within the predetermined area 386, even as the EM sensor 204 continues along path 388. In this way, the EM field generator 302 can be moved again to track or follow the movement of the EM sensor 204, even if the movement of the EM field generator 302 is not as significant as before. Figure 32A and Figure 32B As in the example, it directly corresponds to the movement of the EM sensor 204.

[0236] Figure 34 This is a flowchart of an exemplary method 400 for moving an EM field generator coupled to a robotic arm based on a determined position of an EM sensor within an EM field. Method 400 may be executed, for example, by a processor 380 to provide field generator setup and device tracking. Method 400 begins at block 401, which relates to determining the position of an EM sensor of a medical device within the working volume of an EM field generated by an EM field generator coupled to a first robotic arm. The position of the EM sensor may be determined relative to an EM coordinate system associated with the EM field generator. In some embodiments, the position of the EM sensor may be further determined relative to a robot coordinate system associated with the first robotic arm by applying a registration determined based on the kinematics of the robotic arm as described above.

[0237] Method 400 includes block 403 in which a medical device is moved. Movement of the medical device causes a corresponding movement of an EM sensor positioned thereon. In some embodiments, the medical device includes a manually controlled device, and movement of the medical device is performed manually. In other embodiments, the medical device includes a robotically controllable device. For example, the medical device may be coupled to a second robotic arm, and moving the medical device may include moving the device using the second robotic arm. In some embodiments, moving the device using the second robotic arm includes performing articulations on the second robotic arm. The medical device may be coupled to a device drive mechanism, and moving the medical device using the second robotic arm may include actuating the medical device using the device drive mechanism. Movement of the device can be detected by determining that the position of the EM sensor attached to the device has moved.

[0238] Method 400 includes block 405, which relates to: in response to movement of a medical device, commanding a first robotic arm to move an EM field generator such that an EM sensor remains positioned within the working volume of the EM field generator. For example, if movement of the medical device would move the EM sensor outside the working volume, the first robotic arm can be used to reposition the EM field generator to maintain the EM sensor within the working volume of the EM field.

[0239] In some implementations, commanding the first robotic arm to move the EM field generator causes the EM field generator to track the movement of the medical device. For example, commanding the first robotic arm to move the EM field generator may include: moving the EM field generator such that the EM sensor is positioned or remains positioned within a predetermined area of ​​the working volume of the EM field, such as... Figures 31A to 31B and Figures 33A to 33D As shown in the example. For instance, commanding the first robotic arm to move the EM field generator could include: moving the EM field generator such that the EM sensor is positioned or remains positioned at a predetermined location within the working volume of the EM field, such as... Figures 30A to 30B and Figures 32A to 32B As shown in the example.

[0240] In some embodiments, method 400 may optionally include determining the orientation of the EM sensor within the EM field, and adjusting at least one of the orientation and position of the EM field generator by commanding movement of a first robotic arm based on the determined orientation of the EM sensor. As described above, adjusting at least one of the orientation and position of the EM field can increase the accuracy of the determined position of the EM sensor within the working volume of the EM field.

[0241] Alternatively, method 400 can be executed cyclically as indicated by dashed line 407 to continuously track the EM sensor and adjust the position of the EM field generator accordingly. That is, the position of the EM sensor can be determined at multiple discrete time steps, and the EM field generator can be readjusted based on each newly determined position.

[0242] The examples provided above have demonstrated that robotic systems such as Figure 23 and Figure 29 The illustrated system 300 can be configured to move an EM field generator 302 using a robotic arm 312 to set up the EM field generator 302 relative to an EM sensor 204 and / or track the movement of the EM sensor 204. However, the system 300 is not limited to an embodiment with only one EM sensor 204. For example, the processor 380 can be configured to determine the positions of multiple EM sensors 204 within an EM field, determine the generator position of the EM field generator 302 based on the determined positions of the multiple EM sensors 204, and command the first robotic arm 212 to move the EM field generator 302 to the generator position. That is, the system 300 can be configured for setting up and tracking an EM field generator based on multiple EM sensors 204. Regarding Figures 35A to 37 An example using multiple EM sensors 204 is provided.

[0243] Figure 35A and Figure 35B This is a perspective view illustrating an example of using a robotic arm 212 to move an EM field generator 302 to a field generator position based on the determined positions of multiple EM sensors 204 within an EM field. In some manner similar to the previous examples, this can occur during field generator setup and can be advantageously used to position the EM field generator 302 relative to the multiple EM sensors 204.

[0244] Figure 35A The system in a first state is shown. As shown, in the first state, multiple EM sensors 204 are positioned within the working volume 306 of the EM field generator 302. However, as shown, the multiple EM sensors 204 are not centered within the working volume 306. To center the multiple EM sensors 204 within the working volume 306, a field generator position 392 can be determined. The field generator position 392 can represent a position where the EM field generator 302 can be moved to center the multiple EM sensors 204 within the working volume. In the illustrated embodiment, the generator position 392 is denoted as x. The EM field generator 302 must be moved in the direction of the indicated arrow, thereby causing a corresponding movement of the working volume 306 to reach the generator position 392.

[0245] The generator position 392 can be determined based on the determined positions of the plurality of EM sensors 302. In one example, the processor 380 is configured to determine the generator position 392 based on the centroid of the determined positions of the plurality of EM sensors 204. The generator position 392 can then be determined by finding the position where the EM field generator 302 should be moved such that the centroid of the positions of the plurality of EM sensors 204 is located at a predetermined position (such as the center or other desired position) within the working volume 306. As previously described, the determined positions of the plurality of EM sensors can be mapped to the robot coordinate system using the previously described kinematic registration based on the robot arm 212 to which the EM field generator 302 is attached.

[0246] Figure 35B The system is shown in a second state after the position of the EM field generator 302 has been adjusted to generator position 392. As shown, the plurality of EM sensors 204 are now positioned closer to the center of the working volume 206. Therefore, the system can be advantageous for setting and positioning the EM field generator 302 relative to the plurality of EM sensors 204. Although this example has described "centering" the plurality of EM sensors 204 within the working volume 306, generator position 392 can be determined to position the plurality of EM sensors at other locations within the working volume 306.

[0247] Figure 36A , Figure 36B and Figure 36C This is a perspective view illustrating an example of using a robotic arm 212 to readjust the field generator position 392 of the EM field generator 302 based on the determined positions of multiple EM sensors 204 within the working volume 306, wherein at least one of the multiple EM sensors 204 is moving. For example, the processor 380 may be configured to: detect the movement of at least one of the EM sensors 204; determine a new generator position 392 of the EM field generator 302 based on the detected movement; and command the first robotic arm 212 to move the EM field generator 302 to the new generator position 392.

[0248] like Figure 36A As shown, one of the EM sensors 204 is moving along path 388. It may be desirable to adjust the position of field generator 302 based on this movement. For example, if not adjusted, EM sensor 204 may move along the path to a position outside the working volume 306, at which point tracking of EM sensor 204 will be lost.

[0249] Therefore, as Figure 36BAs shown, the new field generator position 392 can be determined based on the positions of multiple EM sensors 204 and taking into account the movement of one EM sensor 204 along path 388. The new field generator position 392 can be, for example, based on the current centroid of the positions of the multiple EM sensors 204. As shown, the EM field generator 302 will need to move in the direction of the illustrated arrow to reach the new generator position 392. Figure 36C The system is shown after being moved to the new field generator position 392. As shown, the positions of the multiple EM sensors 204 are now re-centered within the working volume 306. This advantageously allows for continuous adjustment of the position of the EM field generator 302 to optimize the position of the working volume 306, such that the positions of the multiple EM sensors 204 remain within the working volume 306 for as long as possible, even when they move away from each other.

[0250] Figure 37 This is a flowchart illustrating an exemplary method 410 for determining the location of an EM field generator mounted on a robot arm based on the determined locations of multiple EM sensors within an EM field. Method 410 begins at block 411, which relates to generating an EM field using an EM field generator coupled to a first robot arm.

[0251] At block 413 of method 410, the implementation can determine the positions of a plurality of EM sensors within an EM field. This can be achieved by: determining the positions of the plurality of EM sensors within the EM field relative to an EM coordinate system associated with an EM field generator; determining a registration between the EM coordinate system and the robot coordinate system based on the determined position of the EM field generator within a robot coordinate system associated with a first robot arm; and determining the positions of the plurality of EM sensors within the robot coordinate system based on this registration. The determination of the position of the EM field generator within the robot coordinate system can be based on the kinematics of the first robot arm.

[0252] At block 415, method 410 involves determining the generator position of an EM field generator based on the determined positions of the plurality of EM sensors. Determining the generator position may be based on the centroid of the determined positions of the plurality of EM sensors 204. The EM field generator position may be a location where the EM field generator can be moved to, which would desirously position the plurality of EM sensors within the working volume of the EM field.

[0253] Box 417 includes a command to the first robotic arm to move the EM field generator to the generator position. For example, the system can use the first robotic arm to move the EM field generator to the generator position. As the EM field generator moves, the position of the working volume is correspondingly repositioned relative to the position of the EM sensor.

[0254] As illustrated by dashed line 419, method 410 can be executed cyclically to continuously adjust the position of the EM field generator, as referenced above. Figures 36A to 36C For example, method 410 may further include: detecting movement of at least one of a plurality of EM sensors; determining a new generator position of the EM field generator based on the detected movement; and commanding a first robotic arm to move the EM field generator to the new generator position.

[0255] ii. Field generator setup and instrument tracking with extended working volume

[0256] In the preceding sections, robotic medical systems incorporating a robotically controllable field generator were discussed, which utilizes the movement of the system's robotic arm to facilitate field generator setup and instrument tracking. In some examples, the robotic arm is shown to move the EM field generator to track EM sensors, for example, to keep the moving EM sensor within the working volume of the EM field generated by the EM field generator. By moving the EM field generator and correspondingly moving the working volume of its EM field, the system described herein can be considered to have an "extended" working volume. That is, by repositioning the EM field generator and the working volume, the location of the EM sensor can be detected and determined within an area larger than the working volume of a stationary field generator.

[0257] This section provides additional examples detailing the concept of “expanding” the working volume of an EM field generator by moving it using a robotic arm. Specifically, among other things, these examples demonstrate how to track EM sensors that are too far apart to simultaneously engage within the working volume of the EM field generator by moving the EM field generator, detecting the position of the EM sensors, and mapping the detected positions of the EM sensors to the robot coordinate system. This provides a significant advantage over existing systems that utilize static field generators, which may “lose track” the EM sensors once they move outside the fixed working volume. Examples in this section refer to… Figure 23 and Figure 29 The exemplary system 300 can be used, but it can also be implemented using other systems that include a robotic controllable field generator.

[0258] Now refer to Figures 38A to 38C The first example is described with accompanying drawings showing the system 300 at different stages to illustrate how the system 300 expands the operating volume 306 of the EM field generator 302. (See attached figures.) Figure 38AAs shown, some components of a system in its first phase are illustrated. This system includes an EM field generator 302 configured to generate an EM field with a working volume 306. The position of an EM sensor 204 within this EM field can be determined relative to an EM coordinate system 308, as previously described. The EM field generator 302 is coupled to a robot arm 212 configured to move to adjust the position of the EM field generator 302. When the robot arm 212 moves the EM field generator 302, the position of the working volume 306 of the EM field generator 302 also moves.

[0259] Figure 38A Two EM sensors, 204A and 204B, are also shown. As shown, in the first stage, neither EM sensor 204A nor 204B is located within the working volume 306 of the field generator 302, and therefore the positions of EM sensors 204A and 204B are undetermined in the first stage. Furthermore, the two EM sensors 204A and 204B are positioned far enough apart that they cannot both be simultaneously located within the working volume 306. Therefore, as... Figure 38B and Figure 38C As shown, in order to track the positions of EM sensors 204A and 204B, the robot arm 212 can be used to move the EM field generator 302 to a location where each of the EM sensors 204A and 204B can be determined and mapped to a different position in the robot coordinate system 216 in order to “expand” the working volume 306 of the EM field generator.

[0260] For example, a processor 380 of system 300, communicating with EM field generator 302 and the robot arm 212 to which EM field generator 302 is attached, can be configured to move EM field generator 302 to a first position where EM sensor 204 is positioned within working volume 306. With EM field generator 302 in the first position, processor 380 can determine the position of the first EM sensor 204A within working volume 306 relative to EM coordinate system 308, and map the position of the first EM sensor 204A to robot coordinate system 216 based on the kinematics of the robot arm with EM field generator 302 in the first position. Processor 380 can then cause robot arm 212 to move EM field generator 302 to a second position where second EM sensor 204B is positioned within working volume. Furthermore, when the EM field generator 302 is in the second position, the processor 380 can determine the position of the second EM sensor 204B within the working volume 306 relative to the EM coordinate system 308, and map the position of the second EM sensor 204B to the robot coordinate system 216 based on the kinematics of the robot arm 212 when the EM field generator 302 is in the second position. In this way, the positions of the two EM sensors 204A and 204B can be represented within the robot coordinate system 216.

[0261] Figure 38B The system is shown in an exemplary second phase, in which the EM field generator 302 has been moved using the robotic arm 212 to a first position where the first EM sensor 204A is positioned within the working volume 306. With the EM field generator 302 in this position, the position of the EM sensor 204A can be determined relative to the EM coordinate system 308, and then this position can be mapped to the robot coordinate system 216 using the previously described kinematic registration based on the robotic arm 212. Figure 38B In the diagram, EM sensor 204A has been blacked out to indicate its position (although initially determined in EM coordinate system 308) and has been mapped to robot coordinate system 216.

[0262] Figure 38CThe system is shown in an exemplary third stage, in which the EM field generator 302 has been moved using the robotic arm 212 to a second position where the second EM sensor 204B is positioned within the working volume 306. With the EM field generator 302 in this position, the position of the EM sensor 204B can be determined relative to the EM coordinate system 308, and then this position can be mapped to the robot coordinate system 216 using the previously described kinematic registration based on the robotic arm 212. As previously mentioned, the EM sensor 204B has been blacked out to indicate that its position has been mapped to the robot coordinate system 216. The EM sensor 204A is also still blacked out to indicate its position within the robot coordinate system 216.

[0263] consider Figure 38C As can be seen, the positions of the two EM sensors 204A and 204B can be represented simultaneously or substantially concurrently in the robot coordinate system 216, even if the EM sensors 204A and 204B are far enough apart that they cannot simultaneously align within the working volume 306 of the EM field generator 302. In some embodiments, the processor 380 is further configured to move the EM field generator 302 back and forth between a first position and a second position to frequently update (e.g., at time steps of approximately 40 Hz or smaller and larger) and track the positions of the first EM sensor 204A and the second EM sensor 204B.

[0264] In addition, references have been presented. Figures 38A to 38C The example described simply illustrates the position in the first place ( Figure 38B ) and second position ( Figure 38C The system is configured to continuously or nearly continuously detect any EM sensor 204 located within the EM field and map those positions to the robot coordinate system 216. For example, the processor 380 may be configured to move the EM field generator 302 using the robot arm 212, and when an EM sensor is detected within the working volume 306, the processor 380 may be configured to: (i) determine the position of the EM sensor 204 within the working volume 306 relative to the EM coordinate system 308, and (ii) map the position of the EM sensor 204 to the robot coordinate system 216 based on the kinematic pose of the first robot arm 212.

[0265] In some cases, the initial position of the EM sensor 204 may be unknown. To locate the EM sensor 204, the system can be configured to use a robotic arm 212 to move the EM field generator 302 along a search path or trajectory (e.g., along a predetermined path / pattern or range of motion). The search path can be configured to sweep the working volume 306 across a treatment volume larger than the working volume 306 to locate the EM sensor within the treatment volume. By sweeping across the treatment volume, the initial position of any EM sensor 204 within the treatment volume can be determined.

[0266] Figures 39A to 39D An embodiment of a robotic medical system is shown, comprising a robotic controllable field generator 302 that utilizes a robotic arm 212 to move along a search path 510 to detect the position of an EM sensor 204. Figure 39A In the example, the initial positions of the two EM sensors 204A and 204B are unknown. To identify the positions of EM sensors 204A and 204B, Figure 39A An exemplary search path 510 is shown for using a robotic arm 212 to move an EM field generator 302. In the illustrated embodiment, the search path 510 includes a generally sinusoidal path configured to scan the working volume 306 across a treatment volume or treatment site in which the EM sensor 204 is intended. Figure 39A The illustrative search path 510 is provided by way of example, and it is possible to include any number of paths of different shapes.

[0267] Figure 39B The system is shown in its second stage as the EM field generator 302 is moved along search path 510. Figure 39B In the illustrated second stage, the first EM sensor 204A is now located within the working volume 306 of the EM field generator 302 and can be detected within this working volume. The position of the EM sensor 204A can be determined relative to the EM coordinate system 308, and then this position can be mapped to the robot coordinate system 216 using the previously described kinematic registration based on the robot arm 212. Figure 39B In this configuration, EM sensor 204A has been blacked out to indicate its position (although initially determined in EM coordinate system 308) and has been mapped to robot coordinate system 216. EM field generator 302 can continue along search path 510.

[0268] Figure 39C The system is shown in its third stage as it moves along search path 510 and even further, with the EM field generator 302 in motion. Figure 39CIn the illustrated third stage, the second EM sensor 204B is now positioned within the working volume 306 of the EM field generator 302 and can be detected within this working volume. The position of the EM sensor 204B can be determined relative to the EM coordinate system 308, and then this position can be mapped to the robot coordinate system 216 using the previously described kinematic registration based on the robot arm 212. Figure 39B In the diagram, EM sensor 204B has been blacked out to indicate that it has been mapped to robot coordinate system 216. EM field generator 302 can continue along search path 510. The position of EM sensor 204A is also shown as blacked out to indicate its position mapped to the robot coordinate system.

[0269] Figure 39D The system is shown in its fourth stage after the EM field generator 302 has completed search path 510. As shown, the positions of all EM sensors 204 (EM sensors 204A, 204B in the illustrated example) have been identified and mapped to robot coordinate system 216. In some embodiments, the system is configured to follow search path 510 at the start of the procedure to identify the positions of the EM sensors 204. Search path 510 may be repeated periodically to determine if any new EM sensors 204 have been introduced.

[0270] In some implementations, the initial position of the EM sensor 204 can be determined in other ways. For example, when the EM sensor 204 is included on a robotic controllable device, the system can estimate or determine the initial position of the EM sensor 204 based on the robot's kinematics. For EM sensors 204 that are not coupled to a robotic component of the system, in some implementations, a user can indicate or input the initial position of the EM sensor 204 to the system (e.g., via a user input device, controller, graphical user interface, etc.). For example, in the case of an EM patch sensor placed on a patient's chest to track breathing, the user placing the EM patch sensor can indicate to the system where the patch sensor has been placed.

[0271] As noted above, in some cases, the mapped position of the EM sensor 204 to the robot coordinate system 216 may not represent the actual position of the sensor. Therefore, after determining the initial position of the EM sensor, the system can determine a new tracking path along which the EM field generator 302 can move to continue tracking the position of the EM sensor 204. The tracking path may, for example, be shorter than the search path, which allows for faster or more frequent remapping of the EM sensor 204's position. (Reference) Figures 40A to 40D An example is described.

[0272] Figures 40A to 40D continue Figures 39A to 39DExamples are provided, and an exemplary tracking path 512 for the EM field generator 302 is further illustrated. The tracking path 512 may be determined based on the determined position of the EM sensor 204 within the robot coordinate system 216. For example, the tracking path 512 may be determined such that the EM field generator 302 moves back and forth between the determined positions of the EM sensor 204 within the robot coordinate system 216, allowing for frequent remapping and frequent position updates. For example, when the EM field generator 302 is moved along the tracking path using the robot arm 212, the processor 380 may be configured to: (i) redetermine the position of the EM sensor 204 within the working volume 306 relative to the EM coordinate system 308, (ii) remap the position of the EM sensor 204 into the robot coordinate system 216 based on the kinematic pose of the robot arm 212, and (iii) determine the updated tracking path 510.

[0273] like Figure 40A As shown, tracking path 512 is comparable ( Figures 39A to 39D The search path 510 is short. This can be advantageous because it limits unnecessary movement of the EM field generator (which could lead to collisions with other objects in space) and limits the time when no EM sensor is located within the working volume 306. In the illustrated embodiment, the tracking path 512 comprises a line that moves the EM field generator 302 back and forth between positions where each of the EM sensors 204A, 204B can be detected within the working volume 306 and remapped into the robot coordinate system 216. Other shapes of the tracking path 512 are also possible.

[0274] Figure 40A An EM field generator 302 is shown positioned at a first end of the tracking path 512, where a first EM sensor 204A is positioned within a working volume 306. In this position, the position of the first EM sensor 204A can be detected relative to the EM coordinate system 308 and remapped into the robot coordinate system 216. Figure 40B An EM field generator 302 is shown positioned at the second end of the tracking path 512, where a second EM sensor 204B is positioned within the working volume 306. In this location, the position of the second EM sensor 204B can be detected relative to the EM coordinate system 308 and remapped into the robot coordinate system 216. Figure 40A and Figure 40B In the diagram, EM sensors 204A and 204B are blacked out to indicate that their positions have been mapped into robot coordinate system 216 (e.g., when detected while moving EM field generator 302 along search path 510, such as...). Figures 39A to 39D (as described in the previous example).

[0275] Figure 40C This illustrates how the mapped position of the EM sensor 204A to the robot coordinate system may not be real. For example, as... Figure 40C As shown, when the EM field generator 302 is positioned at the second end of the tracking path 512 such that the first EM sensor 204A is not positioned within the working volume 306, the first EM sensor 204A may be moved to a new position indicated as EM sensor 204A'. It should be noted that the previous mapped position of the EM sensor 204A is still shown (blacked out). In this position, the system has not yet determined that the EM sensor 204A has moved to the new position of EM sensor 204A'. In some embodiments, each sensor 204 is connected to a dedicated port of the system, allowing the system to identify and differentiate between sensors. Other mechanisms and methods for differentiating between sensors are also possible.

[0276] However, Figure 40D This illustrates that as the EM field generator 302 moves back along the tracking path 512, it can detect the new position of the EM sensor 204A'. In this position, the new position of the first EM sensor 204A' can be detected relative to the EM coordinate system 308 and remapped into the robot coordinate system 216. To indicate that the new position of the EM sensor 204A' has been remapped into the robot coordinate system 216, this new position is... Figure 40D The middle part has been blacked out.

[0277] Figure 40D It is also shown that a new tracking path 512' can be determined based on the new position of EM sensor 204A'. Therefore, when EM sensor 204 is moved, the new position of EM sensor 204 can be remapped into robot coordinate system 216, and the updated tracking path 512' can be determined. This process can be repeated throughout the procedure to advantageously allow tracking of individual EM sensors 204 within an extended working volume, which is made possible by utilizing robot arm 212 to move EM field generator 302 and mapping the detected positions of EM sensors 204 into the robot coordinate system based on the kinematic pose of robot arm 212.

[0278] Figure 41A This is a flowchart illustrating an exemplary method 600 for expanding the working volume of a robotic controllable field generator. Method 600 can be used, for example, to implement the above-referenced... Figures 38A to 38CThe function described above. Method 600 begins at block 601, where the EM field generator is moved to a first position. The first position may be a location where the EM sensor is detected within the working volume of the EM field generator. The EM field generator can be moved using a robotic arm to which it is attached. In some embodiments, moving the EM field generator to the first position includes moving the EM field generator along a search or tracking path.

[0279] The method proceeds to box 602. At box 602, the position of the first EM sensor is determined relative to the EM coordinate system. The EM coordinate system is associated with the EM field of the EM field generator. Next, at box 603, the position of the EM sensor in the EM coordinate system (determined at box 602) is mapped to the robot coordinate system. The robot coordinate system is associated with, for example, the robot arm to which the EM field generator is attached, and other robot components of the system. Mapping the position of the EM sensor to the robot coordinate system can be based on the kinematic pose of the first robot arm, as previously noted, which establishes the relationship between the EM coordinate system and the robot coordinate system because the EM field generator is attached to the robot arm.

[0280] Then, method 600 can proceed to block 604. At block 604, the EM field generator is moved to a second position where the second EM sensor is detected within the working volume. As previously described, the EM field generator can be moved to the second position using the robot arm to which it is attached. At block 605, the position of the second EM sensor can be determined relative to the EM coordinate system. Next, at block 606, the position of the second EM sensor in the EM coordinate system (determined at block 605) is mapped to the robot coordinate system again using robot arm-based kinematic registration. Finally, line 607 shows that method 600 can be executed as a loop to continuously re-detect the positions of the first and second EM sensors and remap them to the robot coordinate system.

[0281] Figure 41B This is a flowchart illustrating another exemplary method 610 for expanding the working volume of a robotic controllable field generator. Method 610 begins at block 611, which relates to moving the EM field generator using a robotic arm attached to it. At block 612, whenever an EM sensor is detected within the working volume of the EM field generator, the position of the EM sensor can be determined relative to the EM coordinate system. At block 613, the position of the EM sensor can be mapped to the robot coordinate system. This mapping can be based on the kinematics of the robotic arm, as previously described. Finally, line 614 shows that method 610 can be performed as a loop to continuously re-detect the positions of the first and second EM sensors and remap them to the robot coordinate system.

[0282] iii. Alignment of percutaneous instruments with the EM target

[0283] The robotic, controllable EM field generator 302 can also facilitate the alignment of percutaneously inserted tools with other tools inserted endoscopically into the body. For example, such as Figure 27B As shown, the ureteroscope 502 can be guided endoscopically to a position within the kidney. The operator may then wish to perform percutaneous insertion of a needle (or other instrument) or use a needle (or other instrument) to meet the ureteroscope 502. In some embodiments, the EM field generator 302 may include a needle guide (e.g., a channel, tube, or other structure) through which the needle (or other instrument) can be inserted. In other embodiments, the EM field generator 302 may define a space through which the needle guide, etc., can be advanced and / or positioned. For example, the needle guide may be configured such that it allows only one degree of freedom (insertion and / or retraction) of the needle through which it is inserted. The EM field generator 302 may be aligned with the EM sensor 204 on the ureteroscope such that when the needle is inserted through the needle guide, the needle is guided directly toward the EM sensor 204. One such use case is in percutaneous procedures where internal organs or other tissues are accessed via needle puncture of the skin. A needle, rigidly attached to a robotic end effector and having a known conversion (rigid body definition) with respect to the robotic end effector, can thus be robotically aligned with an EM beacon (target) inside the patient's body. The needle can then be inserted through the skin along the robotically aligned trajectory.

[0284] Figures 42A to 42C An implementation scheme of a robotic medical system with a robotic controllable field generator 302 is shown, such as... Figure 23 and Figure 29 The system 300, a robotic controllable field generator, is configured to facilitate the alignment of a percutaneously insertable instrument 214 with an EM target 204A. The percutaneously insertable instrument 214 may include, for example, needles, access sheaths, laparoscopic instruments, or other types of percutaneously insertable instruments. Figures 42A to 42C In one example, it might be desirable to precisely insert the percutaneously insertable device 214 at a specific location within the body. To facilitate percutaneous insertion, an EM target 204A (which may be an EM sensor) can be placed at the desired location of the percutaneously insertable device 214 within the body. In some embodiments, the percutaneously insertable device 214 may also include an EM sensor 204B positioned thereon.

[0285] exist Figure 42AIn the illustrated embodiment, the percutaneously insertable instrument 214 may be a robotic, controllable instrument. The percutaneously insertable instrument 214 is shown attached to a second robotic arm 212B configured to position and insert the percutaneously insertable instrument 214. As shown, the percutaneously insertable instrument 214 extends along axis 502. Typically, it is desirable to insert the percutaneously insertable instrument 214 along axis 502. However, for precise insertion of the percutaneously insertable instrument 214, axis 502 must be aligned with EM target 204A.

[0286] Therefore, the system processor 380 can be configured to determine a registration based on the kinematic pose of the first robotic arm 212A attached to the EM field generator 302, which maps the position in the EM coordinate system associated with the EM field generator 302 to a position in the robot coordinate system 216. The processor 380 can be further configured to determine the position of the EM target 204A in the robot coordinate system 216 based on the registration, and based on the position of the EM target 204A in the robot coordinate system 216, the processor can be configured to move the second robotic arm 212B to align the axis 502 of the percutaneously insertable instrument 214 with the EM target. Finally, the processor 380 can be configured to use the second robotic arm 212B to insert the percutaneously insertable instrument 214 along the axis 502 toward the EM target 204A.

[0287] As previously described, registration can be determined based on the position of the EM field generator 302 in the robot coordinate system 216, and the position of the EM field generator in the robot coordinate system 216 can be determined based on the kinematic pose of the first robot arm 212A. The processor 380 can be further configured to determine the position of the EM target 204A in the robot coordinate system 216 by determining the position of the EM target 204A in the EM coordinate system 308 and using registration to map the position of the EM target 204A in the EM coordinate system 308 to the position of the EM target 204A in the robot coordinate system 216.

[0288] In some implementations, the EM target 204A may be an EM sensor positioned on another robotic medical device (such as an endoscope navigated through the body) configured for insertion into a patient. This robotic medical device may be coupled to a third robotic arm configured to control the robotic medical device.

[0289] Figure 42A The system is shown in a first state in which the axis 502 of the percutaneously insertable instrument 214 is not aligned with the EM target 204A. Figure 42B The system is shown in its second state after axis 502 is aligned with EM target 204A. Figure 42CThe system in the third state is shown, with a percutaneously insertable instrument 214 inserted along axis 502 and meeting with EM target 204A.

[0290] Figure 43A and Figure 43B An implementation scheme of a robotic medical system with a robotic controllable field generator 302 is shown, such as... Figure 23 and Figure 29 The system 300, a robotic controllable field generator, is configured to facilitate the alignment of an instrument guide 520 mounted on the field generator 302 with an EM target 204A. The instrument guide 520 is configured such that a percutaneously insertable instrument can be inserted along an axis 522 during insertion into a patient. The instrument guide 520 can restrict movement of the percutaneously insertable instrument to movement only along the axis 522, such that when the axis 522 is aligned with the EM target 204, the percutaneously insertable instrument can be guided toward the EM target 204. In some embodiments, the instrument guide 520 is removably coupled to the EM field generator 302. In other embodiments, the instrument guide 520 may be permanently coupled to or integrated into the EM field generator 302. In a related aspect, it should be noted that the instrument guide 520 may be configured to have a... Figure 43A and Figure 43B Examples of different shapes / curves and / or lengths.

[0291] Similar to the previously described example, the axis 522 of the instrument guide 520 can be aligned with the EM target 204 to orient the percutaneously insertable instrument guide toward the EM target 20A. For example, the processor 380 can be configured to determine a registration based on the kinematic pose of the first robotic arm 212, which maps the position within the EM coordinate system 308 associated with the EM field generator 308 to a position within the robot coordinate system 216. The processor 380 can also determine the position of the EM target 204 within the robot coordinate system 216 based on the registration, and based on the position of the EM target 204 within the robot coordinate system 216, the processor 380 can move the first robotic arm 212 to align the insertion axis 522 of the instrument guide 520 with the EM target 204.

[0292] Figure 43A The system is shown in a first state in which the axis 522 of the instrument guide 520 is not aligned with the EM target 204. Figure 43B The system is shown in a second state after axis 522 is aligned with EM target 204. In the second state, a percutaneously insertable instrument can be inserted through instrument guide 520 along axis 522 toward EM target 204.

[0293] The system may also include a percutaneously insertable instrument. The percutaneously insertable instrument may include needles, access sheaths, laparoscopic instruments, or another type of percutaneously insertable instrument. The percutaneously insertable instrument may extend along an axis. In some embodiments, the system includes a second robotic arm coupled to and configured to move the percutaneously insertable instrument. In such cases, the processor may be further configured to use the second robotic arm to align the axis of the percutaneously insertable instrument with the insertion axis, and to use the second robotic arm to insert the percutaneously insertable instrument through an instrument guide along the insertion axis toward the EM target 204. In some embodiments, the percutaneously insertable instrument may include an instrument manually inserted through the instrument guide 520.

[0294] In some embodiments, EM target 204 includes an EM sensor positioned on a medical device configured for insertion into a patient. The robotic medical device may be an endoscope. The system may include a third robotic arm coupled to and configured to control the robotic medical device.

[0295] Figure 44A and Figure 44B An embodiment of a robotic medical system with a robotic controllable field generator 302 is shown, which is configured to facilitate the alignment of the instrument guide 530 with the EM target 204. This example is similar to... Figure 43A and Figure 43B The example differs in that the instrument guide 530 is attached to the second robotic arm 212B instead of the EM field generator 302. Similarly, the system processor 380 can be configured to determine a registration based on the kinematic pose of the first robotic arm 212A to which the EM field generator 302 is attached. This registration maps the position within the EM coordinate system 308 associated with the EM field generator 302 to a position within the robot coordinate system 216. The processor 380 can further determine the position of the EM target 204 within the robot coordinate system based on the registration, and, based on the position of the EM target within the robot coordinate system, move the second robotic arm 212B to align the insertion axis 532 of the instrument guide with the EM target 204. Figure 44A The system is shown before the axis 532 of the instrument guide 530 is aligned with the EM target 204, and Figure 44B The system after alignment is shown. Figure 44B In the illustrated position, a percutaneously insertable instrument can be inserted through the instrument guide 530 toward the EM target 204.

[0296] In some embodiments, the percutaneously insertable instrument is coupled to a third robotic arm configured to move the percutaneously insertable instrument. In such cases, the processor may be further configured to use the third robotic arm to align the axis of the percutaneously insertable instrument with the insertion axis 532 of the instrument guide 530, and to use the third robotic arm to insert the percutaneously insertable instrument through the instrument guide 530 along the insertion axis 532 toward the EM target 204. In other embodiments, the percutaneously insertable instrument may be manually inserted through the instrument guide 530.

[0297] The system may also include a robotic medical device configured for insertion into a patient. EM target 204 may include EM sensors on the robotic medical device. The robotic medical device may be coupled to another robotic arm configured to control the robotic medical device.

[0298] Figure 45A This is a flowchart illustrating a method 700 for aligning a percutaneously insertable instrument with an EM target using a robotic controllable field generator. Method 700 begins at box 701, where a registration is performed to map the position in the EM coordinate system to a position in the robot coordinate system. This registration can be determined based on the kinematic pose of the robotic arm to which the EM field generator is attached, as previously described.

[0299] Next, at box 702, the position of the EM target in the robot coordinate system can be determined based on registration. As previously described, this may involve determining the position of the EM target within the EM field generated by the EM field generator and mapping that position to the robot coordinate system.

[0300] At frame 703, based on the determined position of the EM target, the axis of the percutaneously insertable instrument is aligned with the EM target. (As described above...) Figure 42B An example is shown. Finally, at box 704, a percutaneously insertable instrument can be inserted along the axis toward the EM target. (As previously described...) Figure 42C An example is shown in the image.

[0301] Figure 45B This is a flowchart illustrating method 710 for aligning an instrument guide for a percutaneously insertable instrument with an EM target using a robotic controllable field generator. At block 711, method 700 involves determining a registration that maps the position in the EM coordinate system to the position in the robot coordinate system. At block 712, the position of the EM target in the robot coordinate system is determined based on the registration. Finally, at block 713, method 700 involves aligning the axis of the instrument guide with the EM target. The above has been illustrated with reference to the instrument guide on the EM field generator. Figure 43A and Figure 43B And a device guide mounted on the second robotic arm is shown. Figure 44A and Figure 44B An example is described.

[0302] iv. Distortion detection

[0303] The robotic controlled EM field generator 302 can also be used for distortion detection. For example, when a static EM sensor is present in a region of interest (e.g., a needle insertion site, biopsy site, or any static location within the working volume of the EM field generator), the robotic controlled EM field generator can be commanded to move. The EM field generator 302 can be moved by its attached robotic arm 212, and the commanded robot motion (in the robot coordinate system) can be compared with a recorded EM sensor trajectory (in the EM coordinate system). The difference between the two trajectories can be a measure or indicator of distortion in the EM signal within the working volume of the EM field generator 302. A similar principle can be used for non-static EM sensors where the motion is known or has finite uncertainty. The accuracy of distortion detection may depend on or be constrained by the uncertainty of the knowledge of the EM sensor's motion / position.

[0304] Figure 46A An example is shown of an embodiment of a robotic medical system in which an EM field generator 302 is moved relative to a fixed EM position sensor 204 to detect EM distortion. As illustrated in the embodiment, the EM field generator 302 may be coupled to a first robotic arm 212. The EM field generator 302 may be configured to generate an EM field and associated with an EM coordinate system in which the position of an EM sensor, such as the illustrated EM sensor 204, can be determined. The first robotic arm 212 may be configured to move to reposition the EM field generator 302. The first robotic arm 212 may be associated with a robot coordinate system. As described above, the motion of the arm in the robot coordinate system is known due to the known kinematics of the robotic arm 212. Therefore, since the EM field generator 302 is coupled to the first robotic arm 212, the position of the EM field generator 302 in the robot coordinate system 212 can also be determined based on the kinematic pose of the arm. As described in more detail above, this relationship can establish a registration that can be used to determine the position of the EM sensor 204 in the EM field within the EM coordinate system relative to the robot coordinate system.

[0305] exist Figure 46A In the illustrated embodiments, to determine or detect EM distortion within an EM field, an EM sensor 204 may be provided at a fixed and stationary location. For example, the EM sensor 204 may be attached to a non-moving object, such as a patient platform or other non-moving device. In some embodiments, the EM sensor 204 is positioned on a medical device, such as a robotic controllable medical device. If this is the case, then as referenced... Figures 46A to 47During the described distortion detection steps, the medical device can remain stationary, so that the position of the EM sensor 204 remains fixed and stationary.

[0306] To detect EM distortion, a processor communicating with the first robotic arm 212 (and, by extension, the EM field generator 302) can be configured to cause the first robotic arm 212 to move the EM field generator 302 along a robot trajectory 850. Figure 46A In the diagram, robot trajectory 850 is represented by a dashed line, and the system is shown as being positioned approximately midway along robot trajectory 850, with the EM field generator 302 positioned approximately midway along the path. In some embodiments, the movement of the EM field generator 302 by the first robotic arm 212 along robot trajectory 850 can be accomplished under the guidance or control of an operator such as a physician, who can command the movement using a controller or other user interface. In other embodiments, the movement of the EM field generator 302 by the first robotic arm 212 along robot trajectory 850 can be provided automatically, for example, as part of an automated EM distortion detection process operated by the system.

[0307] exist Figure 46A It is important to recognize that, in this example, the EM field generator 302 moves along the robot trajectory 850, and this occurs while the EM sensor 204 remains stationary or fixed, as described above. Figure 46A The EM sensor trajectory 852 is also shown, represented as a dashed line in this figure. However, the EM sensor trajectory 852 is not the result of physical movement of the EM sensor 204, as the EM sensor remains stationary, as noted above. Instead, the EM sensor trajectory 852 is generated due to the movement of the EM field generator 302 relative to the stationary EM sensor 204. For example, as the first robotic arm 212 moves the EM field generator 302 along the robot trajectory 852, the position of the EM sensor in the EM coordinate system associated with the EM field generator 302 is recorded, thereby generating the EM trajectory 850.

[0308] Therefore, the robot trajectory 850 is caused by the movement of the EM field generator 302 by the first robot arm 212. The robot trajectory 850 can be detected, determined, and / or recorded based on the known kinematic movements of the first robot arm 212 during its movement. In contrast, the EM sensor trajectory 852 is caused by the movement of the EM field generator 302 relative to the stationary EM sensor 204. When the EM field generator 302 moves and the EM sensor 204 remains stationary, the EM sensor trajectory 852 can be detected, determined, and / or recorded based on the detection of the EM sensor 204 within the EM field of the EM field generator 302.

[0309] Because the EM sensor 204 remains stationary during the movement of the EM field generator 302 by the first robot arm 212, the robot trajectory 850 is expected to correspond to the EM sensor trajectory 852 in the absence of any EM distortion. This is because the movement of the EM field generator 302 should match the relative motion between the moving EM field generator 302 and the static EM sensor 204. Therefore, the difference between the robot trajectory 850 and the EM sensor trajectory 852 can be analyzed to determine whether EM distortion exists. In some embodiments, the analysis may also provide a measure of the degree or severity of the EM distortion.

[0310] Figure 46B A comparison of robot trajectory 850 and EM sensor trajectory 852 is shown. In this example, the starting point of each trajectory has been aligned, although this is not required in all embodiments. Figure 46B As shown, it is evident that the robot trajectory 850 and the EM sensor trajectory 852 do not correspond perfectly. The differences between the trajectories can be determined to detect EM distortion. In some embodiments, the detected EM distortion indicates the EM distortion at the location of the fixed EM sensor 204.

[0311] Not all differences between the robot trajectory 850 and the EM sensor trajectory 852 are necessarily caused by EM distortion. For example, due to various reasons including manufacturing tolerances, motor control limitations, and sensor accuracy, the robot system may not be able to determine the robot motion perfectly accurately. Therefore, some differences between the robot trajectory 850 and the EM sensor trajectory 852 may be caused by inaccuracies in determining the robot motion. However, in general, the inaccuracies in determining the robot motion may be small, for example, in the sub-millimeter range. Similarly, the EM field generator 302 may suffer from EM noise, which may cause a reduction in the resolution of the EM sensor position determined within the field. This may lead to inaccurate recording of the EM sensor trajectory 852, resulting in differences that are not directly attributable to EM distortion. However, again, the EM field generator may be configured or selected such that the EM sensor noise is approximately 1 mm or less in some embodiments. Therefore, for some embodiments of the robot system, the inaccuracies in determining the robot motion, along with the inaccuracies in recording the EM sensor position within the EM field, can lead to potential inaccuracies of approximately 1 mm or less.

[0312] Therefore, to determine the presence of EM distortion, it is beneficial to analyze the difference between the robot trajectory 850 and the EM sensor trajectory 852 relative to a threshold selected to exclude or reduce contributions due to other factors. In some embodiments, the threshold may be, for example, 1 mm, 1.25 mm, 1.5 mm, 1.75 mm, 2 mm, 2.25 mm, 2.5 mm, 2.75 mm, 3 mm, or greater. In many cases, it is beneficial to set the threshold such that it is greater than one or both of the error factor associated with the motion of the first robot arm or the error factor associated with the noise of the EM sensor or generator.

[0313] Comparing robot trajectory 850 and EM sensor trajectory 852 can be achieved in several ways. For example, as... Figure 46B As shown, the shapes of the robot trajectory 850 and the EM sensor trajectory 852 can be directly compared. The shapes can be analyzed to determine how closely they correspond, and if the deviation between the shapes exceeds a threshold, the system can determine the presence of EM distortion and / or the degree of EM distortion. In another embodiment, analyzing the robot trajectory 850 and the EM sensor trajectory 852 to determine the differences between them may include comparing multiple points along the robot trajectory 850 with corresponding multiple points along the EM sensor trajectory 852. In some embodiments, the multiple points are determined relative to the time associated with moving the EM field generator 302 using the first robot arm 212. For example, a point on the robot trajectory 850 at the start of movement (e.g., at t=0) can be compared with a corresponding point on the EM sensor trajectory 852. Subsequent points can then be compared at subsequent discrete time steps.

[0314] In some implementations, if EM distortion is detected, the system can be configured to use the first robotic arm 212 to reposition the EM field generator 302 at a location that reduces EM distortion. For example, in some implementations, the system can position the EM field generator 302 at a location along the robot trajectory 850 where the difference between the robot trajectory 850 and the EM sensor trajectory 852 is reduced.

[0315] In some implementations, this can be achieved by moving the EM field generator 302 along a second robot trajectory to attempt to identify a location where EM distortion can be reduced. Therefore, the EM field generator 302 can be moved along a second robot trajectory different from the first robot trajectory 850. For example, the EM field generator 302 can be moved along the second robot trajectory through a space different from the space covered by the first robot trajectory 850. As previously described, this is done while the EM sensor 204 remains stationary. The second robot trajectory can be recorded along with a second, corresponding EM sensor trajectory. These trajectories can be analyzed to determine if a location for reducing EM distortion can be determined. If it can be determined, the EM field generator 302 can be moved to that location using the robot arm 212. If it cannot be determined, additional robot trajectories can be attempted until a suitable location for reducing EM distortion with the EM field generator 302 can be found.

[0316] In the above about Figure 46A and Figure 46B In the described example, robot trajectory 850 is defined relative to the global robot reference frame, while EM sensor trajectory 852 is defined relative to the EM field generator coordinate system. In this case, to compare robot trajectory 850 with EM sensor trajectory 852, the EM field generator coordinate system must be registered to the global robot reference frame. This can be achieved, for example, as described above, because the EM field generator 302 is attached to the robot arm 212, thereby establishing a kinematic relationship between the global robot reference frame and the EM field generator coordinate system.

[0317] Furthermore, for ease of understanding and explanation, the above regarding Figure 46A and Figure 46B The example described is slightly simplified, assuming that only the position of the EM field generator 302 relative to the EM sensor 204 has changed, while its relative orientation remains unchanged. In a more complex example, both the position and orientation of the EM field generator 302 relative to the EM sensor 204 can change, and the detected position (and / or orientation) of the EM sensor 204 can be analyzed to determine whether EM distortion exists (and / or to determine its degree). In such cases, EM distortion can be determined relative to a global reference frame 854 associated with the trolley to which the robot arm 212 is attached, such as... Figure 46C As shown in the figure, the robot arm 212 to which the EM field generator 302 is attached moves along the robot trajectory 850, which alters the position and orientation of the EM field generator 302 relative to the EM sensor 204. Because the cart and the EM sensor 204 do not move, any detected changes in position and / or orientation measured relative to the global reference frame 854 will be due to distortion (and negligible kinematic errors).

[0318] EM distortion can be caused by any or all components located within the working volume of the EM field generator. Therefore, a similar process can be performed while moving other components to attempt to find their optimal or preferred positions.

[0319] Figure 47 This is a flowchart illustrating an exemplary method 800 for EM distortion detection. Method 800 is similar in many respects to the above description. Figures 46A to 46C The process described herein. Method 800 can be performed using a robotic medical system such as the previously described system, in which the EM field is typically robotically controllable via a robotic arm attached to the system.

[0320] Method 800 begins at block 801. Block 801 may involve moving an EM field generator coupled to a first robotic arm along a robot trajectory while the EM sensor remains stationary at a location. As described above, the robot trajectory may be determined based on the kinematics of the arm to which the EM field generator is attached. In some embodiments, the robot trajectory may be determined relative to a global reference frame, which may be associated, for example, with a trolley to which the robotic arm is attached. In some embodiments, block 801 is executed by a processor (or processors) of the robotic medical system. For example, the processor may be configured to send commands to the first robotic arm to cause the EM field generator to move along the robot trajectory. In some cases, the EM sensor may be fixed to a determinable position while the EM field generator moves along the robot trajectory.

[0321] At box 802, as the EM field generator moves along the robot trajectory, the EM sensor trajectory within the EM field associated with the EM field generator can be detected based on sensor data generated by the EM sensor. In this way, and as previously noted, the EM trajectory can be the position of the EM sensor within the EM field recorded during the same time period as the robot arm moves the EM field generator according to the robot trajectory.

[0322] At box 803, the robot trajectory and the EM sensor trajectory are analyzed to determine the differences between them. This analysis can be performed in various ways, including comparing the shape of the trajectories and / or comparing multiple discrete points along the trajectories. Such comparisons may take into account variations in position and / or orientation.

[0323] At box 804, EM distortion can be detected at the location of the EM sensor based on the difference between trajectories and a comparison between thresholds. The threshold can be selected or determined to reduce or eliminate contributions from factors other than EM distortion, such as errors in recording or determining robot motion and / or detecting the location of the EM sensor within the EM field.

[0324] Figure 48This is a flowchart illustrating another exemplary method 900 for EM distortion detection. Method 900 can be performed using a robotic medical system such as those previously described, in which the EM field is typically robotically controllable via a robotic arm attached to the system. Method 900 begins at block 901, where a first robotic arm is coupled to an EM field generator. The EM field generator is configured to generate an EM field and is associated with an EM coordinate system. The first robotic arm is associated with a robot coordinate system, and the position of the coupled EM field generator can be determined within the robot coordinate system based on the kinematic pose of the first robotic arm. In some embodiments, the position can be further determined relative to a global coordinate system, such as the coordinate system associated with a trolley to which the robotic arm is attached. The position of the EM sensor within the working volume of the EM field of the EM field generator can be determined, and the position is mapped to the robot coordinate system (or global coordinate system) based on the arm's pose, as previously described.

[0325] At block 902, when the EM field generator is at a first EM field generator position relative to the EM sensor, method 900 includes: determining a first registration between the EM coordinate system and the robot coordinate system based on the kinematics of the first robot arm at the first EM field generator position; and determining a first EM sensor position of the EM sensor in the robot coordinate system based on the first registration.

[0326] At block 903, method 900 includes: moving an EM field generator relative to an EM sensor from a first EM field generator position to a second EM field generator position using a first robotic arm, wherein the EM sensor remains stationary during the movement. In some embodiments, the movement of the EM field generator by the first robotic arm may be performed under the guidance or control of an operator such as a physician, who may command the movement using a controller or other user interface. In other embodiments, the movement of the EM field generator by the first robotic arm along a robot trajectory may be provided automatically, for example, as part of an automated EM distortion detection process operated by the system.

[0327] At box 904, with the EM field generator at the second EM field generator position, a second registration between the EM coordinate system and the robot coordinate system is determined based on the kinematics of the first robot arm at the second EM field generator position. Furthermore, the second EM sensor position within the robot coordinate system (or global coordinate system) is determined based on this second registration. This second registration is necessary because the EM field generator has moved, and the new kinematic pose of the robot arm at the second position is used to establish a new relationship between the robot coordinate system and the EM coordinate system.

[0328] At block 905, method 900 includes determining the difference between the positions of a first EM sensor and a second EM sensor. Since the EM sensors have not yet moved, it is expected that the determined positions will be the same. However, EM distortion caused by the different positions of the robot arm and the EM field generator (or other components that have moved) can cause the determined positions to vary. In some embodiments, variations in orientation may also be considered. At block 906, EM distortion can be detected based on a comparison between the difference and a threshold (e.g., if the difference exceeds the threshold). As noted above, other factors can cause the difference. As previously stated, a threshold can be selected or determined to reduce the contribution of these other non-EM distortion factors.

[0329] When observed relative to a threshold, a difference exceeding the threshold can indicate the presence of EM distortion. However, by testing only two EM field generator positions (and / or orientations), it may be difficult to determine which position provides the most accurate location determination. Therefore, it is advantageous to continue testing multiple other EM field generator positions (and / or orientations) by repeating the following associated steps: moving the EM field generator to the new position using a robotic arm, determining a new EM-to-robot coordinate registration based on the new position of the EM field generator using the kinematics of the robotic arm, and determining the position of the EM sensor in the robot coordinate system based on the newly determined registration.

[0330] In some implementations, the process can continue by constructing an EM distortion map by determining the distortion at multiple locations. The location of the determined EM field generator, providing the EM sensor location with minimal variation from other locations, can be determined as having low EM distortion, while the location of the determined EM field generator, providing the EM sensor location with maximum variation from other locations, can be determined as having high EM distortion. In this way, the system or operator can determine how to optimally position the EM field generator and other components to reduce EM distortion.

[0331] EM distortion (and typical EM distortion) detected using the methods described above can be static or dynamic. Static EM distortion is constant and can be identified and removed from EM measurements by applying corresponding correction values. Static EM distortion can be caused by multiple factors, including objects in the operating room that are generally stationary and therefore do not change during the procedure, as well as large scale factors such as the Earth's magnetic field. Dynamic EM distortion is caused by factors that change during the procedure. These factors can include any moving robotic components of the system, as well as other components (or even people) that can move through the operating room. Distortion detection can be used both to detect and correct static distortion and to find the optimal location for reducing dynamic distortion in moving parts.

[0332] v. Multimodal sensor fusion

[0333] The robotic controlled EM field generator 302 is also useful in allowing multimodal sensor fusion. As used herein, multimodal sensor fusion can refer to the simultaneous or concurrent use of different sensor types in a cooperative manner during a single procedure. For example, in the case of an EM field generator 302 rigidly mounted on a robotic arm 212, EM sensing technology can be registered to other imaging and sensing modalities mounted on the robotic arm 212 or in the robotic system. Registration of the EM sensing modality with other imaging and sensing modalities attached to or part of the robotic arm, cart, or base can be beneficial, for example, in providing simplified displays (where data from multiple sensing modalities can be displayed together in a unified manner), augmented reality, etc. In this section, an example in which EM sensing technology can be used with ultrasound technology to provide multimodal functionality will be described. This allows, for example, the accurate identification of a needle tip (where the needle tip includes an EM sensor) in an ultrasound image. In general, the identification of a needle tip in an ultrasound image may not be straightforward, as artifacts can distort the image. Registering the ultrasound imaging plane and EM field to a single space, such as a robot coordinate system, can facilitate the determination of the needle tip in the ultrasound image. To achieve this, the ultrasound probe can be rigidly mounted on a robot arm, as will be described in more detail below. If the coordinate system of the ultrasound imaging plane relative to the EM field generator 302 is kinematically known, it is possible to track the EM sensor inside the needle in real time and overlay it onto the ultrasound image.

[0334] Attaching an ultrasound probe to a robotic arm can facilitate the registration or calibration of the ultrasound probe's imaging plane. Calibration of the ultrasound probe's imaging plane typically requires capturing three different ultrasound images from three different known locations. These three ultrasound images, along with their corresponding known locations, can be used to calibrate the imaging plane. Conventionally, additional equipment is required to calibrate the ultrasound probe used in a surgical robotic system. For example, an external position tracking system must be set up to determine the locations where the three ultrasound images were captured. As a specific example, an optical sensor (such as an infrared LED) can be attached to the ultrasound probe. Position tracking systems such as Optitrack can detect the position of the optical sensor attached to the ultrasound probe and are used to determine the ultrasound probe's position. In a surgical robotic setup, this can be disadvantageous because of the need for additional equipment.

[0335] By attaching an ultrasound probe to a robotic arm, the position of the ultrasound probe can be easily determined relative to the robot coordinate system associated with the arm based on the arm's kinematics, similar to the method described above regarding EM field generators attached to the robotic arm. A base ultrasound image can be captured at this base position. The robotic arm can then move the ultrasound probe to a first position for capturing the first ultrasound image, and this first position can be determined based on the arm's kinematics. The robotic arm can then move the ultrasound probe to a second position for capturing a second ultrasound image, and this second position can be determined based on the arm's kinematics at the second position. The imaging plane of the ultrasound probe can then be calibrated using the three images (base, first, and second) and positions (base, first, and second) without requiring any external or additional equipment, such as conventional position tracking devices.

[0336] Figure 49A This illustration depicts a robotic medical system with an ultrasound probe 1050 attached to a robotic arm 212 during a procedure for calibrating the imaging position of the ultrasound probe. In the illustrated embodiment, the robotic arm 212 is coupled to the ultrasound probe 1050. For example, the ultrasound probe 1050 may be attached to the distal end of the robotic arm 212 or to an instrument device manipulator (IDM) positioned on the arm. The robotic arm 212 is configured to move to adjust the position of the ultrasound probe 1050. The robotic arm 212 is also associated with a robot coordinate system, as previously described. Figure 49A In the illustrated implementation, three positions, Bi, Bj, and Bk, are shown. The robotic arm 212 moves to move the ultrasonic probe 212 between these three positions. At each position, the kinematics of the arm can be used to determine the position of the ultrasonic probe in the robot coordinate system.

[0337] At a first position Bi, the ultrasound probe 1050 captures a first image. Then, the robotic arm 212 moves the ultrasound probe 1050 to a second position Bj. At the second position Bj, the ultrasound probe 1050 captures a second ultrasound image. Then, the robotic arm 212 moves the ultrasound probe 1050 to a third position Bj. At the third position Bj, the ultrasound probe 1050 captures a third ultrasound image. The system can calibrate the imaging plane of the ultrasound probe 1050 relative to the robot coordinate system based on: the first kinematics of the first ultrasound image and the first robotic arm with the ultrasound probe at the first ultrasound probe position; the second kinematics of the second ultrasound image and the first robotic arm with the ultrasound probe at the second ultrasound probe position; and the third kinematics of the third ultrasound image and the first robotic arm with the ultrasound probe at the second ultrasound probe position.

[0338] With the imaging plane calibrated and registered to the robot coordinate system, the robot system can utilize multiple sensing modes concurrently. Figure 49BAn exemplary multimodal robot system is shown. Figure 49B As shown, the robotic system may include a first robotic arm 212A, a second robotic arm 212B, and a third robotic arm 212C. The robotic arms can be attached to a trolley, patient platform, or other common structure, such that each robotic arm is associated with a common robot coordinate system. In the illustrated embodiment, an ultrasound probe 1050 with its imaging plane calibrated is attached to the first robotic arm 212A. An EM field generator 302 is attached to the second robotic arm 212B. The EM field generator 302 is configured to generate an EM field within which the position of an EM sensor, such as EM sensor 204, can be determined. Using the EM-to-robot coordinate system registration procedure described above, the position of EM sensor 204 can be determined within the robot coordinate system. Since the imaging plane of the ultrasound probe is also registered to the robot coordinate system, the imaging plane and the position of EM sensor 204 are both determined in a common space (robot coordinate system) and therefore can both be displayed on a common display, such as... Figure 49B As shown.

[0339] exist Figure 49B In this device, the EM sensor 204 can be positioned on the distal end of the endoscope inserted into the patient's treatment area. The endoscope can be a robotic, controllable medical device attached to and controlled by a different robotic arm. Alternatively, the endoscope can be a manually controlled endoscope. Figure 49B The diagram also illustrates the attachment of medical devices, such as needles, to a third robotic arm 212C. The position of the needle, which can be rigidly attached to the third robotic arm 212C, can also be determined relative to the robot coordinate system. Therefore, the position of the needle can also be displayed along with the imaging plane and EM sensor data. In this way, various sensor technologies can be integrated into a single robotic system and used concurrently during a single procedure. In this example, the rendezvous of the needle with the endoscope is facilitated by robot data, ultrasound data, and EM data, which improves the accuracy of the rendezvous and provides a comprehensive and improved operating experience.

[0340] Figure 50This is a flowchart illustrating an exemplary method 1000 for calibrating the imaging plane of an ultrasound probe used with a robotic medical system. Method 1000 begins at block 1001, where an ultrasound probe is coupled to a first robotic arm. The first robotic arm is configured to move to adjust the position of the ultrasound probe and is associated with a robot coordinate system. At block 1002, the ultrasound probe is moved to a first ultrasound probe position using the first robotic arm. The first ultrasound probe position may be a location where a first ultrasound image will be captured to calibrate the imaging plane of the ultrasound probe. The first position may be determined based on the kinematics of the first robotic arm. At block 1003, the first ultrasound image is captured using the ultrasound probe positioned at the first ultrasound probe position. At block 1004, the ultrasound probe is moved to a second ultrasound probe position using the first robotic arm. The second ultrasound probe position may be a location where a second image can be captured, which occurs at block 1005. The second image may also be used to calibrate the imaging plane of the ultrasound probe. The second position may be determined based on the kinematics of the arm. As indicated by arrow 1006, boxes 1004 and 1005 can be repeated to capture a third image at a third location, for example, as shown below. Figure 49A As shown. In some implementations, these steps may be repeated a number of times to capture additional images at the additional locations.

[0341] At box 1007, the imaging plane of the ultrasound probe can be calibrated relative to the robot coordinate system based on: a first ultrasound image and a first robot arm with the ultrasound probe at the first ultrasound probe position; a second ultrasound image and a second robot arm with the ultrasound probe at the second ultrasound probe position; and a third ultrasound image and a third robot arm with the ultrasound probe at the third ultrasound probe position. When calibrated, the imaging plane can be determined relative to the robot coordinate system and used in conjunction with other sensing modes (such as EM) that can also be determined relative to the robot coordinate system.

[0342] In some embodiments, method 1000 may further include: attaching an EM field generator to a second robotic arm, wherein the EM field generator is configured to generate an EM field and is associated with an EM coordinate system, and wherein the second robotic arm is configured to move to adjust the position of the EM field generator and is associated with a robot coordinate system. Method 1000 may further include: determining a registration between the EM field coordinate system and the robot coordinate system based on the kinematics of the second robotic arm. This can be achieved as previously described. The first and second robotic arms may be attached to a trolley or patient platform that defines the relationship between the first and second robotic arms within the robot coordinate system. This allows both the EM field generator and the ultrasound probe to be brought into the robot coordinate system. Method 1000 may further include: positioning an EM sensor within the EM field and determining the position of the EM sensor within the robot coordinate system based on the registration. Additionally, method 1000 may further include: displaying the imaging plane of the ultrasound probe and superimposing an indication of the determined EM sensor position onto the displayed imaging plane of the ultrasound probe, for example, as shown in the image. Figure 49B As shown.

[0343] While multimodal functionality has been described with respect to EM and ultrasound, other modalities may also be used. For example, in some embodiments, depth sensors may be attached to the robotic arm. Depth sensors can be used to register the positions of the patient, bed or platform, fluoroscopic C-arm used during the procedure, and / or other items in the surgical space to a global robotic reference frame. In some embodiments, these positions may be represented as heatmaps or point clouds, respectively, such as... Figure 51A and Figure 51B As shown. Depth sensor camera, for example, such as Figure 51C As shown, it can be configured to measure the distance to objects within its field of view. Calibration of the depth sensor (e.g., registering the depth sensor output to the robot coordinate system or the global coordinate system) can be performed in the same manner as calibrating the ultrasonic probe described above. Generally, depth sensors are calibrated by the manufacturer relative to pixels and depth information. For use with, for example, a robotic system as described herein, it may be necessary to calibrate the depth sensor relative to the robot system's pose, as mentioned above, which can be achieved in a similar manner to that used for the previously described ultrasonic probe.

[0344] In some implementations, the robotic medical system may utilize multiple different sensor functions combined with each other. For example, the system may include an ultrasound probe, a camera or depth sensor, and an EM field generator, the output of each of which can be calibrated and registered to a robot or global coordinate system. In these implementations, the ultrasound probe can provide imaging data about the patient's interior. Similarly, a camera or depth sensor can provide imaging or other information about external anatomical structures, such as the patient and the position of external devices in the operating room, such as a C-arm. This can be advantageous because knowing the patient's position relative to the robot coordinate system allows the system to identify, for example, needle insertion sites. Additionally, knowing the position of other objects in the operating room, such as the C-arm, relative to the robot coordinate system allows the system to know when an object approaches the field generator, which may cause distortion.

[0345] 3. Implementation System and Terminology .

[0346] The specific implementations disclosed herein provide systems, methods, and apparatus for robotic controllable field generators.

[0347] It should be noted that, as used herein, the terms “couple,” “coupling,” “coupled,” or other variations of the word “couple” can indicate an indirect or direct connection. For example, if a first component is “coupled” to a second component, the first component may be indirectly connected to the second component or directly connected to the second component via another component.

[0348] Phrases referring to specific computer-implemented processes and functions described herein may be stored as one or more instructions on a processor-readable or computer-readable medium. The term "computer-readable medium" means any available medium that can be accessed by a computer or processor. By way of example, and not limitation, such media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, compact disc read-only memory (CD-ROM) or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be noted that computer-readable media may be tangible and non-transitory. As used herein, the term "code" may mean software, instructions, code, or data that can be executed by a computing device or processor.

[0349] The methods disclosed herein include one or more steps or actions for implementing the methods. The method steps and / or actions may be interchanged without departing from the scope of the claims. In other words, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims unless proper operation of the described method requires a specific order of steps or actions.

[0350] As used herein, the term "multiple" means two or more. For example, multiple components indicates two or more components. The term "determine" encompasses a variety of actions, and therefore, "determine" can include calculation, operation, processing, derivation, investigation, lookup (e.g., searching in a table, database, or another data structure), ascertainment, etc. Additionally, "determine" can include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), etc. Furthermore, "determine" can include parsing, selecting, picking, building, etc.

[0351] Unless otherwise explicitly stated, the phrase “based on” does not mean “based on only”. In other words, the phrase “based on” describes both “based on only” and “based on at least”.

[0352] The prior description of the disclosed specific embodiments is provided to enable any person skilled in the art to make or use the inventions disclosed herein. Various modifications to these specific embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other specific embodiments without departing from the scope of the invention. For example, it should be understood that those skilled in the art will be able to employ multiple corresponding alternatives and equivalent structural details, such as equivalent ways of fastening, mounting, connecting, or engaging tool components, equivalent mechanisms for generating specific actuating movements, and equivalent mechanisms for delivering electrical energy. Therefore, the disclosure herein is not intended to be limited to the specific embodiments explicitly described, but is to be given the broadest scope consistent with the principles and novel features disclosed herein.

Claims

1. A robotic medical system, comprising: a first robotic arm configured to be coupled to an electromagnetic (EM) field generator and configured to move the EM field generator; and one or more processors configured to: determine an EM position of an EM sensor within an EM field in an EM coordinate system associated with the EM field generator; determine a position of the EM field generator in a robotic coordinate system associated with the first robotic arm; determine a registration between the EM coordinate system and the robotic coordinate system based on the position of the EM field generator; and based on the registration, determine a position of the EM sensor in the robotic coordinate system.

2. The system of claim 1, wherein the one or more processors are configured to determine the position of the EM field generator based on kinematics of the first robotic arm.

3. The system of claim 1, wherein the EM field generator comprises a compact field generator (cFG).

4. The system of claim 1, wherein the first robotic arm is configured to be removably coupled to the EM field generator.

5. The system of claim 1, further comprising: a second robotic arm configured to control movement of a medical instrument, wherein the robotic coordinate system is further associated with the second robotic arm.

6. The system of claim 5, wherein the first robotic arm and the second robotic arm are coupled to a movable cart.

7. The system of claim 5, wherein the first robotic arm and the second robotic arm are coupled to a patient platform configured to support a patient during a medical procedure.

8. The system of claim 5, wherein the EM sensor is positioned on the medical instrument.

9. The system of claim 1, wherein the one or more processors are further configured to cause the first robotic arm to move the EM field generator to allow access to a patient during a medical procedure.

10. The system of claim 1, wherein the one or more processors are further configured to: cause the first robotic arm to move the EM field generator to a new position relative to the EM sensor; and determine a new EM position of the EM sensor within the EM field in the EM coordinate system associated with the EM field generator, wherein the new EM position has improved accuracy compared to the EM position.

11. The system of claim 1, wherein the one or more processors are further configured to cause the first robotic arm to move the EM field generator to center the EM sensor within a working volume of the EM field generator based on the position of the EM field generator and a position of the first robotic arm.

12. A method for performing a robotic medical procedure, the method comprising: generating an electromagnetic (EM) field with an EM field generator coupled to a first robotic arm, the EM field being associated with an EM coordinate system; determining an EM position of an EM sensor within the EM field in the EM coordinate system associated with the EM field generator; ​ determine a position of the EM field generator within a robot coordinate system based on kinematics of the first robot arm; determine a registration between the EM coordinate system and the robot coordinate system associated with the first robot arm based on the determined position of the EM field generator; determine a position of an EM sensor within the EM coordinate system; and determine a position of the EM sensor in the robot coordinate system based on the registration.

13. The method of claim 12, wherein the determination of the position of the EM field generator within the robot coordinate system is based on kinematics of the first robot arm.

14. The method of claim 12, further comprising: removably couple the EM field generator to the first robot arm.

15. The method of claim 12, further comprising: cause the first robot arm to move the EM field generator.

16. The method of claim 15, wherein causing the first robotic arm to move the EM field generator comprises: cause the first robot arm to move the EM field generator to a position that allows access to a patient during a medical procedure.

17. The method of claim 15, wherein causing the first robotic arm to move the EM field generator comprises: move the EM field generator relative to a new position that is associated with an increased accuracy of tracking the EM sensor within the EM field coordinate system.

18. The method of claim 15, wherein causing the first robotic arm to move the EM field generator comprises: center the EM sensor within a working volume of the EM field generator.

19. The method of claim 12, further comprising: control a medical instrument coupled to a second robot arm, wherein the robot coordinate system is associated with the second robot arm.

20. The method of claim 19, wherein the first robot arm and the second robot arm are coupled to a movable cart.

21. The method of claim 20, further comprising: position the movable cart relative to a patient platform configured to support a patient during the robotic medical procedure such that there is a known spatial relationship between a robot coordinate system and the patient platform.

22. The method of claim 19, wherein the first robot arm and the second robot arm are coupled to a patient platform configured to support a patient during the robotic medical procedure.

23. The method of claim 19, wherein the EM sensor is positioned on the medical instrument.

24. The method of claim 12, wherein the EM field generator comprises a compact field generator (cFG).

25. A non-transitory computer-readable storage medium comprising computer program instructions that, when executed by one or more processors, cause the one or more processors to: determine a position of an electromagnetic (EM) field generator within a robot coordinate system based on kinematics of a first robot arm, the EM field generator coupled to the first robot arm; determine a registration between an EM coordinate system and the robot coordinate system associated with the first robot arm based on the determined position of the EM field generator, the EM coordinate system associated with an EM field generated by the EM field generator; determine a position of an EM sensor within the EM coordinate system; and determine a position of the EM sensor in the robot coordinate system based on the registration.

26. The non-transitory computer-readable storage medium of claim 25, wherein the EM field generator comprises a compact field generator (cFG).

27. The non-transitory computer-readable storage medium of claim 25, wherein the first robotic arm is configured to be removably coupled to the EM field generator.

28. The non-transitory computer-readable storage medium of claim 25, wherein the computer program instructions, when executed, further cause the one or more processors to: control a second robotic arm configured to control movement of a medical instrument, wherein the robotic coordinate system is further associated with the second robotic arm.

29. The non-transitory computer-readable storage medium of claim 28, wherein the first robotic arm and the second robotic arm are coupled to a movable cart.

30. The non-transitory computer-readable storage medium of claim 28, wherein the first robotic arm and the second robotic arm are coupled to a patient platform configured to support a patient during a medical procedure.

31. A robotic medical system, comprising: a first robotic arm configured to be coupled to an electromagnetic (EM) field generator and to move the EM field generator, the EM field generator configured to generate an EM field having a work volume, a location of an EM sensor within the EM field being determinable relative to an EM coordinate system; and one or more processors configured to: move the EM field generator using the first robotic arm; and when an EM sensor is detected within the work volume: (i) determine a location of the EM sensor within the work volume relative to the EM coordinate system, and (ii) map the location of the EM sensor into a robotic coordinate system associated with the first robotic arm based on a kinematic pose of the first robotic arm.

32. The system of claim 31, wherein the one or more processors are configured to move the EM field generator using the first robotic arm along a search path, the search path configured to sweep the work volume through a treatment volume larger than the work volume to locate an EM sensor within the treatment volume.

33. The system of claim 32, wherein the one or more processors are further configured to: determine a tracking path based on the determined location of the EM sensor within the robotic coordinate system; move the EM field generator using the first robotic arm along the tracking path so as to track the location of the EM sensor; and move the EM field generator using the first robotic arm along the tracking path.

34. The system of claim 33, wherein when the EM field generator is moved along the tracking path, the one or more processors are configured to: (i) redetermine the location of the EM sensor within the work volume relative to the EM coordinate system, (ii) remap the location of the EM sensor into the robotic coordinate system based on the kinematic pose of the first robotic arm, and (iii) determine an updated tracking path. ​ 35. A robotic medical system, comprising: a first robotic arm coupled to an electromagnetic (EM) field generator and configured to move the EM field generator, the EM field generator configured to generate an EM field having a work volume, a location of an EM sensor within the EM field being determinable relative to an EM coordinate frame; one or more processors configured to: (i) with the EM field generator in a first position: determine a location of a first EM sensor within the work volume relative to the EM coordinate frame, and map the location of the first EM sensor into a robotic coordinate frame associated with the first robotic arm based on kinematics of the first robotic arm with the EM field generator in the first position, (ii) move the EM field generator to a second position; and (iii) with the EM field generator in the second position: determine a location of a second EM sensor within the work volume relative to the EM coordinate frame, and map the location of the second EM sensor into the robotic coordinate frame based on the kinematics of the first robotic arm with the EM field generator in the second position.

36. The system of claim 35, wherein the one or more processors are further configured to move the first robotic arm such that the first EM sensor and the second EM sensor are not simultaneously within the work volume of the EM field generator.

37. The system of claim 35, wherein the one or more processors are further configured to move the EM field generator between the first position and the second position to track the locations of the first EM sensor and the second EM sensor.

38. A method, comprising: moving an electromagnetic (EM) field generator positioned on a first robotic arm using the first robotic arm; and when an EM sensor is detected within a work volume: (i) determining a location of the EM sensor within a work volume of an EM field generated by the EM field generator relative to an EM coordinate frame, and (ii) mapping the location of the EM sensor into a robotic coordinate frame associated with the first robotic arm based on a kinematic pose of the first robotic arm.

39. A method, comprising: (i) with an electromagnetic (EM) field generator coupled to a first robotic arm in a first position: determining a location of a first EM sensor within a work volume of an EM field generated by the EM field generator relative to the EM coordinate frame, mapping the location of the first EM sensor into a robotic coordinate frame associated with the first robotic arm based on kinematics of the first robotic arm with the EM field generator in the first position, (ii) moving the EM field generator to a second position using the first robotic arm; and (iii) with the EM field generator in the second position: determining a location of a second EM sensor within the work volume relative to the EM coordinate frame, and mapping the location of the second EM sensor into the robotic coordinate frame based on the kinematics of the first robotic arm with the EM field generator in the second position. (iii) in the event that the EM field generator is in the second position: determining a position of a second EM sensor within the working volume relative to the EM coordinate system, mapping the position of the second EM sensor into the robot coordinate system based on the kinematics of the first robot arm in the event that the EM field generator is in the second position.

Images