System and method for driving a medical instrument

Abstract

The present disclosure relates to systems and methods for driving medical instruments. Certain aspects relate to systems and techniques for driving medical instruments having an inner body and an outer body. In one aspect, a system includes a medical instrument including an outer body and an inner body configured to be driven through a lumen in the outer body. The system can also include a set of one or more instrument manipulators configured to control movement of the outer body and the inner body, and a set of one or more processors configured to: receive a change drive mode command; and change a drive mode of the medical instrument from a paired drive mode to an unpaired drive mode in response to receiving the change drive mode command.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 201980003375.4, filed on February 12, 2019, entitled "System and Method for Driving a Medical Device".

[0002] Cross-reference to related applications

[0003] This application claims the benefit of U.S. Provisional Application No. 62 / 630,112, filed February 13, 2018, which is incorporated herein by reference in its entirety. Technical Field

[0004] The systems and methods disclosed herein relate to systems and methods for driving medical devices, and more specifically to techniques for driving medical devices in which an inner body is configured to be driven through a cavity of an outer body. Background Technology

[0005] Medical procedures such as endoscopy (e.g., bronchoscopy) may involve inserting medical instruments into a patient's network of cavities (e.g., airways) for diagnostic and / or therapeutic purposes. Surgical robotic systems can be used to control the insertion and / or manipulation of medical instruments during the medical procedure. A surgical robotic system may include at least one robotic arm with a manipulator assembly that can be used to control the positioning of the medical instrument before and during the medical procedure. Some medical instruments may include an outer body and an inner body configured to be driven through a cavity of the outer body. For some medical procedures, independent control of the inner and outer bodies of the medical instrument may be desired. Summary of the Invention

[0006] The systems, methods, and apparatuses disclosed herein have several innovative aspects, none of which individually is responsible for the desired properties disclosed herein.

[0007] In one aspect, a robotic surgical system is provided, comprising: a medical device including an outer body and an inner body configured to be driven through a cavity in the outer body; a set of one or more robotic arm assembly instrument manipulators configured to control movement of the outer and inner bodies; a set of one or more user input devices; a set of one or more processors; and at least one computer-readable storage device communicating with the set of processors and storing computer-executable instructions on the at least one computer-readable storage device to cause the set of processors to: receive a change drive mode command via the set of user input devices; and, in response to receiving the change drive mode command, change the drive mode of the medical device from a paired drive mode to an unpaired drive mode, wherein, in response to receiving the drive command from the set of input devices, the distance between the end of the inner body and the end of the outer body is maintained at a predetermined distance.

[0008] In another aspect, a robotic surgical system is provided, comprising: a medical device including an outer body and an inner body configured to be driven through a cavity in the outer body; a set of one or more instrument manipulators configured to control movement of the outer and inner bodies; a set of one or more user input devices; a set of one or more processors; and at least one computer-readable storage device communicating with the set of processors and storing computer-executable instructions on the at least one computer-readable storage device to cause the set of processors to: receive an engagement command for engaging the medical device via the set of user input devices; designate one of the outer and inner bodies as a primary body and the other of the outer and inner bodies as a secondary body; determine a distance between an end of the inner body and an end of the outer body; determine a common engagement factor based on the determined distance; engage the primary body via the set of instrument manipulators based on the engagement command; and engage the secondary body via the set of instrument manipulators based on the engagement command and the common engagement factor.

[0009] In another aspect, a robotic surgical system is provided, comprising: a medical device including an outer body and an inner body configured to be driven through a cavity in the outer body; a set of one or more instrument manipulators configured to control movement of the outer and inner bodies; a set of one or more feedback devices; a set of one or more processors; at least one computer-readable memory communicating with the set of processors and storing on the at least one computer-readable memory a model of a mapped portion of a cavity network, a target position relative to the model, and a path along the model from an entry point to the target, the memory also storing computer-executable instructions that cause the set of processors to: identify portions of the cavity network along a path having a shape matching a parking aid feature; and generate a parking indication at at least a portion of the set of feedback devices at a position relative to the model corresponding to the identified portion, the parking indication indicating the position of the end of the outer body. Attached Figure Description

[0010] The disclosed aspects will be described below in conjunction with the accompanying drawings, which are provided for illustrative purposes and not for limiting the scope of the disclosure, wherein like reference numerals denote like elements.

[0011] Figure 1 An embodiment of a cart-based robotic system arranged for diagnostic and / or therapeutic bronchoscopy procedures is shown.

[0012] Figure 2 Depicting Figure 1 Another aspect of robotic systems.

[0013] Figure 3 The apparatus is shown in the configuration for ureteroscopy. Figure 1 The implementation method of the robot system.

[0014] Figure 4 The diagram shows an arrangement for use in vascular processes. Figure 1 The implementation method of the robot system.

[0015] Figure 5 An embodiment of a stage-based robotic system arranged for a bronchoscopy procedure is shown.

[0016] Figure 6 Provided Figure 5 Alternative views of the robot system.

[0017] Figure 7 An example system configured to deploy and retract a robotic arm is shown.

[0018] Figure 8An implementation of a stage-based robotic system configured for a ureteroscopy procedure is shown.

[0019] Figure 9 An implementation of a stage-based robotic system configured for laparoscopic procedures is shown.

[0020] Figure 10 It shows a pitch or tilt adjustment. Figures 5 to 9 Implementation methods of platform-based robot systems.

[0021] Figure 11 Provided Figures 5 to 10 A detailed diagram of the interface between the platform and the column in a platform-based robotic system.

[0022] Figure 12 An exemplary device driver is shown.

[0023] Figure 13 An exemplary medical device with a pair of device actuators is shown.

[0024] Figure 14 An alternative design for the instrument actuator and the instrument is shown, wherein the axis of the actuator is parallel to the axis of the slender axis of the instrument.

[0025] Figure 15 A block diagram illustrating a positioning system according to an example embodiment is depicted, the positioning system estimating Figures 1 to 10 The location of one or more components of a robotic system, for example Figure 13 and Figure 14 The location of the instruments.

[0026] Figure 16 An embodiment of a surgical robot system that can be configured to drive a medical device having an outer body and an inner body, according to aspects of this disclosure, is shown.

[0027] Figure 17A Embodiments of a medical device according to aspects of this disclosure are described.

[0028] Figure 17B An embodiment of a paired drive mode for a medical device according to aspects of this disclosure is shown.

[0029] Figure 18 This is a flowchart illustrating an example method for changing the drive mode of a medical device, operable by a surgical robot system or a component thereof, according to aspects of this disclosure.

[0030] Figure 19 An embodiment of an internal body drive mode for a medical device according to aspects of this disclosure is shown.

[0031] Figure 20 An embodiment of an external body drive mode for a medical device according to aspects of this disclosure is shown.

[0032] Figure 21 A block diagram illustrating a technique for altering the driving mode of a medical device according to aspects of this disclosure is provided.

[0033] Figure 22 An embodiment of automated pairing for a medical device according to aspects of this disclosure is shown.

[0034] Figure 23 Another embodiment of automated pairing for a medical device according to aspects of this disclosure is shown.

[0035] Figure 24 A block diagram illustrating another technique for altering the driving mode of a medical device, based on aspects of this disclosure, is provided.

[0036] Figure 25 An implementation of an initial follow-up method for a medical device based on aspects of this disclosure is shown.

[0037] Figure 26 Another embodiment for initial follow-up of a medical device according to aspects of this disclosure is shown.

[0038] Figure 27 A block diagram is provided illustrating another technique for altering the driving mode of a medical device according to aspects of this disclosure.

[0039] Figure 28 This is a flowchart illustrating an example method operated by a surgical robot system or a component thereof for co-engaging the outer and inner bodies of a medical device, according to aspects of this disclosure.

[0040] Figure 29A This includes diagrams illustrating techniques for determining the co-engagement ratio in an in-body driven mode, based on aspects of this disclosure.

[0041] Figure 29B This includes a diagram illustrating a technique for determining the co-engagement ratio in an external body-driven mode, based on aspects of this disclosure.

[0042] Figure 30 An embodiment of tension monitoring for a medical device according to aspects of this disclosure is shown.

[0043] Figure 31 An embodiment of the automatic relaxation of a medical device during retraction according to aspects of this disclosure is shown.

[0044] Figure 32 This is a flowchart illustrating an example method operated by a surgical robot system or a component thereof for parking assistance of a medical device, according to aspects of this disclosure.

[0045] Figure 33 An example of parking instructions based on aspects of this disclosure is shown. Detailed Implementation

[0046] 1. Overview.

[0047] The aspects of this disclosure can be integrated into robot-enabled medical systems capable of performing various medical procedures, including minimally invasive procedures such as laparoscopy and non-invasive procedures such as endoscopy. During endoscopy, the system can perform bronchoscopy, ureteroscopy, gastroenterology, etc.

[0048] In addition to performing a wide range of procedures, the system offers additional benefits such as enhanced imaging and guidance to assist physicians. Furthermore, the system provides physicians with the ability to perform procedures from an ergonomic position without requiring cumbersome arm movements and positioning. Additionally, the system offers physicians the ability to perform procedures with improved ease of use, allowing one or more instruments within the system to be controlled by a single user.

[0049] For illustrative purposes, various embodiments will be described below with reference to the accompanying drawings. It should be understood that many other implementations of the disclosed concepts are possible, and various advantages can be obtained by utilizing the disclosed implementations. This document contains headings for reference and to help locate the various sections. These headings are not intended to limit the scope of the concepts described therein. Such concepts can be applied throughout the specification.

[0050] A. Robotic System - Cart

[0051] Depending on the specific process, robotic-enabled medical systems can be configured in various ways. Figure 1An embodiment of a trolley-based robotic-enabled system 10 arranged for diagnostic and / or therapeutic bronchoscopy procedures is illustrated. During bronchoscopy, system 10 may include a trolley 11 having one or more robotic arms 12 to deliver medical instruments (e.g., a steerable endoscope 13, which may be a bronchoscope specifically designed for bronchoscopy procedures) to a natural orifice entry point (i.e., in this example, the patient's mouth positioned on a table) to deliver diagnostic and / or therapeutic tools. As shown, trolley 11 may be positioned near the patient's upper torso to provide access to the entry point. Similarly, robotic arms 12 may be actuated to position the bronchoscope relative to the entry point. This can also be used when performing GI procedures using a gastroscope (a specialized endoscope for gastrointestinal (GI) procedures). Figure 1 The device in the middle. Figure 2 An exemplary implementation of the cart is described in more detail.

[0052] Continue to refer to Figure 1 Once the trolley 11 is properly positioned, the robotic arm 12 can automatically, manually, or in combination, insert the steerable endoscope 13 into the patient's body. As shown, the steerable endoscope 13 may include at least two telescopic portions, such as an inner guide portion and an outer sheath portion, each coupled to a separate instrument actuator from the set of instrument actuators 28, each instrument actuator coupled to the end of a separate robotic arm. This linear arrangement of the instrument actuators 28, facilitating coaxial alignment of the guide portion and the sheath portion, creates a “virtual track” 29, which can be repositioned in space by manipulating one or more robotic arms 12 to different angles and / or positions. The virtual track described herein is depicted using dashed lines in the figures, and therefore does not depict any physical structure of the system. Translation of the instrument actuators 28 along the virtual track 29 causes the inner guide portion to extend or retract relative to the outer sheath portion, or causes the endoscope 13 to advance or retract relative to the patient. The angle of the virtual track 29 can be adjusted, translated, and pivoted based on clinical application or physician preference. For example, in bronchoscopy, the angle and position of the virtual track 29 shown represent a trade-off between bringing the physician closer to the endoscope 13 and minimizing friction caused by the endoscope 13 bending into the patient's mouth.

[0053] Precise commands from the robotic system can be used to guide the endoscope 13 along the patient's trachea and lungs after insertion until the target destination or surgical site is reached. To enhance navigation through the patient's lung network and / or reach the desired target, the endoscope 13 can be manipulated to telescopically extend the inner guide portion from the outer sheath portion for enhanced engagement and a larger radius of curvature. The use of separate instrument actuators 28 also allows the guide portion and sheath portion to be driven independently of each other.

[0054] For example, endoscope 13 can be guided to deliver a biopsy needle to a target, such as a lesion or nodule in a patient's lung. A needle extending along the length of the endoscope can be deployed along the working channel to obtain a tissue sample for analysis by a pathologist. Based on the pathological findings, additional tools can be deployed along the working channel of the endoscope for additional biopsies. After the nodule is identified as malignant, endoscope 13 can be used to deliver endoscopic tools to remove potentially cancerous tissue. In some cases, diagnostic and therapeutic procedures may need to be delivered in a separate procedure. In those cases, endoscope 13 can also be used to deliver reference objects to “mark” the location of the target nodule. In other cases, diagnostic and therapeutic procedures can be delivered during the same procedure.

[0055] System 10 may also include a movable tower 30, which can be connected to the cart 11 via support cables to provide control, electronics, fluid dynamics, optics, sensors, and / or power support to the cart 11. Placing such functionality in the tower 30 allows for easier adjustment and / or repositioning of the smaller form factor of the cart 11 by the operating physician and his / her staff. Furthermore, the functional allocation between the cart / table and the support tower 30 reduces clutter in the operating room and facilitates improved clinical workflow. While the cart 11 can be positioned close to the patient, the tower 30 can be stowed away in a remote location to avoid obstructing the path during procedures.

[0056] In support of the aforementioned robotic system, tower 30 may include one or more components of a computer-based control system, which stores computer program instructions, for example, in a non-transitory computer-readable storage medium such as a permanent magnetic storage drive or a solid-state drive. The execution of these instructions—whether occurring in tower 30 or in cart 11—can control the entire system or one or more subsystems thereof. For example, when executed by the processor of the computer system, the instructions can cause components of the robotic system to actuate associated brackets and arm mounts, actuate the robotic arm, and control medical devices. For example, in response to receiving a control signal, motors in the joints of the robotic arm can position the arm into a specific posture.

[0057] Tower 30 may also include pumps, flow meters, valve controllers, and / or fluid inlets to provide controlled flushing and suction capabilities to a system that can be deployed via endoscope 13. These components may also be controlled using a computer system of tower 30. In some embodiments, flushing and suction capabilities may be delivered directly to endoscope 13 via (one or more) separate cables.

[0058] Tower 30 may include voltage and surge protectors designed to provide filtered and protected power to trolley 11, thereby avoiding the need to place power transformers and other auxiliary power components in trolley 11, resulting in a smaller and more portable trolley 11.

[0059] Tower 30 may also include support devices for sensors deployed throughout the robotic system 10. For example, tower 30 may include optoelectronic devices for detecting, receiving, and processing data received from optical sensors or cameras throughout the robotic system 10. In conjunction with a control system, such optoelectronic devices can be used to generate real-time images for display in any number of consoles (including those in tower 30) deployed throughout the system. Similarly, tower 30 may also include electronic subsystems for receiving and processing signals received from deployed electromagnetic (EM) sensors. Tower 30 may also be used to house and position an EM field generator for detection by EM sensors in or on a medical device.

[0060] In addition to other consoles available in the rest of the system (e.g., a console mounted on top of the cart), tower 30 may also include console 31. Console 31 may include a user interface and display screen, such as a touchscreen, for the physician operator. Consoles in system 10 are generally designed to provide preoperative and real-time information on robot control and procedures, such as navigation and positioning information for endoscope 13. When console 31 is not the only console available to the physician, it may be used by a second operator, such as a nurse, to monitor the patient's health or vital signs and the operation of the system, as well as to provide process-specific data, such as navigation and positioning information.

[0061] Tower 30 can be coupled to cart 11 and endoscope 13 via one or more cables or connections (not shown). In some embodiments, support functions from tower 30 can be provided to cart 11 via a single cable, thereby simplifying and decluttering the operating room. In other embodiments, specific functionalities can be coupled in separate cables and connections. For example, while power to the cart can be provided via a single power cable, support for control, optics, flow control, and / or navigation can also be provided via separate cables.

[0062] Figure 2 Provided Figure 1 The illustration shows a detailed implementation of a trolley-based robot enabling system. The trolley 11 typically includes an elongated support structure 14 (commonly referred to as a "post"), a trolley base 15, and a control console 16 at the top of the post 14. The post 14 may include supports for one or more robotic arms 12. Figure 2The deployment of one or more brackets, such as bracket 17 (optionally "arm support"), is shown in three. Bracket 17 may include individually configurable arm mounts that rotate along a vertical axis to adjust the base of the robotic arm 12 for better positioning relative to the patient. Bracket 17 also includes a bracket interface 19 that allows the bracket 17 to translate vertically along the post 14.

[0063] The bracket interface 19 is connected to the column 14 via a slot, such as slot 20, which is positioned on the opposite side of the column 14 to guide the vertical translation of the bracket 17. Slot 20 contains a vertical translation interface for positioning and holding the bracket at various vertical heights relative to the trolley base 15. The vertical translation of the bracket 17 allows the trolley 11 to adjust the reach of the robotic arm 12 to accommodate various table heights, patient sizes, and physician preferences. Similarly, individually configurable arm mounts on the bracket 17 allow the robotic arm base 21 of the robotic arm 12 to be configured at various angles.

[0064] In some embodiments, slot 20 may be supplemented with a slot cover flush and parallel to the slot surface to prevent dust and fluid from entering the internal chamber of column 14 and the vertical translation interface during vertical translation of bracket 17. The slot cover can be deployed via a pair of spring reels located near the vertical top and bottom of slot 20. The cover is coiled within the reels until deployed to extend and retract from its coiled state as bracket 17 translates vertically up and down. The spring loading of the reels provides a force to retract the cover back into the reels as bracket 17 translates towards the reels, while maintaining a tight seal as bracket 17 translates away from the reels. The cover can be connected to bracket 17 using, for example, a bracket in bracket interface 19, to ensure proper extension and retraction of the cover during translation of bracket 17.

[0065] The column 14 may internally include mechanisms such as gears and motors, which are designed to mechanically translate the bracket 17 using vertically aligned lead screws in response to control signals generated in response to user input (e.g., input from console 16).

[0066] A robotic arm 12 typically includes a robotic arm base 21 and an end effector 22 separated by a series of links 23 connected by a series of joints 24, each joint including an independent actuator, and each actuator including an independent controllable motor. Each independent controllable joint represents an independent degree of freedom available to the robotic arm. Each arm 12 has seven joints, thus providing seven degrees of freedom. Multiple joints result in multiple degrees of freedom, thereby allowing “redundant” degrees of freedom. Redundant degrees of freedom allow the robotic arm 12 to position its respective end effector 22 at specific locations, orientations, and trajectories in space using different link positions and joint angles. This allows the system to locate and guide medical devices from desired points in space, while allowing physicians to move the arm joints to clinically advantageous positions away from the patient for better access while avoiding arm collisions.

[0067] The trolley base 15 balances the weight of the column 14, bracket 17, and arm 12 on the floor. Therefore, the trolley base 15 houses heavier components, such as electronics, motors, power supplies, and components that enable the trolley to move and / or be secured. For example, the trolley base 15 includes rollable wheel-shaped casters 25 that allow the trolley to be easily moved around the operating room before the procedure. Once in place, the casters 25 can be secured using wheel locks to hold the trolley 11 in place during the procedure.

[0068] The console 16, positioned at the vertical end of column 14, allows both a user interface for receiving user input and a display screen (or dual-purpose device, such as touchscreen 26) to provide both preoperative and intraoperative data to the physician user. Potential preoperative data on touchscreen 26 may include preoperative planning, navigation, and mapping data derived from preoperative computed tomography (CT) scans and / or annotations from preoperative patient interviews. Intraoperative data on the display may include optical information from tools and sensors, coordinate information from sensors, and important patient statistics such as respiration, heart rate, and / or pulse. Console 16 can be positioned and tilted to allow the physician to access the console from the side of column 14 opposite to bracket 17. From this position, the physician can operate console 16 from behind cart 11 while observing console 16, robotic arm 12, and patient. As shown, console 16 also includes handles 27 for assisting in manipulating and stabilizing cart 11.

[0069] Figure 3An embodiment of a robot-enabled system 10 arranged for ureteroscopy is shown. During ureteroscopy, a trolley 11 can be positioned to deliver a ureteroscope 32—a procedure-specific endoscope designed to traverse the patient's urethra and ureter—to the patient's lower abdominal region. During ureteroscopy, it is desirable to align the ureteroscope 32 directly with the patient's urethra to reduce friction and force on sensitive anatomical structures in that region. As shown, the trolley 11 can be aligned at the foot of the table to allow the robotic arm 12 to position the ureteroscope 32 for direct linear access into the patient's urethra. From the foot of the table, the robotic arm 12 can insert the ureteroscope 32 directly through the urethra into the patient's lower abdomen along a virtual track 33.

[0070] After insertion into the urethra, using control techniques similar to those used in bronchoscopy, the ureteroscope 32 can be navigated to the bladder, ureter, and / or kidney for diagnostic and / or therapeutic applications. For example, the ureteroscope 32 can be guided into the ureter and kidney to break up accumulated kidney stones using a laser or ultrasonic lithotripsy device deployed along the working channel of the ureteroscope 32. After lithotripsy is complete, the resulting stone fragments can be removed using a basket deployed along the ureteroscope 32.

[0071] Figure 4 A similar implementation of a robot-enabled system arranged for vascular procedures is shown. During a vascular procedure, system 10 can be configured such that a trolley 11 can deliver a medical device 34, such as a manipulable catheter, to an entry point in the femoral artery in the patient's leg. The femoral artery presents both a relatively large diameter for navigation and a relatively less circuitous and tortuous path to the patient's heart, which simplifies navigation. As during a ureteroscopy, trolley 11 can be positioned toward the patient's leg and lower abdomen to allow the robotic arm 12 to provide a virtual track 35 for direct, linear access to the femoral artery entry point in the patient's thigh / hip region. After artery insertion, the medical device 34 can be guided and inserted via a translational instrument actuator 28. Alternatively, the trolley can be positioned around the patient's upper abdomen to reach alternative vascular entry points, such as the carotid and brachial arteries near the shoulder and wrist.

[0072] B. Robot system - unit.

[0073] The implementation of robot-enabled medical systems can also incorporate patient tables. By removing the trolley, the incorporation of tables reduces the amount of major equipment in the operating room, allowing for better patient access. Figure 5An embodiment of such a robot-enabled system arranged for a bronchoscopy procedure is shown. System 36 includes a support structure or column 37 for supporting a platform 38 (shown as a "table" or "bed") on the floor. Much like a trolley-based system, the end effector of the robotic arm 39 of system 36 includes an instrument actuator 42, which is designed to manipulate elongated medical instruments, such as [instrument name missing], via or along a virtual track 41 formed by the linear alignment of the instrument actuator 42. Figure 5 The bronchoscope 40 is used. In practice, a C-arm for providing fluoroscopic imaging can be positioned above the patient's upper abdominal region by placing the transmitter and detector around the stage 38.

[0074] Figure 6 Alternative views of system 36 without a patient and medical instruments are provided for discussion purposes. As shown, column 37 may include one or more brackets 43, shown as annular in system 36, from which one or more robotic arms 39 may be based. The brackets 43 may translate along a vertical column interface 44 extending along the length of column 37 to provide different vantage points from which the robotic arms 39 may be positioned to reach the patient. One or more brackets 43 may be rotated about column 37 using mechanical motors positioned within column 37 to allow the robotic arms 39 to approach multiple sides of table 38, such as, for example, the sides of the patient. In embodiments with multiple brackets, the brackets may be individually positioned on the column and may translate and / or rotate independently of the other brackets. While the brackets 43 need not be about column 37 or even circular, the annular shape shown facilitates rotation of the brackets 43 about column 37 while maintaining structural balance. The rotation and translation of the bracket 43 allows the system to align medical instruments such as endoscopes and laparoscopes to different entry points on the patient.

[0075] Arm 39 can be mounted on a bracket via a set of arm mounts 45 comprising a series of joints, which can be individually rotated and / or telescopically extended to provide additional configurability to the robotic arm 39. Additionally, arm mounts 45 can be positioned on a bracket 43 such that, when the bracket 43 is properly rotated, arm mounts 45 can be positioned on the same side of the stage 38 (e.g., ...). Figure 6 As shown in the diagram), it is positioned on the opposite side of platform 38 (as shown in the diagram). Figure 9 (as shown in the figure), or positioned on the adjacent side of platform 38 (not shown).

[0076] Column 37 structurally provides support for platform 38 and a path for the vertical translation of the bracket. Internally, column 37 may be equipped with guide screws for guiding the vertical translation of the bracket and motors that mechanize the translation of the bracket based on the guide screws. Column 37 may also transmit electrical and control signals to bracket 43 and robotic arm 39 mounted on bracket 43.

[0077] Taiwan base 46 and Figure 2 The trolley base 15 in the trolley 11 shown serves a similar function, accommodating heavier components to balance the table / bed 38, column 37, bracket 43, and robot arm 39. The table base 46 may also incorporate rigid casters for stability during the process. Casters deployed from the bottom of the table base 46 can extend in opposite directions on either side of the base 46 and retract when the system 36 needs to be moved.

[0078] continue Figure 6 System 36 may also include a tower (not shown) that distributes the functionality of system 36 between the table and the tower to reduce the form factor and volume of the table. As in previously disclosed embodiments, the tower may provide the table with various support functions, such as processing, computing and control capabilities, power, flow control and / or optics, and sensor processing. The tower may also be movable and positioned away from the patient to improve accessibility for physicians and keep the operating room uncluttered. Additionally, placing components in the tower allows for more storage space in the base of the table for potential deployment and retraction of a robotic arm. The tower may also include a console that provides both a user interface for user input, such as a keyboard and / or a suspended manipulator, and a display screen (or touchscreen) for preoperative and intraoperative information, such as real-time imaging, navigation, and tracking information.

[0079] In some implementations, the platform base can retract and store the robotic arm when not in use. Figure 7 A system 47 for retracting and deploying a robotic arm is illustrated in an embodiment of a platform-based system. In system 47, a bracket 48 can be vertically translated into a base 49 to house the robotic arm 50, arm mount 51, and bracket 48 within the base 49. A base cover 52 can be translated and retracted to open to deploy the bracket 48, arm mount 51, and arm 50 around a post 53, and can be closed to retract and deploy the bracket 48, arm mount 51, and arm 50 to protect them when not in use. The base cover 52 can be sealed with a membrane 54 along the edge of the opening of the base cover 52 to prevent dust and fluid from entering when closed.

[0080] Figure 8An embodiment of a robot-enabled table-based system configured for a ureteroscopy procedure is illustrated. In the ureteroscopy, table 38 may include a rotating portion 55 for positioning the patient at an angle to the column 37 and table base 46. The rotating portion 55 may rotate or pivot about a pivot point (e.g., below the patient's head) to position the bottom portion of the rotating portion 55 away from the column 37. For example, pivoting the rotating portion 55 allows a C-arm (not shown) to be positioned above the patient's lower abdomen without competing for space with the column (not shown) below table 38. By rotating a bracket 35 (not shown) about the column 37, a robotic arm 39 can insert a ureteroscope 56 directly into the patient's groin region along a virtual track 57 to reach the urethra. During the ureteroscopy, stirrups 58 may also be fixed to the rotating portion 55 of table 38 to support the position of the patient's legs during the procedure and to allow unobstructed access to the patient's groin region.

[0081] During laparoscopy, minimally invasive instruments (elongated in shape to fit the size of one or more incisions) are inserted into the patient's anatomy through one or more small incisions in the abdominal wall. After the patient's abdominal cavity is expanded, the instruments, commonly known as laparoscopes, can be oriented to perform surgical tasks such as grasping, cutting, resection, and suturing. Figure 9 An implementation of a robot-enabled platform-based system configured for laparoscopic procedures is shown. Figure 9 As shown, the bracket 43 of system 36 can be rotated and vertically adjusted to position the pair of robotic arms 39 on opposite sides of table 38, so that the laparoscope 59 can be positioned through small incisions on both sides of the patient to reach his / her abdominal cavity using arm mount 45.

[0082] To accommodate the laparoscopic procedure, the robot-enabled platform system can also tilt the platform to the desired angle. Figure 10 An implementation of a robot-enabled medical system with pitch or tilt adjustment is shown. For example... Figure 10 As shown, system 36 can adapt to the tilt of platform 38 to position one part of the platform at a greater distance from the ground than another part. Additionally, arm mount 45 can rotate to match the tilt, ensuring that arm 39 maintains the same planar relationship with platform 38. To accommodate steeper angles, column 37 may also include a telescopic portion 60 that allows vertical extension of column 37 to prevent platform 38 from contacting the floor or colliding with base 46.

[0083] Figure 11Detailed illustrations of the interface between stage 38 and column 37 are provided. The pitch-rotation mechanism 61 can be configured to change the pitch angle of stage 38 relative to column 37 with multiple degrees of freedom. The pitch-rotation mechanism 61 is achieved by positioning orthogonal axes 1 and 2 at the column-stage interface, each axis being actuated by separate motors 3 and 4 in response to electrical pitch angle commands. Rotation along one screw 5 achieves tilt adjustment along one axis 1, while rotation along another screw 6 achieves tilt adjustment along the other axis 2.

[0084] For example, pitch adjustment is particularly useful when attempting to position the table in a trendelenburg position—that is, positioning the patient's lower abdomen higher than the floor for lower abdominal surgery. The trendelenburg position allows the patient's internal organs to slide down to their upper abdomen due to gravity, thus clearing the abdominal cavity to allow minimally invasive instruments to enter and perform lower abdominal surgical procedures, such as laparoscopic prostatectomy.

[0085] C. Instrument drivers and interfaces.

[0086] The end effectors of the system's robotic arm include: (i) an instrument actuator (alternatively referred to as an "instrument drive mechanism" or "instrument device manipulator"), which incorporates electromechanical devices for actuating the medical device; and (ii) a removable or detachable medical device, which may lack any electromechanical components such as motors. This dichotomy may be driven by the need to sterilize medical devices used in medical procedures, and the inability to adequately sterilize expensive capital equipment due to its complex mechanical components and sensitive electronics. Therefore, medical devices can be designed to be detached, removed, and interchanged from the instrument actuator (and thus from the system) for individual sterilization or disposal by a physician or physician staff. In contrast, the instrument actuator does not need to be altered or sterilized and can be draped for protection.

[0087] Figure 12 An example instrument actuator is shown. The instrument actuator 62, positioned at the end of a robotic arm, includes one or more drive units 63 arranged along parallel axes to provide controlled torque to a medical device via a drive shaft 64. Each drive unit 63 includes: a separate drive shaft 64 for interacting with the device; a gear head 65 for converting rotation of a motor shaft into a desired torque; a motor 66 for generating the drive torque; an encoder 67 for measuring the speed of the motor shaft and providing feedback to a control circuit system; and control circuitry 68 for receiving control signals and actuating the drive unit. Each drive unit 63 is independently controlled and motorized, and the instrument actuator 62 can provide multiple (e.g., ...) to the medical device. Figure 12The diagram shows four independent drive outputs. In operation, the control circuit 68 receives control signals, sends motor signals to the motor 66, compares the motor speed, as measured by the encoder 67, with the desired speed, and modulates the motor signals to generate the desired torque.

[0088] For processes requiring a sterile environment, robotic systems can incorporate a drive interface between the instrument actuator and the medical device, such as a sterile adapter connected to a sterile drape. The primary purpose of the sterile adapter is to transmit angular motion from the drive shaft of the instrument actuator to the drive input of the device, while maintaining physical separation between the drive shaft and the drive input, thus preserving sterility. Therefore, an example sterile adapter may include a series of rotary inputs and outputs designed to mate with the drive shaft of the instrument actuator and the drive input on the device. The sterile drape, consisting of a thin, flexible material such as transparent or translucent plastic, is designed to cover the capital device, such as the instrument actuator, robotic arm, and trolley (in trolley-based systems) or table (in table-based systems). The use of this drape allows the capital device to be positioned near the patient while still within an area that does not require sterilization (i.e., a non-sterile area). On the other side of the sterile drape, the medical device can dock with the patient in an area that requires sterilization (i.e., a sterile area).

[0089] D. Medical devices.

[0090] Figure 13 An example medical device with paired instrument actuators is shown. Like other devices designed for use with robotic systems, the medical device 70 includes an elongated shaft 71 (or elongated body) and an instrument base 72. The instrument base 72, also referred to as an "instrument handle" due to its intended design for manual interaction by a physician, typically includes a rotatable drive input 73, such as a receptacle, pulley, or reel, designed to mate with a drive output 74 extending through a drive interface on an instrument actuator 75 at the end of a robotic arm 76. When physically connected, latched, and / or coupled, the mating drive input 73 of the instrument base 72 can share an axis of rotation with the drive output 74 in the instrument actuator 75 to allow torque to be transmitted from the drive output 74 to the drive input 73. In some embodiments, the drive output 74 may include a spline designed to mate with a receptacle on the drive input 73.

[0091] The elongated shaft 71 is designed to be delivered through anatomical openings or cavities, such as those in endoscopy, or through minimally invasive incisions, such as those in laparoscopy. The elongated shaft 66 can be flexible (e.g., having endoscope-like properties) or rigid (e.g., having laparoscopy-like properties), or a customized combination of both flexible and rigid portions. When designed for laparoscopy, the end of the rigid elongated shaft can be connected to an end effector and a surgical tool such as a gripper or scissors, the end effector comprising an articulated wrist formed by a U-shaped clamp having a rotation axis, the surgical tool being actuated based on forces from a tendon when the drive input rotates in response to torque received from the drive output 74 of the instrument driver 75. When designed for endoscopy, the end of the flexible elongated shaft can include a manipulable or controllable bending segment that can be engaged and bent based on torque received from the drive output 74 of the instrument driver 75.

[0092] Torque from the instrument actuator 75 is transmitted along the elongated shaft 71 using tendons within the shaft 71. These individual tendons, such as traction cables, can be individually anchored to individual drive inputs 73 within the instrument handle 72. From the handle 72, the tendons are guided along one or more traction chambers within the elongated shaft 71 and anchored at the distal portion of the shaft 71. In laparoscopy, these tendons can be coupled to end-mounted actuators such as wrists, grippers, or scissors. In such an arrangement, the torque applied to the drive input 73 transmits tension to the tendons, thereby actuating the end-mounted actuator in a certain way. In laparoscopy, the tendons can rotate the joint about the axis, thereby moving the end-mounted actuator in one direction or the other. Alternatively, the tendons can be connected to one or more jaws of a gripper at the distal end of the elongated shaft 71, where tension from the tendons causes the gripper to close.

[0093] During endoscopic examination, tendons can be coupled via adhesives, control rings, or other mechanical fasteners to a curved or engaging segment positioned along an elongated axis 71 (e.g., at the end). When securely attached to the end of the curved segment, torque applied to a drive input 73 is transmitted along the tendon, causing the softer curved segment (sometimes referred to as the engaging segment or region) to bend or engage. Along the non-curved segment, it can be advantageous to cause individual traction cavities to spiral or helically guide the tendons along the wall of the endoscope axis (or within the wall of the endoscope axis) to balance the radial forces generated by tension in the traction line. For a particular purpose, the angle of spiral and / or the spacing between them can be varied or designed, wherein a steeper spiral results in less axial compression under load, while a smaller spiral results in greater axial compression under load and exhibits limited bending. At the other end of the spectrum, the traction cavities can be oriented parallel to the longitudinal axis of the elongated axis 71 to allow controlled engagement in the desired curved or engaging segment.

[0094] In endoscopic procedures, an elongated shaft 71 accommodates multiple components to assist in robotic procedures. The shaft may include a working channel for deploying surgical instruments, irrigation, and / or aspiration to the surgical area at the distal end of the shaft 71. The shaft 71 may also accommodate wires and / or optical fibers to transmit signals to / from an optical assembly at the distal tip, which may include an optical camera. The shaft 71 may also accommodate optical fibers to propagate light from a proximal light source, such as a light-emitting diode, to the distal end of the shaft.

[0095] At the distal end of the instrument 70, the distal tip may also include an opening for delivering tools for diagnostic and / or treatment, irrigation, and aspiration to the surgical site. The distal tip may also include a port for capturing images of the internal anatomical space, such as a fiberoptic endoscope or a digital camera. Relatedly, the distal tip may also include a port for illuminating the anatomical space when using the camera.

[0096] exist Figure 13 In the example, the drive shaft axis, and therefore the drive input axis, is orthogonal to the axis of the slender shaft. However, this arrangement complicates the rolling maneuverability of the slender shaft 71. When the tendon extends from the drive input 73 and enters the traction cavity within the slender shaft 71, rolling the slender shaft 71 along its axis while keeping the drive input 73 stationary can lead to undesirable tangling of the tendon. Such tangling of the tendon can disrupt any control algorithm designed to predict the movement of the flexible slender shaft during endoscopic procedures.

[0097] Figure 14Alternative designs for instrument actuators and instruments are shown, wherein the axes of the drive units are parallel to the axis of the slender shaft of the instrument. As shown, a circular instrument actuator 80 includes four drive units, wherein the drive outputs 81 of the four drive units are aligned parallel to each other at the end of the robot arm 82. The drive units and their respective drive outputs 81 are housed in a rotating assembly 83 of the instrument actuator 80, which is driven by one of the drive units within the assembly 83. In response to torque provided by the rotating drive unit, the rotating assembly 83 rotates along a circular bearing that connects the rotating assembly 83 to the non-rotating portion 84 of the instrument actuator. Electrical and control signals can be transmitted from the non-rotating portion 84 of the instrument actuator 80 to the rotating assembly 83 via electrical contacts, which can be maintained by rotation via brush slip ring connections (not shown). In other embodiments, the rotating assembly 83 may be responsive to a separate drive unit integrated into a non-rotating portion 84 and therefore not parallel to the other drive units. The rotating mechanism 83 allows the instrument driver 80 to rotate the drive unit and its respective drive output 81 as a single unit around the instrument driver axis 85.

[0098] As with the previously disclosed embodiments, the instrument 86 may include an elongated shaft portion 88 and an instrument base 87 (shown as having a transparent housing for discussion purposes), the instrument base 87 including a plurality of drive inputs 89 (e.g., receptacles, pulleys, and reels) configured to receive drive outputs 81 from the instrument driver 80. Unlike the previously disclosed embodiments, the instrument shaft 88 extends from the center of the instrument base 87, wherein the axis of the instrument shaft 88 is substantially parallel to the axes of the drive inputs 89, rather than as... Figure 13 It is orthogonal as in the design.

[0099] When coupled to the rotating assembly 83 of the instrument driver 80, the medical instrument 86, including the instrument base 87 and the instrument shaft 88, rotates about the instrument driver axis 85 in conjunction with the rotating assembly 83. Since the instrument shaft 88 is positioned at the center of the instrument base 87, it is coaxial with the instrument driver axis 85 when attached. Therefore, rotation of the rotating assembly 83 causes the instrument shaft 88 to rotate about its own longitudinal axis. Furthermore, when the instrument base 87 rotates together with the instrument shaft 88, any tendons connected to the drive input 89 in the instrument base 87 do not become entangled during rotation. Therefore, the parallel structure of the axes of the drive output 81, drive input 89, and instrument shaft 88 allows the shaft to rotate without causing any control tendons to become entangled.

[0100] E. Navigation and control.

[0101] Traditional endoscopy can involve the use of fluorescence imaging (e.g., delivered via a C-arm) and other forms of radiation-based imaging modalities to provide intracavitary guidance to the operator. In contrast, the robotic system envisioned by this disclosure can provide radiation-free navigation and positioning, reducing physician radiation exposure and the amount of equipment required in the operating room. As used herein, the term "positioning" can refer to determining and / or monitoring the location of an object in a reference coordinate system. Techniques such as preoperative mapping, computer vision, real-time EM tracking, and robot command data can be used, individually or in combination, to achieve a radiation-free operating environment. In other cases where radiation-based imaging modalities are still used, preoperative mapping, computer vision, real-time EM tracking, and robot command data can be used, individually or in combination, to improve upon information obtained solely through radiation-based imaging modalities.

[0102] Figure 15 This is a block diagram illustrating a positioning system 90 for estimating the position of one or more components of a robotic system, such as the position of an instrument, according to an exemplary embodiment. The positioning system 90 may be a group of one or more computer devices configured to execute one or more instructions. The computer devices may be embodied by a processor (or multiple processors) and computer-readable storage from one or more components discussed above. By way of example and not limitation, the computer devices may... Figure 1 Tower 30 shown Figures 1 to 4 The cart shown Figures 5 to 10 The bed shown is among them.

[0103] like Figure 15 As shown, the positioning system 90 may include a positioning module 95 that processes input data 91 to 94 to generate position data 96 for the distal tip of the medical device. The position data 96 may be data or logic representing the position and / or orientation of the device's distal end relative to a reference frame. The reference frame may be relative to the patient's anatomy or relative to a known object such as an EM field generator (see the discussion below regarding EM field generators).

[0104] The individual input data 91 to 94 are now described in more detail. Preoperative mapping can be accomplished using a set of low-dose CT scans. The preoperative CT scans are reconstructed into three-dimensional images, which are visualized, for example, as “slices” of cross-sectional views of the patient’s internal anatomy. During the overall analysis, an image-based model of the anatomical cavities, spaces, and structures of the patient’s anatomical structures, such as the patient’s lung network, can be generated. Techniques such as centerline geometry can be determined and approximated from the CT images to develop a three-dimensional volume of the patient’s anatomy, referred to as the preoperative model data 91. The use of centerline geometry is discussed in U.S. Patent Application No. 14 / 523,760, the entire contents of which are incorporated herein by reference. Network topology models can also be derived from CT images and are particularly well-suited for bronchoscopy.

[0105] In some implementations, the instrument may be equipped with a camera to provide visual data 92. The positioning module 95 may process the visual data to enable one or more vision-based position tracking methods. For example, preoperative model data may be used in conjunction with visual data 92 to enable computer vision-based tracking of a medical instrument (e.g., an endoscope or the movement of an instrument through the working channel of an endoscope). For example, using preoperative model data 91, the robotic system may generate a library of desired endoscopic images from the model based on the desired path of the endoscope's movement, each image linked to a location within the model. During surgery, this library may be referenced by the robotic system to compare real-time images captured at a camera (e.g., a camera at the end of the endoscope) with images in the image library to aid in positioning.

[0106] Other computer vision-based tracking techniques use feature tracking to determine camera motion, and thus, endoscope motion. Certain features in the localization module 95 can identify circular geometries corresponding to anatomical cavities in the preoperative model data 91 and track changes in those geometries to determine which anatomical cavity is selected, as well as track the relative rotation and / or translational motion of the camera. The use of a topology map can further enhance vision-based algorithms or techniques.

[0107] Optical flow (another computer vision-based technique) can analyze the displacement and translation of image pixels in a video sequence within visual data 92 to infer camera movement. By comparing multiple frames in several iterations, the movement and position of the camera (and therefore the endoscope) can be determined.

[0108] The positioning module 95 can use real-time EM tracking to generate the real-time position of the endoscope in a global coordinate system, which can be registered with the patient's anatomy represented by a preoperative model. In EM tracking, an EM sensor (or tracker) including one or more sensor coils embedded in one or more locations and orientations of the medical instrument (e.g., an endoscopic tool) measures changes in the EM field created by one or more static EM field generators positioned at known locations. The positional information detected by the EM sensor is stored as EM data 93. The EM field generator (or transmitter) can be placed close to the patient to create a low-intensity magnetic field detectable by the embedded sensor. The magnetic field induces a small current in the sensor coil of the EM sensor, which can be analyzed to determine the distance and angle between the EM sensor and the EM field generator. These distances and orientations can be "registered" with the patient's anatomy (e.g., a preoperative model) during surgery to determine the geometric transformations that align a single position in the coordinate system with its position in the preoperative model of the patient's anatomy. Once registered, embedded EM trackers in one or more locations of the medical device (e.g., the distal tip of an endoscope) can provide real-time indication of the medical device’s progress through the patient’s anatomy.

[0109] Robot commands and kinematic data 94 can also be used by the positioning module 95 to provide positioning data 96 for the robot system. Device pitch and yaw generated by engagement commands can be determined during preoperative calibration. During surgery, these calibration measurements can be used in conjunction with known insertion depth information to estimate the instrument's position. Alternatively, these calculations can be combined with EM, vision, and / or topology modeling for analysis to estimate the medical device's position within the network.

[0110] like Figure 15 As shown, several other input data can be used by the positioning module 95. For example, although in Figure 15 Although not shown, the device using shape sensing fiber can provide shape data, which the positioning module 95 can use to determine the position and shape of the device.

[0111] The localization module 95 can use input data 91 to 94 in combination (one or more). In some cases, such combination may use a probabilistic method, where the localization module 95 assigns confidence weights to each determined location based on the input data 91 to 94. Therefore, in cases where EM data may be unreliable (e.g., where EM interference may be present), the confidence of the location determined by EM data 93 may be reduced, and the localization module 95 may rely more heavily on visual data 92 and / or robot commands and kinematic data 94.

[0112] As discussed above, the robotic systems discussed herein can be designed to combine one or more of the above technologies. Computer-based control systems for tower, bed, and / or trolley-based robotic systems can store computer program instructions, for example, in non-transitory computer-readable storage media such as permanent magnetic storage drives or solid-state drives. When executed, these computer program instructions enable the system to receive and analyze sensor data and user commands, generate control signals for the entire system, and display navigation and positioning data, such as the position of the instrument in a global coordinate system, anatomical diagrams, etc.

[0113] 2. Medical device driven.

[0114] Embodiments of this disclosure relate to systems and techniques for driving medical devices having inner and outer bodies. For example, a medical device may include two or more scalable bodies that can be independently driven (e.g., forward, retraction, engagement, rotation, etc.) based on commands received from a user of the system. As the number of independently controllable bodies increases, the number of commands that can be independently mapped to control the driving of the medical device also increases, thereby increasing the complexity of the system. Furthermore, each body may be configured with multiple degrees of freedom that can be used to control the respective body, resulting in a greater complexity in the number of control variables. Higher medical device control complexity may be associated with a larger number of available driving techniques that may not be achievable with less complex driving systems.

[0115] For example, a standard single-body endoscope can be configured to provide the user with multiple degrees of freedom, such as insertion, rolling, and engagement in various directions. In an example multi-body endoscope including an inner and outer body, the system can provide 10 degrees of freedom (e.g., for each body, the degrees of freedom could include one insertion degree of freedom and four independent traction lines, each traction line providing an independent engagement degree of freedom). Other implementations can include more or fewer degrees of freedom, which can provide, for example, rolling in one or more of the outer and inner bodies. With 10 degrees of freedom, there can be many different combinations of techniques for achieving similar actuation functionality. Additionally, some techniques may have advantages over others, such as reducing stress or wear on the medical device, thus allowing the medical device or its components to be used for a longer period before needing replacement.

[0116] While certain aspects of this disclosure can be described within the context of a dual-body system including an outer and an inner body, this disclosure is not limited to dual-body medical devices. For example, a medical device may also include a robot-controlled surgical instrument configured to be driven through a cavity in the inner body. The surgical instrument may be attached to a third robotic arm assembly for independent control. Therefore, those skilled in the art will recognize that the concepts described below, generally applicable to medical devices including an outer and an inner body, can also be applied to three-body systems or systems including a greater number of independently driven bodies.

[0117] To reduce the cognitive load on the user, the system may include a user input device configured to receive input commands with fewer degrees of freedom than those achievable by the structure of the medical device. For example, the user input device may be configured to receive user commands mapped to three degrees of freedom (e.g., insertion, yaw, and pitch). The system can then map these user commands to robot commands corresponding to the physical degrees of freedom of the inner and outer bodies used to control the medical device. Furthermore, this mapping may involve determining which of the inner and outer bodies to apply the command to, and the command may involve driving both the inner and outer bodies and / or coordinating their movement in a sequential or coordinated manner in some applications.

[0118] Figure 16 An embodiment of a surgical robot system, which can be configured to drive a medical device having an outer body and an inner body, according to aspects of this disclosure is shown. Although Figure 16 This disclosure relates to embodiments involving the attachment of one or more robotic arms and / or instrument manipulators to a trolley, but is not limited thereto, and the techniques described herein are as follows: Figure 6 The illustration shows one or more applicable robotic arms and / or instrument manipulators that can be attached to a column supporting a patient platform.

[0119] Return to Figure 16 The illustration shows a system 100 that may include a trolley 105, one or more instrument manipulators 115 and 125, and a medical device 130. The trolley 105 may include a processor (not shown), a memory (not shown), a display 107 configured to present coded data related to navigation and / or drive of the medical device 130, and a set of one or more user input devices 150 (e.g., a suspension console, a main controller, or other user input controller). However, depending on the implementation, one or more of the processor, memory, and display 107 may be located on or within a separate device, such as... Figure 1The movable tower 30 shown is located on or within the movable tower 30. Alternatively, in other implementations, a feedback device other than the display 107 may be used in place of or attached to the display 107. Other possible feedback devices include tactile devices, speakers, force feedback actuated via one or more of the actuators 115 and 125, one or more light-emitting diodes (LEDs), etc.

[0120] In some implementations, one or more user input devices 150 include one or more joysticks 151 and switching inputs 153 (such as buttons). The joysticks can be used as inputs capable of generating commands for insertion and / or retraction of the medical device and engagement of the engaging portion of the medical device 130. The system can use the switching inputs 153 to generate commands to change drive modes (e.g., switch drive mode commands) to switch between various drive modes of the medical device 130. Further details regarding drive modes and the conditions for changing or switching between drive modes are provided below.

[0121] Instrument manipulators 115 and 125 may include a first instrument manipulator and a second instrument manipulator 125, respectively driven by a first robotic arm 110 and a second robotic arm 120. However, aspects of this disclosure also apply to systems having one or more instrument manipulators 115 and 125, which may be driven by other actuation mechanisms different from the first robotic arm 110 and the second robotic arm 120. As used herein, the term instrument manipulator (also referred to as an instrument device manipulator (IDM)) generally refers to a component that provides a detectable connection to a medical device (or a portion thereof). IDMs (such as...) Figure 12 The drive unit 63 shown can be configured to control the movement and / or manipulation of a medical device (including any end effectors attached to the medical device). A first device manipulator 115 can be attached to the end of a first robotic arm 110 and a second device manipulator can be attached to the end of a second robotic arm 120. By actuating one or more motors of the first robotic arm 110, the motors can be operated to adjust the posture or position of the first robotic arm 110, and thus adjust the device manipulator 115 (e.g., by adjusting the position and / or orientation of one or more joints 113 of the first arm), and thereby control the manipulatorable device 130 attached to the device manipulator 115. Similar to the first robotic arm 110, the second robotic arm 120 can be operated to drive the second device manipulator 125 to manipulate the manipulatorable device 130.

[0122] Figure 16The medical device 130 in this embodiment includes an outer body 131 attached to a first instrument manipulator 115 and an inner body 133 attached to a second instrument manipulator 125 120. However, Figure 16 The example shown is merely one example of a medical device 130, and other embodiments may include a medical device 130 controlled by a single instrument manipulator 115 or a medical device 130 requiring three or more instrument manipulators for operation. Depending on the embodiment and the medical procedure being performed, each of the first and second medical devices may include one of an inner guide portion, an outer sheath portion, a needle, forceps, a brush, etc.

[0123] The outer body 131 and inner body 133 can be configured to advance / insert into (or retract from) the patient's body along a first axis 140. As described above, the first axis 140 can be referred to as a virtual track. The virtual track can be defined by the aligned axes of the instrument manipulators 115 and 125, and therefore can also coincide with the central axis of the medical device 130. Movement of the first instrument manipulator 115 and the second instrument manipulator 125 along the virtual track 140 can control the advance of the outer body 131 and inner body 133 into the patient's body and their retraction from the patient's body.

[0124] In one embodiment, one of the joysticks 151 is used to control the engagement of the medical device 130, including engagement of one or more of the inner and outer bodies 131 and 133, and the other of the joysticks 151 is used to control the engagement of the medical device 130, including engagement of one or more of the inner and outer bodies 131. Depending on the actuation mode, input received from the joysticks 151 may be mapped to both the inner and outer bodies 131 and 133, or may be mapped to only one of the inner and outer bodies 131 and 133 each time.

[0125] A. Example medical devices and drive modes.

[0126] Figure 17A Embodiments of a medical device according to aspects of this disclosure are depicted. The illustrated medical device 200 includes an outer body 210 (also referred to as a sheath) and an inner body 220 (also referred to as a guide). In some implementations, the robotic surgical system may include a group of one or more robotic arm assemblies configured to control the movement of the outer and inner bodies. For example, the outer body 210 and inner body 220 may be coupled to instrument manipulators connected to the robotic arms (see, for example, Figure 16Therefore, the outer and inner bodies 210 and 220 can be independently driven via manipulation of the respective robotic arms and mechanical manipulators connected to the robotic arms. The outer body 210 may also define a cavity 215, through which the inner body 220 is configured to be driven.

[0127] Figure 17B An embodiment of a paired drive mode for a medical device according to aspects of this disclosure is illustrated. The system can be configured to drive the medical device in one of a plurality of drive modes, which may include paired drive modes and at least one unpaired drive mode.

[0128] Figure 18 This is a flowchart illustrating an example method for changing the drive mode of a medical device, operable by a surgical robot system or its components (one or more) according to aspects of this disclosure. For example, Figure 18 The steps of method 1600 shown can be executed by the processor of the surgical robot system. For convenience, method 1600 is described as being executed by the system's processor.

[0129] Method 1600 begins at block 1601. A processor may be included as part of a system comprising a medical device having an outer body and an inner body configured to be driven through a cavity in the outer body, a set of one or more instrument manipulators configured to control movement of the outer and inner bodies (e.g., instrument manipulators may be coupled to a robotic arm assembly), a set of one or more user input devices, a set of one or more processors, and at least one computer-readable storage device communicating with the set of processors and storing computer-executable instructions on the computer-readable storage device to cause the set of processors to perform method 1600.

[0130] At box 1605, the processor accesses the user input device (e.g., Figure 16 The user input device 150 receives a command to change the drive mode. The system can also be configured to change the drive mode of the medical device 200 from a paired drive mode to an unpaired drive mode in response to receiving the command. Therefore, at block 1610, in response to receiving the command, the processor changes the drive mode of the medical device from a paired drive mode to an unpaired drive mode, wherein, in response to receiving a drive command from the set of input devices in the paired drive mode, the distance between the end of the inner body and the end of the outer body is maintained at a predetermined distance. Method 1600 ends at block 1615.

[0131] Continue to refer to Figure 16 , Figure 17A and Figure 17BIn some implementations, in a paired drive mode, the distance between the end of the inner body 220 and the end of the outer body 210 is maintained at a predetermined distance 240. The predetermined distance 240 between the ends of the outer body 210 and the inner body 220 can be maintained in response to receiving a drive command from the set of input devices. The drive command may include one or more commands that manipulate the medical device. For example, the drive command may include one or more of the following commands: insertion commands (e.g., advancing or retracting the medical device) and / or engagement commands (e.g., increasing or decreasing the bending or engagement of the end of the medical device). It should be understood that the term "distance" as used herein may refer to a single length (e.g., 5 mm) or a range of lengths (5 mm to 8 mm).

[0132] Refer again Figure 17B As the medical device 200 is inserted along the insertion direction 230, multiple snapshots of the medical device 200 are shown sequentially from top to bottom. Dashed lines are provided relative to the initial position of a predetermined distance 240 between the ends of the outer body 210 and the inner body 220. The predetermined distance 240 is maintained between the ends of the outer body 210 and the inner body 220 as the medical device 200 advances through the snapshots shown. In paired drive mode, the predetermined distance 240 can also be maintained in response to other manipulation commands. For example, the predetermined distance 240 can be maintained in response to both a retraction command and an engagement command.

[0133] like Figure 17A and Figure 17B As shown, the ends of the inner body 220 and the outer body 210 may be chamfered. These chamfered edges allow the ends of the inner body 220 and the outer body 210 to advance more easily through the cavity network during the medical procedure. Additionally, when the inner body 220 includes chamfered edges, certain advantages can exist in maintaining a predetermined distance 240 such that the end of the inner body 220 extends from the end of the outer body 210. For example, a chamfer on each end of the outer body 210 and the inner body 220 allows the end to advance more easily along the cavity network without getting stuck on features of the cavity network walls. If the end of the inner body 220 does not extend from the end of the outer body 210, the advantages associated with the chamfer on the end of the inner body 220 cannot be utilized.

[0134] The predetermined distance 240 can be selected based on one or more considerations of the physical structure of the medical device 240 and / or the robotic arm. For example, the length of each of the outer body 210 and the inner body 220 may have a certain amount of manufacturing variation, which may cause the distance between the ends of the outer body 210 and the inner body 220 to shift relative to the corresponding distance between the ends of the outer body and the inner body without such manufacturing variation. The lengths of the outer body 210 and the inner body 200 may also change over time, such as shrinking in length, which may cause a difference between the distance between the ends of the outer body 210 and the inner body 220 calculated by the system and the actual distance between the ends.

[0135] Furthermore, when the distance between the end of the outer body 210 and the end of the inner body 220 is less than a threshold distance, image data captured by a camera formed on the end of the inner body may be occluded by the outer body 210, and the outer body 210 may be visible in the image captured by the camera. Therefore, it may be desirable to extend the end of the inner body 220 from the end of the outer body 210 so that the end of the outer body 220 is invisible in the captured image.

[0136] Therefore, the predetermined distance 240 can be selected to be greater than the tolerance for such manufacturing differences in length, such that when loaded onto the manipulator of the robotic arm, the end of the inner body 210 will extend from the end of the outer body 220. In other implementations, the predetermined distance 240 can also be selected to account for any differences in the placement of the manipulator relative to the commanded placement, which may have tolerance errors for placement. Therefore, the predetermined distance 240 can also account for differences in the distance between the ends of the outer body 210 and the inner body 220 introduced due to differences in the placement of the manipulator and the commanded position.

[0137] Another consideration for selecting the predetermined distance 240 may include the distance between the ends of the outer body 210 and the inner body 220, exceeding which might hinder the insertion of the medical device 200. For example, as described below, the outer body 210 and the inner body 220 may be engaged together to provide a greater engagement than could be achieved by either body 210 or 200 alone. As used herein, the co-engagement of the outer body 210 and the inner body 220 in the same direction may generally be referred to as “co-engagement.” When the end of the inner body 220 extends too far from the end of the outer body 210, the co-engagement of the medical device 200 may produce undesirable wobbling, which may be difficult for the user to manipulate effectively. Therefore, the predetermined distance 240 may be selected to be less than a threshold at which co-engagement results in undesirable wobbling.

[0138] In some embodiments, the end of the inner body 220 can be retracted into the outer body 210, such that the distance between the ends of the outer body 210 and the inner body 220 is maintained at a predetermined distance. By keeping the end of the inner body 220 within a cavity defined in the outer body 210, wear on the inner body 220 can be reduced, thereby increasing the lifespan of the inner body 220. In some implementations, it may be desirable to drive the medical device 200 with the end of the inner body 220 retracted into the outer body 210. For example, when the inner body 220 includes a sharp edge such as a needle, driving it while it extends from the outer body 210 may result in unintentional injury to the patient. When driven with the inner body 220 retracted, the medical device 200 can experience less sweeping during engagement, can have a sharper turning radius, and can protect the end of the inner body 220. However, such retracted driving may also cause obstruction of the field of view of a camera positioned at the end of the inner body 220. Furthermore, a sharper turning radius can lead to premature damage to the inner body 220.

[0139] When driven in paired mode, the predetermined distance 240 that can be maintained between the ends of the outer body 210 and the inner body 220 can include a range of distances. For example, a range of distances can be selected based on the above considerations affecting the driving of the medical device 200 in paired mode, such that driving the medical device 200 is not adversely affected by the distance between the ends being too small or too large. The system can monitor the distance between the ends of the outer body 210 and the inner body 220 during driving in paired mode and adjust the distance between the ends of the outer body 210 and the inner body 220 in response to the measured distance being less than a first threshold distance or greater than a second threshold distance. By maintaining the distance between the ends within a certain range, the system is able to compensate for measurement errors that may be introduced due to errors in the position of the device manipulator, the extension or retraction of one or more of the outer body 210 and the inner body 220, etc., during or before the process.

[0140] In some implementations, the system may store multiple predetermined distances, and a specific predetermined distance may be selected for driving in paired modes. The predetermined distances may be associated with user preferences, the type of medical procedure being performed (e.g., bronchoscopy, ureteroscopy, gastrointestinal examination, etc.), and / or each individual medical device. The system may receive a preference selection identifying the value of the predetermined distance. This selection may include an identifier of the system's user, and the identified user may be associated with a given predetermined distance pre-selected according to the user's preferences. The system may adjust the predetermined distances based on the received selections. For example, the system may have various modes, such as a motion mode or a conservative mode, to allow for the possible selection of different performance profiles based on user preferences. Each mode may have different parameters for the predetermined distances and / or engagement factors. The predetermined distances and co-engagement factors can influence the engagement profile and, conversely, affect wear and tear on the medical devices.

[0141] In some implementations, when the driving mode of the medical device changes from a paired driving mode to an unpaired driving mode, or vice versa, the system can adjust a predetermined distance based on the position of the distal end of the external body relative to the patient's cavity network. For a given anatomical structure, varying the predetermined distance and common engagement factor to achieve an engagement profile may be desirable.

[0142] When driven in paired mode while maintaining a predetermined distance between the end of the outer body 210 and the end of the inner body 220, the system can determine that the distance between the end of the inner body and the end of the outer body is not equal to the predetermined distance (or is not within the distance range). In some embodiments, the system can determine the distance between the ends of the inner and outer bodies based on their respective lengths and robot insertion data, which can be used to determine the distance at which the corresponding inner or outer body has been inserted into the patient. In one implementation, the system can store the lengths of the inner and outer bodies in a storage device that can be mounted on a portion of the inner and outer bodies. For example, an RFID tag can be attached to each of the inner and outer bodies, and the system can be configured to read the lengths of the inner and outer bodies from the RFID tag. The lengths of the inner and outer bodies can be measured after manufacturing, and the measurements can be stored in the storage device.

[0143] In response to the determination that the distance between the end of the inner body and the end of the outer body is not equal to a predetermined distance, the system can advance or retract one of the outer and inner bodies until the distance between the ends of the inner and outer bodies is maintained at the predetermined distance. For example, the system can switch to an outer body drive mode to advance the outer body 210 or switch to an inner body drive mode to retract the inner body 220 until the distance is substantially equal to (e.g., within a defined tolerance) the predetermined distance. The system can switch back to a paired mode so that, in response to the distance returning to the tolerance of the predetermined distance, both the outer and inner bodies can be driven by a set of instrument manipulators to maintain the predetermined distance.

[0144] Specifically, the system can determine that the end of the inner body extends less than a predetermined distance from the end of the outer body, and in response to this determination, change the driving mode of the medical device to an inner body driving mode, and advance the inner body based on an insertion command received from the user until the end of the inner body extends the predetermined distance from the end of the outer body. The system can then change the driving mode to a paired driving mode. Alternatively, the system can determine that the end of the inner body extends more than a predetermined distance from the end of the outer body, and in response to this determination, change the driving mode of the medical device to an outer body driving mode, and advance the outer body based on an insertion command received from the user until the end of the inner body extends the predetermined distance from the end of the outer body. The system can then change the driving mode to a paired driving mode.

[0145] The system can also be configured to operate in an internal body-driven mode (also known as a guide-driven mode). In internal body-driven mode, the system can be configured to map input received from user input devices to drive commands for the internal body without providing further commands to the external body. In some implementations, the system can be configured to receive internal body-driven mode commands via a set of user input devices and, in response to receiving such commands, change the drive mode of the medical device to internal body-driven mode, in which the internal body advances or retracts while the external body remains stationary. Internal body-driven mode can also map engagement commands to the internal body. In some embodiments, the system can also engage the external body in internal body-driven mode. The techniques for mapping engagement commands to the internal and external bodies according to the drive mode will be discussed in more detail below.

[0146] Figure 19An embodiment of an internal body drive mode for a medical device according to aspects of this disclosure is illustrated. The system can be configured to operate via an internal body drive mode command. The system can also be configured to change the drive mode of the medical device 300 to an internal body drive mode in response to receiving an internal body drive mode command. In the internal body drive mode, insertion and retraction commands can be mapped to the internal body while the external body remains substantially stationary.

[0147] exist Figure 19 In the diagram, multiple snapshots of the medical device 300 are shown sequentially from top to bottom as the inner body 320 is inserted along the insertion direction 330 while the position of the outer body 310 is substantially maintained. Dashed lines are provided relative to the initial position of a predetermined distance 340 between the ends of the outer body 310 and the inner body 320. For example, when switching from a paired drive mode to an inner body drive mode, the ends of the outer body 310 and the inner body 320 may initially be separated by a distance within a threshold range of the predetermined distance 340. The outer body 310 remains substantially stationary as the inner body 320 advances in response to a forward command in the snapshots shown. Although the position of the ends of the outer body 310 is shown as stationary, changes in the environment (e.g., the patient's respiratory movements) and / or forces applied to the outer body 310 due to movement of the inner body 320 may cause a minimal change in the position of the outer body 310. Figure 19 The forward command is shown, but the retraction command can also be executed in the inner body drive mode, where the inner body 320 retracts to the end of the outer body 310 while the outer body 310 remains essentially stationary.

[0148] The system can also be configured to operate in an external body drive mode (also known as a sheath drive mode). In external body drive mode, the system can be configured to map input received from user input devices to drive commands for the external body without providing further commands to the internal body. In some implementations, the system can be configured to receive external body drive mode commands via a set of user input devices and, in response to receiving such commands, change the drive mode of the medical device to external body drive mode, in which the external body advances or retracts while the internal body remains stationary. As discussed in detail below, external body drive mode can also map engagement commands to the internal body.

[0149] Figure 20An embodiment of an external body drive mode for a medical device according to aspects of this disclosure is illustrated. The system can be configured to receive an external body drive mode command via a user input device. The system can also be configured to change the drive mode of the medical device 300 to an external body drive mode in response to receiving the external body drive mode command. In the external body drive mode, insertion and retraction commands can be mapped to the external body, while the inner body remains substantially stationary.

[0150] exist Figure 20 In the diagram, multiple snapshots of the medical device 400 are shown from top to bottom as the outer body 410 advances along the insertion direction 430 while the position of the inner body 420 remains substantially stationary. A dashed line is drawn relative to a predetermined distance 440 between the ends of the outer body 410 and the inner body 420, at which further advancement of the outer body 410 will automatically switch to a paired mode. While the end of the outer body 410 advances in response to an advance command in the snapshots shown, the inner body 420 remains substantially stationary. Although the position of the end of the inner body 420 is shown as stationary, changes in the environment (e.g., the patient's breathing movements) and / or forces applied to the inner body 420 due to movement of the outer body 410 may cause a minimal change in the position of the inner body 420. Figure 20 The forward command is shown, but the retraction command can also be executed in the outer body drive mode, where the end of the outer body 410 retracts away from the end of the inner body 420 while the inner body 420 remains essentially stationary.

[0151] Figure 21 A block diagram illustrating a technique for changing the drive mode of a medical device according to aspects of this disclosure is provided. In the illustrated embodiment, the medical device can be driven in one of three drive modes: a paired drive mode 505, an inner body drive mode 510, and an outer body drive mode 515. In some implementations, a user input device may include a button configured to receive a command from a user of the system to change the current drive mode. The system may respond differently to the drive mode change command depending on the current state of the system. For example, when the system is driving the medical device in paired drive mode 505 and receives a drive mode change command, the system may change the drive mode to inner body drive mode 510. This change may include stopping the sheath and activating the drive of the inner body.

[0152] When in the inner body drive mode 510, the system's response to a command to change the drive mode may depend on whether the distance between the ends of the inner and outer bodies is within a predetermined threshold distance (e.g., when the medical device is in a paired position, see...). Figure 17AWhen the distance between the ends of the inner and outer bodies is within a predetermined threshold distance, the system can change the driving mode to a paired driving mode 505 in response to receiving a change driving mode command. When the distance between the ends of the inner and outer bodies is not within the predetermined threshold distance, the system can change the driving mode to an outer body driving mode 515 in response to receiving a change driving mode command.

[0153] Ultimately, when in external body drive mode 515, the system can change the drive mode to internal body drive mode 510 in response to receiving a change drive mode command. This change in drive mode may include stopping the external body at its current position and initiating the drive of the internal body. The system may not require a direct command to change from external body drive mode 515 to paired drive mode 505, as the system can automatically perform this change in drive mode under certain conditions, as discussed below.

[0154] In some implementations, the system can be configured to drive the medical device at a higher speed in paired drive mode 505 compared to either inner body drive mode 510 or outer body drive mode 515. For example, driving the medical device through the initial portion of the cavity network closer to the entry point may require lower precision compared to the portion of the cavity network closer to the target. Near the target, the user can select either inner body drive mode 510 or outer body drive mode 515. Therefore, when lower precision is required, the system can drive the medical device at a faster speed in paired drive mode 505. Alternatively, the system can drive the medical device at a faster speed when within a threshold insertion distance from the entry point and limit the speed after the insertion distance exceeds the threshold insertion distance. The drive speed may include at least one of the following: engagement speed, relaxation speed, insertion speed, and retraction speed.

[0155] The system can also be configured to perform “automatic pairing” (e.g., automatic transition from either inner body drive mode 510 or outer body drive mode 515) by changing the drive mode to paired drive mode 505 when the drive in the current drive mode approaches a paired position. Figure 22 An embodiment of automated pairing for a medical device according to aspects of this disclosure is shown. Specifically, in Figure 22The image shows multiple snapshots of the medical device 600 as the inner body 620 retracts along the retraction direction 630, from top to bottom. In this case, the system is driving the inner body 620 in an inner body drive mode, and the end of the inner body 620 extends from the end of the outer body 610 by a predetermined distance 640, as shown by the dashed line. For example, the system can be configured to receive a retraction command via a set of user input devices in the inner body drive mode. In response to receiving the retraction command, the system can retract the inner body 620 via a set of instrument manipulators of the robotic arm assembly.

[0156] In some embodiments, once the end of the inner body 620 reaches a predetermined distance 640 from the end of the outer body 610 (e.g., the medical device 600 is in a paired position), the system can automatically enter a paired drive mode. That is, the system can determine that the distance between the end of the inner body 620 and the end of the outer body 610 is within the tolerance range of the predetermined distance 640. In response to determining that the distance between the end of the inner body 620 and the end of the outer body 610 is within the tolerance range of the predetermined distance 640, the system can change the drive mode of the medical device 600 to a paired drive mode. Thereafter, according to the paired drive mode, the system can map further drive commands to both the outer body 610 and the inner body 620. Figure 22 In the example shown, the retraction command is maintained, and both the outer body 610 and the inner body 620 are retracted to maintain the predetermined distance 640 between them.

[0157] The system can also be configured to perform "automatic pairing" from the external body drive mode. Figure 23 Another embodiment of the automated pairing of medical devices according to aspects of this disclosure is shown. Specifically, in Figure 23 The image shows multiple snapshots of the medical device 700 as the outer body 710 is inserted along the insertion direction 730, from top to bottom. In this case, the system is driving the outer body 710 in an outer body drive mode, and the end of the inner body 720 extends from the end of the outer body 710 by a predetermined distance 740, as shown by the dashed line. For example, the system can be configured to receive an insertion command via a set of user input devices in the outer body drive mode. In response to receiving the insertion command, the system can insert the outer body 710 via a set of instrument manipulators of the robotic arm assembly.

[0158] In some embodiments, once the end of the outer body 710 reaches a predetermined distance 740 from the end of the inner body 720 (e.g., the medical device 700 is in a paired position), the system can automatically enter a paired drive mode. That is, the system can determine that the distance between the end of the inner body 720 and the end of the outer body 710 is within the tolerance range of the predetermined distance 740. In response to determining that the distance between the end of the inner body 720 and the end of the outer body 710 is within the tolerance range of the predetermined distance 740, the system can change the drive mode of the medical device 700 to a paired drive mode. Thereafter, according to the paired drive mode, the system can map further drive commands to both the outer body 710 and the inner body 720. Figure 23 In the example shown, the insertion command is maintained, and both the outer body 710 and the inner body 720 are inserted to maintain the predetermined distance 740 between them.

[0159] Figure 24 A block diagram is provided illustrating another technique for altering the driving mode of a medical device according to aspects of this disclosure. Specifically, Figure 24 This shows the combination of the above. Figure 22 and Figure 23 The discussion focuses on the "automatic pairing" transition. In the first automatic pairing technique 820, the system can automatically transition from inner body drive mode 810 to pair drive mode 805 in response to the end of the inner body retracting to the pairing position (e.g., within a predetermined distance tolerance). In the second automatic pairing technique 825, the system can automatically transition from outer body drive mode 815 to pair drive mode 805 in response to the end of the outer body being inserted into the pairing position (e.g., within a predetermined distance tolerance).

[0160] In some implementations, the system can determine whether a timing condition is met while executing one of the first automatic pairing technique 820 and the second automatic pairing technique 825. For example, when in the inner body drive mode 810, the user can retract the inner body until the distance between the end of the inner body and the end of the outer body is in a paired position. When the paired position is reached, the system can start a timer or otherwise track or measure the duration for which the inner device has been commanded to retract since reaching the paired position. If the system receives a command to stop the medical device from retracting or advancing when the elapsed time is less than a threshold time period, the system can return to or remain in the inner body drive mode 810. Otherwise, if the timing condition has been met, the system can change the drive mode to the paired mode. Such a timing condition can serve as a technique to prevent the system from automatically changing from one mode to the paired drive mode 805 when the user does not intend to enter the paired drive mode 805.

[0161] Similarly, time conditions can be used to determine whether an external instrument insertion command will cause the system to switch from sheath-driven mode to pair-driven mode.

[0162] The system can also be configured to perform initial adjustments after the medical procedure has started to address misalignment between the ends of the inner and outer bodies. Figure 25 An implementation of an initial follow-up method for a medical device according to aspects of this disclosure is illustrated. In some implementations, the system can be configured to automatically initiate the process in a paired drive mode. However, after the initial setup of the robotic surgical system, the ends of the outer body 910 and the inner body 920 may not be positioned in a paired location (e.g., within a threshold distance of a predetermined distance 940). This may occur, for example, due to a certain amount of manufacturing variation in the length of each of the outer body 910 and the inner body 920, which could cause the distance between the ends of the outer body 210 and the inner body 220 to shift from the predetermined distance depending on the loading position of the robotic arm. The system can store the post-manufacturing measured lengths of the inner body 910 and the outer body 920 in RFID tags attached to the inner body 910 and the outer body 920, respectively. Therefore, the system is able to read the lengths of the inner body 910 and the outer body 920, taking into account the post-manufacturing measured manufacturing variation. Figure 25 The diagram illustrates a possible misaligned initial condition in which the distance between the end of the outer body 910 and the end of the inner body 920 is less than a predetermined distance 940.

[0163] In response to receiving an insertion command, the system can drive only the inner body 920 until the distance between the outer body 910 and the inner body 920 reaches a predetermined distance 940. Afterward, the system can drive both the outer body 910 and the inner body 920 together in a paired mode.

[0164] Figure 26 Another embodiment of the initial follow-up of a medical device according to aspects of this disclosure is shown. For example, another possible misalignment includes a distance greater than a predetermined distance 1040 between the end of the outer body 1010 and the end of the inner body 1020. As described above, the system can be configured to automatically initiate the process in a paired drive mode. However, after the initial setup of the robotic surgical system, the ends of the outer body 1010 and the inner body 1020 may not be positioned in a paired location (e.g., within a threshold distance of the predetermined distance 940). Figure 26 Another possible misaligned initial condition is shown, wherein the distance between the end of the outer body 1010 and the end of the inner body 1020 of the medical device 1000 is greater than a predetermined distance 1040.

[0165] In response to receiving an insertion command, the system can drive only the outer body 1010 until the distance between the outer body 1010 and the inner body 1020 reaches a predetermined distance 1040. Afterward, the system can drive both the outer body 1010 and the inner body 1020 together in a paired mode.

[0166] Figure 27 A block diagram illustrating another technique for altering the driving mode of a medical device, according to aspects of this disclosure, is provided. Specifically, Figure 27 This shows the combination of the above. Figure 25 and Figure 26 The discussion focuses on the "initial follow-up" transition. In the first initial follow-up technique 1120, the system can automatically switch from the inner body driving mode 1110 to the paired driving mode 1105 in response to the distance between the ends of the outer body and the inner body being less than a predetermined distance. In the second initial follow-up technique 1125, the system can automatically switch from the outer body driving mode 1115 to the paired driving mode 1105 in response to the distance between the ends of the outer body and the inner body of the medical device being greater than a predetermined distance.

[0167] B. Joint connection between medical device bodies.

[0168] To advance through the patient's cavity network, it may be necessary to engage a portion near the distal end of the medical device to align the device with the desired insertion direction within the cavity network. When the system is configured to drive the medical device in various drive modes (e.g., paired drive mode, internal body drive mode, and external body drive mode), the system can map engagement commands received via one or more user input devices based on the current drive mode.

[0169] Figure 28 This is a flowchart illustrating an example method operated by a surgical robot system or its components(s) for co-engaging the outer and inner bodies of a medical device according to aspects of this disclosure. For example, Figure 28 The steps of method 1700 shown can be executed by the processor of the surgical robot system. For convenience, method 1700 is described as being executed by the system's processor.

[0170] Method 1700 begins at block 1701. A processor may be included as part of a system comprising: a medical device containing an outer body and an inner body configured to drive an inner body through a cavity within the outer body; a set of one or more instrument manipulators (e.g., coupled to a robotic arm assembly) configured to control movement of the outer and inner bodies; a set of one or more user input devices; a set of one or more processors; and at least one computer-readable storage device communicating with the set of processors and having computer-executable instructions stored on the computer-readable storage device that cause the set of processors to perform method 1700.

[0171] At block 1705, the processor receives an engagement command for engaging a medical device via a set of user input devices. In some embodiments, the engagement command may not include selecting the outer or inner body of the medical device for engagement. Therefore, the system can be configured to map the engagement command to one or both of the outer and inner bodies based on the current driving mode. For example, when in inner body driving mode, the user may wish to engage the end of the inner body. However, depending on the distance between the end of the inner body and the end of the outer body, it may be desirable to engage both the inner and outer bodies to provide additional engagement (e.g., a smaller radius of curvature) or to avoid mutual resistance in the engagement of the inner and outer bodies.

[0172] One potential problem that can arise when engaging only one of the outer and inner bodies at a time is "muscling." As used herein, mucling generally refers to a situation where engagement applied to one of the outer and inner bodies is opposite to engagement applied to the other. This can occur, for example, when a first force is applied to a tendon in the outer body to maintain the current engagement of the outer body, while a second force is applied to the inner body in response to an engagement command. Thus, mucling can occur between the first and second forces, resulting in opposing forces between the outer and inner bodies.

[0173] To prevent unintended forced engagement and improve the achievable engagement amount of the medical device, the system can "co-engage" the outer and inner bodies in response to a received engagement command. In some embodiments, the amount of co-engagement may depend on the distance between the ends of the outer and inner bodies, and may also depend on the current drive mode. Figure 28 In method 1700, at block 1701, the processor designates one of the external and internal bodies as a primary body, and the other as a secondary body. The system can then apply a joining command to the primary body and can cause the secondary bodies to join together. The system can select the primary and secondary bodies based on the current driving mode. The following table summarizes a technique for selecting the primary and secondary bodies.

[0174] Table 1

[0175]

[0176] The technique described above for selecting the primary and secondary ontology is only one implementation method, and other techniques may be used, such as selecting the inner ontology as the primary ontology and the outer ontology as the secondary ontology in a pairwise driving mode.

[0177] During co-engagement, at block 1715, the processor determines the distance between the end of the inner body and the end of the outer body. The system can determine the distance between the ends using one of a variety of different techniques, such as robot data based on the position of a manipulator. At block 1720, the processor determines a co-engagement factor based on the determined distance between the end of the outer body and the end of the inner body. In some embodiments, the co-engagement factor can vary depending on the distance between the end of the outer body and the end of the inner body. In other embodiments, the co-engagement factor can be a static value applied when the distance between the end of the outer body and the end of the inner body is within a predetermined distance range.

[0178] At box 1725, the processor engages the primary body via a set of instrument manipulators based on an engagement command. That is, the system can apply the full amount of engagement received in the engagement command to the primary body. However, this disclosure is not limited thereto, and in some cases, the system can adjust the amount of engagement applied to the primary body. At box 1730, the processor engages the secondary body via a set of instrument manipulators based on an engagement command and a common engagement factor. The common engagement factor can define the technique for determining the amount of engagement to be applied to the secondary body based on one or more of the following: the engagement command, the amount of engagement applied to the primary body, and the current drive mode. The method ends at box 1735.

[0179] In some implementations, the co-engagement factor includes a co-engagement ratio that correlates the amount of engagement applied to the primary body with the amount of engagement applied to the secondary body. That is, the co-engagement ratio can define a ratio used to determine the amount of engagement applied to the secondary body based on the amount of engagement applied to the primary body. In some implementations, the co-engagement ratio is determined based on the distance between the ends of the outer body and the inner body. Therefore, the amount of engagement applied to the secondary body can vary based on the distance between the ends. Furthermore, the technique used to determine the co-engagement ratio can depend on the current driving mode.

[0180] Because the system maintains the distance between the ends of the outer and inner bodies when in paired drive mode, the co-engagement ratio can be substantially constant. However, in other embodiments, the co-engagement ratio can be adjusted in paired drive mode based on commands received, for example, via a set of user input devices. Therefore, in some implementations, the user can select the co-engagement ratio to adjust the amount of engagement achievable by the medical device. In these implementations, the user can also manually select the co-engagement ratio in other drive modes (e.g., external drive mode and internal drive mode).

[0181] Figure 29A This includes a diagram illustrating a technique for determining the co-engagement ratio in an in-body driven mode, based on aspects of this disclosure. Figure 29A In this implementation, the co-engagement ratio is shown by a curve or slope 1205. The inner body insertion depth (which relates to the distance between the ends of the outer body and the inner body) is plotted along the X-axis, and the co-engagement ratio value is plotted along the y-axis. The graph includes: a dashed line 1210 indicating paired positions; a point 1215 indicating the start of the co-engagement region; and a point 1220 indicating the end of the co-engagement region. In some implementations, point 1215 may be a point where the ends of the outer body and the inner body are aligned (e.g., there is no distance between the ends), and point 1220 may represent the inner body insertion depth beyond which the co-engagement ratio drops to zero.

[0182] In the illustrated embodiment, co-engagement is disabled outside a predetermined range of distance between the ends of the outer and inner bodies (this may include setting the co-engagement value to zero). The predetermined range of distance is indicated by the inner body insertion depth between points 1215 and 1220. Outside this range, the system may not co-engage the secondary devices, but only provide engagement commands to the primary devices. At the start of the co-engagement region at point 1215, the co-engagement ratio can be set to a predetermined value. Furthermore, when the distance between the ends of the outer and inner bodies falls within the predetermined range of distance, the co-engagement ratio can be determined based on a predetermined function that correlates the determined distance with the co-engagement ratio. Figure 29A As shown, the co-engagement ratio determined based on a predetermined function can decrease as the determined distance (e.g., the inner body insertion depth) increases. The curve defined by this function can be a smooth curve 1205, which defines a gradual transition from the starting point 1215 of the co-engagement region to the ending point 1220 of the co-engagement region. In some implementations, the predetermined function can be defined by a sigmoid function having parameters that can be selected to control the speed and shape of the curve 1205.

[0183] The system can use based on constraints Figure 29AThe common engagement factor determined by the function of curve 1205 shown is used to determine the amount of engagement to be applied to the outer body (e.g., tension to be applied to one or more tendons of the outer body) based on the engagement command received in the inner body drive mode. For example, the system may apply the full amount of the engagement command to the inner body and a portion of the engagement command, as determined using an engagement ratio, to the outer body.

[0184] Figure 29B This includes a diagram illustrating a technique for determining the co-engagement ratio in an external body-driven mode, based on aspects of this disclosure. Figure 29A In the implementation method, by means of... Figure 29A The curve 1205 shown in the figure, similar to the curve or slope 1225, illustrates the co-engagement ratio. The graph includes a dashed line 1230 indicating paired positions, a point 1235 representing the start of the co-engagement region, and a point 1240 representing the end of the co-engagement region. In some implementations, point 1235 may be the point where the end of the outer body aligns with the end of the inner body (e.g., there is no distance between the ends), and point 1240 may represent the inner body insertion depth beyond which the co-engagement ratio drops to zero.

[0185] Curve 1225 can be defined in a manner similar to curve 1205 in the internal drive mode. However, in some implementations, the initial value at the starting point 1235 of the common engagement region of the external body drive mode can have a lower value than the initial value in the internal body drive mode. Alternatively, parameters defining the characteristics of curve 1225 (e.g., the speed and shape of the curve) can be adjusted in a manner different from that in the internal drive mode.

[0186] This system can use based on limitations Figure 29A The common engagement factor, determined as a function of curve 1225, is used to determine the amount of engagement to be applied to the inner body (e.g., tension to be applied to one or more tendons of the inner body) based on the engagement command received in the outer body drive mode. For example, the system may apply the full amount of the engagement command to the outer body and a portion of the engagement command to the inner body as determined using the engagement ratio.

[0187] The system can also be configured to use a common engagement factor (as determined according to one of the embodiments described above) based on a relaxation command received via a set of user input devices to relax the medical instruments. In some embodiments, the system can relax the primary body via a set of instrument manipulators of the robotic arm assembly based on a relaxation command, and can relax the secondary body via a set of instrument manipulators of the robotic arm assembly based on the relaxation command and the common engagement factor. The common engagement factor can be determined based on one or more of the following: the distance between the ends of the outer and inner bodies, the current drive mode, and the relaxation command. The relaxation command may include a command to relax the tension in one or more tendons previously used to engage the outer and / or inner bodies. Additionally, similar to the common engagement embodiments discussed above, the outer and inner bodies can be used as the primary and secondary bodies, respectively, according to Table 1.

[0188] In some cases, once the primary body is relaxed (e.g., tension is no longer applied to the tendons in the primary body), the secondary body may still have tension in one or more tendons within its own tendons. Therefore, in response to determining that the primary body has been relaxed, the system can relax the secondary body independently of the common engagement factor based on a relaxation command via a set of instrument manipulators of the robotic arm assembly. For example, since certain problems such as hard squeezing will no longer occur when there is no tension in the tendons of the primary body, once the primary mechanism is relaxed, it may no longer be necessary to adjust the engagement of the secondary mechanism. Therefore, if the system receives a relaxation command after the primary body has been relaxed, the system will apply the relaxation command to the secondary body without adjusting the relaxation command based on the common engagement ratio.

[0189] In some implementations, the system can also be configured to receive a disable co-engagement command via a set of user input devices. In response to receiving the disable co-engagement command, the system can engage the primary body via a set of manipulators on the robotic arm assembly based on an engagement command, without engaging the secondary body. This provides the user with additional options for manual control of each of the outer and inner bodies, which can be used to perform more complex engagement maneuvers. For example, the user can apply a certain amount of force between the outer and inner bodies to provide additional support to the inner body based on the engagement of the outer body.

[0190] Figure 30 An embodiment of tension monitoring for a medical device according to aspects of this disclosure is shown. In particular, Figure 30The diagram illustrates multiple engagement actions 1300 that the system can take based on a comparison of tension measured in one or more tendons of one of the outer and inner bodies with four threshold tension values. While tension monitoring can be applied to each of the outer and inner bodies individually, for ease of description, an example of tension monitoring will be described for the inner body. Similar techniques can be applied to the outer body, which may include values ​​that are the same as or different from the tension thresholds. In some implementations, the system can monitor the tension value of each tendon in a given inner or outer body individually. However, in other implementations, the system can determine a standard for the tension value in the respective tendons of the inner or outer body.

[0191] In the illustrated implementation, when the measured tension at the tendon of the inner body is less than a first threshold, the system can engage the inner body at a normal engagement speed 1305. For example, when the tension is less than the threshold, the system may not change the engagement applied to the inner body. When the measured tension is greater than the first threshold but less than a second threshold, the system can slow down the engagement speed 1310. By slowing down the engagement speed, the system can prevent the engagement of the inner body from rapidly applying additional force to the patient's cavity, thereby reducing the chance of injury to the patient.

[0192] When the measured tension is greater than a second threshold but less than a third threshold, the system can limit the engagement amount 1315. When the engagement amount is limited, forces applied to the patient's cavity (e.g., through additional engagement) are prevented, reducing the risk of injury. When the measured tension is greater than a third threshold but less than a fourth threshold, the system can automatically relax the inner body 1320. After engagement is limited, the tension in the tendon may increase in certain situations, such as when the inner body retracts into part of the cavity, or when the cavity changes shape and applies force to the tendon. The system can generally regulate the force applied to the tendon in response to the external force applied to it, and thus increase the force applied to the tendon to maintain the current engagement amount. By providing the automatic relaxation function 1320 when the tension is greater than the third threshold, the system can prevent the force applied to the cavity by the inner body from reaching a level that could cause injury to the patient.

[0193] Error 1325 may occur when the measured tension exceeds the fourth threshold. Although the automatic relaxation function 1320 is designed to prevent tension from reaching dangerous levels, the fourth threshold can be set to detect system malfunctions, insufficient relaxation, or other unexpected events. In response to the error, the system can disable the operation of the robotic system. Furthermore, the system can request the medical procedure to stop and the medical device to be removed from the patient. The system can then be reset so that the medical procedure can resume.

[0194] The system can also be configured to automatically deactivate the slowing-joint-speed function 1310 and / or the limiting-joint-speed function 1315. In some embodiments, in response to activation of the slowing-joint-speed function 1310 or the limiting-joint-speed function 1315, the system can disable the corresponding function 1310 or 1315 in response to a measured tension on the tendon being below a corresponding threshold (e.g., a first threshold or a second threshold) for a threshold time period. The system can store different threshold time periods to deactivate each of the slowing-joint-speed function 1310 and the limiting-joint-speed function 1315.

[0195] Figure 31 An embodiment of automatic relaxation during retraction of a medical device according to aspects of this disclosure is shown. Figure 31 In this embodiment, the medical device 1400 includes an outer body 1410 and an inner body that can be in a first position 1420 or a second position 1425. When the inner body is in the first engagement position 1420, the inner body can be assembled to have a radius of curvature smaller than the threshold radius of curvature 1430 shown by the dashed line. When the inner body is in the second engagement position 1425, the inner body can be assembled to have a radius of curvature larger than the threshold radius of curvature 1430.

[0196] A technique for automatically relaxing a medical device during retraction is based on the distance of retraction without receiving a user command other than retraction. For example, the system can also automatically relax the medical device 1400 during retraction after the medical device has retracted a distance greater than a threshold distance. For example, the system can determine that the medical device has retracted a distance greater than a threshold distance and, in response to determining that the medical device has retracted a distance greater than the threshold distance, automatically relax the medical device. The retraction distance greater than the threshold distance can include the retraction of the medical device without receiving any insertion or engagement command. When the system receives an engagement or insertion command, the system can block and / or exit the automatic relaxation command, thereby allowing the user to exercise control over the automatic relaxation. The system can perform automatic relaxation in any of the drive modes (e.g., paired drive mode, inner body drive mode, outer body drive mode) and can measure the determined retraction distance based on the primary body corresponding to the current drive.

[0197] Automatic relaxation based on retraction distance can assist users in relaxing medical devices when they are retracted over long distances (e.g., after a medical procedure has been completed and the device has been removed). Typically, it is desirable to relax medical devices during long retractions, so the system can assist users when they forget to manually relax the device.

[0198] Another implementation of automatic relaxation may include the measurement of the radius of curvature of the medical device 1400. In response to receiving a retraction command from a set of user input devices, the system can be configured to determine the curvature of the medical device (e.g., in…). Figure 31 In the example, the curvature of the inner body at the first position 1420 is less than a threshold curvature 1430. In response to determining that the curvature of the medical device is less than the threshold curvature 1430, the system can automatically relax the medical device upon retraction. By automatically relaxing the medical device 1400, the system can prevent hardening of the medical device due to tension in the portion of the inner body 1420 retracted into the outer body 1410. Since hardening can lead to premature wear of the outer body 1410 and / or the inner body 1420, this automatic relaxation can extend the lifespan of the medical device 1400. In some embodiments, this automatic relaxation can occur immediately upon retraction without measuring the retraction distance. Alternatively, automatic retraction based on the radius of curvature can occur only when the engagement portion (e.g., the length of the inner body 1420 near its end) is retracting into the outer body 1410. The radius of curvature of the inner body 1420 can be determined based on the angle of the command to the inner body 1420 (or alternatively, the engagement amount of the command) and the insertion amount of the command.

[0199] Automatic relaxation of a medical device can also utilize the co-engagement ratio used for commanded relaxation as described above. However, in some embodiments, during automatic relaxation, a primary body is relaxed, and the system can continue to use the co-engagement ratio while engaging a secondary body. This continued use of the co-engagement ratio prevents the outer body from prematurely losing support for the inner body during automatic relaxation.

[0200] C. External body parking.

[0201] By allowing the drive mode to be selected and changed between paired drive mode, external body drive mode, and internal body drive mode, the system can facilitate easier access to portions of the cavity network. For example, particularly sharp turns may require a smaller radius of curvature during engagement, which can be achieved through co-engagement in internal body drive mode. Furthermore, due to the diameter of the external body, it may be necessary to park the external body and continue advancing into the cavity network in internal body drive mode to allow the medical device to fit into a cavity with a diameter smaller than that of the external body.

[0202] This system can access preoperative and / or intraoperative models of the cavity network through which a medical device advances. The model can be stored in memory and may include a mapped portion of the cavity network. The memory can also store the target's position relative to the model and the path along the model from the entry point to the target. By using the model, the target's position, and / or path, the system can determine where the sheath can be positioned to follow the path and / or reach the target.

[0203] Figure 32 This is a flowchart illustrating an example method for parking assistance of a medical device, operable by a surgical robot system or a component thereof, according to aspects of this disclosure. For example, Figure 32 The steps of method 1800 shown can be executed by the processor of the surgical robot system. For convenience, method 1800 is described as being executed by the system's processor.

[0204] Method 1800 begins at block 1801. A processor may be included as part of a system comprising: a medical device including an outer body and an inner body configured to be driven through a cavity in the outer body; one or more device manipulators (e.g., coupled to a robotic arm assembly) configured to control movement of the outer and inner bodies; one or more feedback devices; one or more processors; and at least one computer-readable storage device communicating with the processors and storing computer-executable instructions on the at least one computer-readable storage device to cause the processors to perform method 1800.

[0205] At block 1805, the processor identifies a portion of the cavity network along a path having a shape matching the parking aid feature. The parking aid feature may include, for example: a portion of the cavity network having a radius of curvature less than a threshold; a diameter of the cavity network less than a threshold; or a pre-planned location for parking the outer body, selectively chosen by the user. At block 1810, the processor causes a parking indication to be generated on at least a portion of a set of feedback devices at a location relative to the model corresponding to the identified portion, indicating the position of the end of the parking outer body. Method 1800 ends at block 1815.

[0206] Figure 33 An example of parking instructions based on aspects of this disclosure is shown. Figure 33The model 1500 shown can be displayed on a screen and may also include a path 1505 and a parking instruction 1510, which may be a pre-planned path for navigating the medical device toward a target (not shown). The parking instruction may be a visual representation of the position in the model where the outer body is parked and the drive mode is changed to an internal drive mode to allow the inner body to advance further. In other embodiments, in addition to displaying the parking instruction on a screen, the system may include a set of one or more feedback devices, including at least one of the following: a display, a haptic feedback device, and a speaker. Thus, the system can provide the parking instruction to the user through one or more of the feedback mechanisms listed above. For example, the system may provide auditory and / or haptic feedback to the user when the tip of the medical device is within a threshold distance of the parking instruction, and the system may provide different types of feedback when the tip of the medical device reaches the position of the parking instruction.

[0207] In some implementations, the system may not provide a parking instruction until the target is reachable in the inner body drive mode. For example, the inner body may only be able to extend a threshold insertion distance from the end of the outer body. Therefore, the system can determine that the distance from the identification portion to the target is less than the threshold insertion distance of the inner body, and in response to determining that the distance from the identification portion to the target is less than the threshold insertion distance of the inner body, present a parking instruction. By delaying the provision of the parking instruction until the end of the medical device is within the threshold insertion distance from the target, the system can prevent the user from prematurely parking the outer body at a location that would not allow the inner body to reach the target.

[0208] The system can also be configured to receive an external body parking command to park the external body at its current position. The system can determine that the distance from the current position to the target is greater than a threshold insertion distance for the internal body, and in response to determining that the distance from the current position to the target is greater than the threshold insertion distance for the internal body, cause a presentation on at least a portion of a set of displays indicating that the internal body cannot reach the target when the external body is parked at its current position. Therefore, the system can be able to provide a warning to the user that parking the external body at its current position during the remainder of the medical procedure may result in the target being unreachable. The user can use this information to determine whether to continue driving in paired mode without parking the external body, or whether to use an external body driving mode to allow the external body to catch up with the internal body after advancing through a portion of the cavity network.

[0209] The system can also be configured to determine parking aid features based on data collected from previously performed medical procedures. For example, parking aid features can be determined based on previously performed medical procedures, each including a path with a shape matching the parking aid feature. Therefore, the system can determine that a user is more likely to park the external body along certain paths with shapes matching the parking aid feature. Then, based on the analysis of the previously performed medical procedures, the system is more likely to provide an indication of the parking location. The system can also be configured to receive a selection of a portion of the identification cavity network and use the received selection to provide a parking instruction.

[0210] In some implementations, the system can be configured to identify locations along the path where the diameter of the cavities in the cavity network is within a threshold range of the diameter of the outer body. The system can identify portions of the cavity network based on the locations identified along the path. Thus, the system can determine a parking instruction based on a location within the cavity network where the cavity network narrows to the point where it may be difficult for the outer body to advance further.

[0211] In other embodiments, the system may also be configured to detect medical device disengagement and to present an indication of the disengagement on at least a portion of a set of displays. The user can use this feedback to correct the disengagement before proceeding. One technique that can be used to detect disengagement may include comparing a commanded advance of the medical device with one or more measurements of the position of the distal end of the medical device (e.g., visual information from a camera in the medical device, position data received from an EM sensor, etc.). The system may be able to determine that the medical device is disengaged when the measured position data indicates that the distal end of the medical device has not moved in response to a command to advance the medical device. The system may also be able to provide any indication of techniques for mitigating the disengagement based on its severity. For example, disengagement less than a threshold amount may be addressed by advancing the external body in an external body drive mode, while more severe disengagement greater than a threshold amount may require retraction of the medical device.

[0212] In one implementation, the system can be configured to detect dislodgement in response to receiving an internal body forwarding command that causes the internal body to move forward. The system can instruct a set of instrument manipulators of a robotic arm assembly to move the internal body forward, and in response to instructing the set of instrument manipulators of the robotic arm assembly to move the internal body forward, determine, based on outputs from one or more sensors, that the position of the end effector of the internal body has not moved. The system can detect medical device dislodgement based on the determination that the position of the end effector of the internal body has not moved.

[0213] 3. Implementation system and terminology.

[0214] The implementation disclosed herein provides systems, methods, and apparatus for driving medical devices having an inner body and an outer body.

[0215] It should be noted that the terms “couple,” “coupling,” “coupled,” or other variations of the word “coupled,” as used herein, can indicate an indirect or direct connection. For example, if a first component is “coupled” to a second component, the first component may be indirectly connected to the second component via another component or directly connected to the second component.

[0216] The functions described herein for driving medical devices may be stored as one or more instructions on a processor-readable or computer-readable medium. The term "computer-readable medium" refers to any available medium that can be accessed by a computer or processor. By way of example and not limitation, such a medium may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, optical disc read-only memory (CD-ROM) or other optical disc storage devices, magnetic disk storage devices or other magnetic storage devices, or any medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. It should be noted that a computer-readable medium may be tangible and non-transitory. As used herein, the term "code" may refer to software, instructions, code, or data that can be executed by a computing device or processor.

[0217] The methods disclosed herein include one or more steps or actions for implementing the described methods. Method steps and / or actions may be interchanged without departing from the scope of the claims. In other words, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims unless proper operation of the described method requires a specific order of steps or actions.

[0218] As used herein, the term "multiple" means two or more. For example, multiple components means two or more components. The term "determine" encompasses a wide variety of actions, and therefore, "determine" can include calculation, computation, processing, derivation, investigation, lookup (e.g., searching in a table, database, or other data structure), ascertainment, etc. "Determine" can also include receiving (e.g., receiving information), accessing (e.g., accessing data in memory), etc. Furthermore, "determine" can include parsing, selecting, picking, building, etc.

[0219] Unless otherwise explicitly stated, the phrase “based on” does not mean “based on only”. In other words, the phrase “based on” describes both “based on only” and “based on at least”.

[0220] A prior description of the disclosed implementations has been provided to enable those skilled in the art to implement or use the invention. Various modifications to these implementations will be apparent to those skilled in the art, and the general principles defined herein can be applied to other implementations without departing from the scope of the invention. For example, it should be understood that those skilled in the art will be able to employ multiple corresponding alternatives and equivalent structural details, such as equivalent methods of fastening, mounting, coupling, or engaging tool components, equivalent mechanisms for generating specific actuating motions, and equivalent mechanisms for delivering electrical energy. Therefore, the invention is not intended to be limited to the implementations shown herein, but is consistent with the widest scope of the principles and novel features disclosed herein.

[0221] The present invention also includes:

[0222] (1). A robotic surgical system, comprising:

[0223] A medical device comprising an outer body and an inner body configured to be driven through a cavity in the outer body;

[0224] A group of one or more machine manipulators are configured to control the movement of the outer body and the inner body;

[0225] A group of one or more user input devices;

[0226] A group of one or more processors; and

[0227] At least one computer-readable storage device, the at least one computer-readable storage device communicating with the group of processors and storing computer-executable instructions on the at least one computer-readable storage device that cause the group of processors to perform the following operations:

[0228] Receive a command to change the drive mode via the set of user input devices; and

[0229] In response to receiving the change drive mode command, the drive mode of the medical device is changed from a paired drive mode to an unpaired drive mode, wherein when in the paired drive mode, in response to receiving a drive command from the set of input devices, the distance between the end of the inner body and the end of the outer body is maintained at a predetermined distance.

[0230] (2). The robotic surgical system according to (1), wherein each of the ends of the inner body and the outer body is chamfered.

[0231] (3). The robotic surgical system according to (1), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0232] It is determined that the distance between the end of the inner body and the end of the outer body is not equal to the predetermined distance;

[0233] In response to determining that the distance between the end of the inner body and the end of the outer body is not equal to the predetermined distance, the driving mode of the medical device is changed to the unpaired driving mode; and

[0234] The outer body and the inner body are advanced until the distance between the end of the inner body and the end of the outer body is within the predetermined distance.

[0235] (4). The robotic surgical system according to (1), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0236] It is determined that the end of the inner body extends from the end of the outer body by less than the predetermined distance;

[0237] In response to determining that the end of the inner body extends less than the predetermined distance from the end of the outer body, the driving mode of the medical device is changed to the inner body driving mode.

[0238] Proceed the inner body forward until the end of the inner body extends the predetermined distance from the end of the outer body; and

[0239] Change the driving mode of the medical device to the paired driving mode.

[0240] (5). The robotic surgical system according to (1), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0241] Before entering the pairing mode, it is determined that the end of the inner body extends from the end of the outer body by a distance greater than the predetermined distance;

[0242] In response to determining that the end of the inner body extends from the end of the outer body by a distance greater than the predetermined distance, the driving mode of the medical device is changed to the outer body driving mode.

[0243] Proceed the outer body forward until the end of the inner body extends the predetermined distance from the end of the outer body; and

[0244] Change the driving mode of the medical device to the paired driving mode.

[0245] (6). The robotic surgical system according to (1), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0246] The driving mode of the medical device is changed to an internal body driving mode, in which the internal body moves forward or retracts while the external body remains stationary.

[0247] (7). The robotic surgical system according to (1), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0248] The driving mode of the medical device is changed to an external body driving mode, in which the external body moves forward or retracts while the internal body remains stationary.

[0249] (8). The robotic surgical system according to (6), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0250] In the internal body drive mode, a retraction command for retraction is received via the set of user input devices;

[0251] In response to receiving the retraction command, the inner body is retracted via the set of instrument manipulators;

[0252] It is determined that: (a) the distance between the end of the inner body and the end of the outer body is within the tolerance range of the predetermined distance, and (b) the timing condition has been met; and

[0253] In response to determining that the distance between the end of the inner body and the end of the outer body is within the tolerance of the predetermined distance and that the timing condition has been met, the driving mode of the medical device is changed to the paired driving mode.

[0254] (9). The robotic surgical system according to (7), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0255] In the external body drive mode, forward commands for moving forward are received via the set of user input devices;

[0256] In response to receiving the forward command, the outer body is moved forward via the set of instrument manipulators;

[0257] It is determined that: (a) the distance between the end of the inner body and the end of the outer body is within the tolerance range of the predetermined distance, and (b) the timing condition has been met; and

[0258] In response to determining that the distance between the end of the inner body and the end of the outer body is within the tolerance of the predetermined distance and that the timing condition has been met, the driving mode of the medical device is changed to the paired driving mode.

[0259] (10). The robotic surgical system according to (6), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0260] The driving mode of the medical device is changed to an external body driving mode, in which the external body moves forward or retracts while the internal body remains stationary.

[0261] The system receives a drive mode switching command for the medical device via the set of user input devices.

[0262] In response to receiving the drive mode switching command, it is determined that the distance between the end of the inner body and the end of the outer body is not within the tolerance range of the predetermined distance; and

[0263] In response to the switch drive mode command and determining that the distance between the end of the inner body and the end of the outer body is not within the tolerance range of the predetermined distance, the drive mode of the medical device is switched between the inner body drive mode and the outer body drive mode.

[0264] (11). The robotic surgical system according to (6), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0265] The driving mode of the medical device is changed to an external body driving mode, in which the external body moves forward or retracts while the internal body remains stationary.

[0266] The system receives a drive mode switching command for the medical device via the set of user input devices.

[0267] In response to receiving the drive mode switching command, it is determined that the distance between the end of the inner body and the end of the outer body is equal to the predetermined distance; and

[0268] In response to the switching drive mode command and determining that the distance between the end of the inner body and the end of the outer body is equal to the predetermined distance, the drive mode of the medical device is switched between the inner body drive mode and the paired drive mode.

[0269] (12). The robotic surgical system according to (1), wherein:

[0270] The group of one or more instrument manipulators includes at least three instrument manipulators.

[0271] The medical device also includes robot-controlled surgical instruments configured to be driven through cavities in the inner body.

[0272] The surgical instruments of the outer body, the inner body, and the robot controller are respectively coupled to the three instrument manipulators.

[0273] (13). The robotic surgical system according to (1), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0274] In the paired mode, the medical device is driven at a speed different from the driving speed of the medical device in the unpaired mode.

[0275] (14). The robotic surgical system according to (13), wherein the speed at which the medical device is driven includes at least one of the following speeds: engagement speed, relaxation speed, insertion speed and retraction speed.

[0276] (15). The robotic surgical system according to (1), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0277] The user input devices receive a selection indicating a preference for a value representing the predetermined distance.

[0278] The predetermined distance is adjusted based on the received selection.

[0279] (16). The robotic surgical system according to (1), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0280] The predetermined distance is adjusted based on the position of the distal end of the external body relative to the patient's cavity network.

[0281] (17). A robotic surgical system, comprising:

[0282] A medical device comprising an outer body and an inner body configured to be driven through a cavity in the outer body;

[0283] A group of one or more machine manipulators are configured to control the movement of the outer body and the inner body;

[0284] A group of one or more user input devices;

[0285] A group of one or more processors; and

[0286] At least one computer-readable storage device, the at least one computer-readable storage device communicating with the group of processors and storing computer-executable instructions on the at least one computer-readable storage device that cause the group of processors to perform the following operations:

[0287] Receive engagement commands for engaging the medical device via the set of user input devices;

[0288] One of the outer body and the inner body is designated as the primary body, and the other of the outer body and the inner body is designated as the secondary body.

[0289] Determine the distance between the end of the inner body and the end of the outer body;

[0290] The common junction factor is determined based on the determined distance;

[0291] Based on the engagement command, the primary body is engaged via the set of instrument manipulators; and

[0292] The secondary body is engaged via the set of instrument manipulators based on the engagement command and the common engagement factor.

[0293] (18). The robotic surgical system according to (17), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0294] The user input devices receive a drive mode command for driving the medical device in one of a variety of drive modes.

[0295] The primary ontology and the secondary ontology are processed based on the driving mode command.

[0296] (19). The robotic surgical system according to (17), wherein the common engagement factor includes a common engagement ratio that relates the amount of engagement characteristics applied to the primary body to the amount of engagement characteristics applied to the secondary body.

[0297] (20) The robotic surgical system according to (19), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0298] Determine that the determined distance falls outside the predetermined range; and

[0299] In response to determining that the determined distance falls outside a predetermined range, the common engagement ratio is disabled.

[0300] (21). The robotic surgical system according to (17), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0301] Determine that the determined distance falls within a predetermined range; and

[0302] In response to determining that the determined distance falls within a predetermined range, the co-engagement ratio is determined based on a predetermined function relating the determined distance to the co-engagement ratio.

[0303] The co-engagement ratio, determined based on the predetermined function, decreases as the determined distance increases.

[0304] (22). The robotic surgical system according to (21), wherein the predetermined function includes: a first function used in response to the primary body being the inner body and a second function used in response to the primary body being the outer body.

[0305] (23). The robotic surgical system according to (17), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0306] The user input device receives a relaxation command to relax the medical device.

[0307] Based on the relaxation command, the primary body is relaxed via the set of instrument manipulators; and

[0308] The secondary body is relaxed via the set of instrument manipulators based on the relaxation command and the common engagement factor.

[0309] (24). The robotic surgical system according to (23), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0310] It has been determined that the primary body has been relaxed; and

[0311] In response to determining that the primary body has been relaxed, the secondary body is relaxed via the set of instrument manipulators based on the relaxation command, independent of the common engagement factor.

[0312] (25). The robotic surgical system according to (17), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0313] Receive a command to disable co-engagement via the aforementioned set of user input devices; and

[0314] In response to receiving the disable co-engagement command, the primary body is engaged via the set of instrument manipulators without engaging the secondary body, based on the engagement command.

[0315] (26). The robotic surgical system according to (17), wherein each of the outer body and the inner body includes: i) a group of one or more tendons configured to apply tension for engagement of the respective outer or inner body, and ii) a group of one or more tension monitors configured to monitor tension in a respective one or more tendons, the memory further storing computer-executable instructions that cause the group of processors to perform the following operations:

[0316] Based on the output received from the corresponding tension monitor, the tension in one of the tendons of at least one of the outer body and the inner body is measured;

[0317] The measured tension is compared with one or more threshold values; and

[0318] In response to a measured tension greater than at least one of the thresholds, the engagement applied to at least one of the outer and inner bodies is modified.

[0319] (27). The robotic surgical system according to (17), wherein the one or more thresholds include a first threshold to a fourth threshold, and the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0320] In response to the measured tension being greater than the first threshold, the engagement speed is slowed down;

[0321] In response to the measured tension being greater than the second threshold, the engagement is limited to the maximum engagement value;

[0322] In response to a measured tension exceeding the third threshold, at least one of the outer and inner bodies is automatically relaxed; and

[0323] A fault condition is generated in response to the measured tension being greater than the fourth threshold.

[0324] (28). The robotic surgical system according to (17), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0325] The medical device is retracted via the set of user input devices.

[0326] In response to receiving the retraction command, the medical device is retracted via the set of device manipulators;

[0327] It is determined that the medical device has retracted a distance greater than a threshold distance; and

[0328] In response to determining that the medical device has retracted a distance greater than a threshold distance, the medical device is automatically relaxed.

[0329] (29). The robotic surgical system according to (17), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0330] Receive a retraction command for the medical device via the set of user input devices;

[0331] Determining that the curvature of the medical device is less than a threshold curvature; and

[0332] In response to determining that the curvature of the medical device is less than the threshold curvature, the medical device is automatically relaxed.

[0333] (30). The robotic surgical system according to (17), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0334] The robotic surgical system receives instructions from the user via the set of user input devices; and

[0335] The co-engagement factor is determined based on the user's instruction.

[0336] (31). A robotic surgical system, comprising:

[0337] A medical device comprising an outer body and an inner body configured to be driven through a cavity in the outer body;

[0338] A group of one or more machine manipulators are configured to control the movement of the outer body and the inner body;

[0339] A group of one or more feedback devices;

[0340] A group of one or more processors;

[0341] At least one computer-readable storage device, which communicates with the set of processors and stores on the at least one computer-readable storage device a model of a mapped portion of the cavity network, a target position relative to the model, and a path along the model from an entry point to the target. The storage device also stores computer-executable instructions that cause the set of processors to perform the following operations:

[0342] Identify portions of the cavity network along a path having a shape that matches the parking assist features; and

[0343] A parking indication is generated on at least a portion of the set of feedback devices at a position corresponding to the identified portion relative to the model, the parking indication indicating the position where the end of the outer body is parked.

[0344] (32). The robotic surgical system according to (31), wherein the group of one or more feedback devices includes at least one of a display, a haptic feedback device and a speaker.

[0345] (33). The robotic surgical system according to (31), wherein the group of one or more feedback devices includes a group of one or more displays, and the memory further stores computer-executable instructions that cause the group of processors to perform the following operations:

[0346] This causes the model and the parking indication at a location corresponding to the identified portion to be presented on at least a portion of the set of displays, the parking indication indicating the position where the end of the outer body is parked.

[0347] (34). The robotic surgical system according to (33), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0348] Determine that the distance from the identified portion to the target is less than a threshold insertion distance of the inner body; and

[0349] In response to determining that the distance from the identified portion to the target is less than the threshold insertion distance of the inner body, the presentation of the parking instruction is performed.

[0350] (35). The robotic surgical system according to (33) further includes a set of one or more user input devices, wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0351] The user input device receives a parking command for the external body to be parked at the current position.

[0352] Determine that the distance from the current position to the target is greater than the threshold insertion distance of the inner body; and

[0353] In response to determining that the distance from the current position to the target is greater than a threshold insertion distance of the inner body, an indication is presented on at least a portion of the set of displays that the target cannot be reached by the inner body when the outer body is parked at the current position.

[0354] (36). The robotic surgical system according to (33), wherein the parking aid features are determined based on previously performed medical procedures, each of the previously performed medical procedures including a path having a shape that matches the parking aid features.

[0355] (37). The robotic surgical system according to (33) further includes a set of one or more user input devices, wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0356] The user input device receives a selection that identifies the portion of the cavity network as the parking instruction.

[0357] (38). The robotic surgical system according to (33), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0358] Identify the location of the cavity diameters in the cavity network along the path within a threshold range of the diameter of the outer body; and

[0359] The portion of the cavity network is identified based on the location identified along the path.

[0360] (39). The robotic surgical system according to (33), wherein the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0361] Detecting the medical device detachment; and

[0362] This causes an instruction to disengage the medical device to be displayed on at least a portion of the set of displays.

[0363] (40). The robotic surgical system according to (33), wherein the medical device further comprises: a set of one or more sensors configured to measure the position of the end of the inner body, and the memory further stores computer-executable instructions that cause the set of processors to perform the following operations:

[0364] The inner body advance command is received via the set of user input devices to advance the inner body.

[0365] The set of instrument manipulators is commanded to advance the inner body;

[0366] In response to a command to the set of instrument manipulators to advance the internal body, and based on outputs from the one or more sensors, determining that the position of the end of the internal body has not moved; and

[0367] Based on the determination that the position of the end of the inner body has not moved, the medical device is detected to have dislodged.

Claims

1. A robotic surgical system, comprising: A medical device comprising an outer body and an inner body configured to be driven through a cavity in the outer body; A group of one or more machine manipulators are configured to control the movement of the outer body and the inner body; A group of one or more user input devices; A group of one or more processors; as well as At least one computer-readable storage device, the at least one computer-readable storage device communicating with the group of processors and storing computer-executable instructions on the at least one computer-readable storage device that cause the group of processors to perform the following operations: Receive engagement commands for engaging the medical device via the set of user input devices; One of the outer body and the inner body is designated as the primary body, and the other of the outer body and the inner body is designated as the secondary body. Determine the distance between the end of the inner body and the end of the outer body; The common junction factor is determined based on the determined distance; The primary body is engaged via the set of instrument manipulators based on the engagement command; as well as Based on the engagement command and the common engagement factor, the secondary body is engaged via the set of instrument manipulators. The common bonding factor varies according to the distance between the end of the inner body and the end of the outer body.

2. The robotic surgical system according to claim 1, wherein, The memory also stores computer-executable instructions that cause the set of processors to perform the following operations: The user input devices receive a drive mode command for driving the medical device in one of a variety of drive modes. The primary ontology and the secondary ontology are processed based on the driving mode command.

3. The robotic surgical system according to claim 1, wherein, The common bonding factor includes a common bonding ratio, which relates the amount of bonding characteristics applied to the primary body to the amount of bonding characteristics applied to the secondary body.

4. The robotic surgical system according to claim 3, wherein, The memory also stores computer-executable instructions that cause the set of processors to perform the following operations: Determine that the determined distance falls outside the predetermined range; and In response to determining that the determined distance falls outside a predetermined range, the common engagement ratio is disabled.

5. The robotic surgical system according to claim 3, wherein, The memory also stores computer-executable instructions that cause the set of processors to perform the following operations: Determine that the determined distance falls within a predetermined range; and In response to determining that the determined distance falls within a predetermined range, the co-engagement ratio is determined based on a predetermined function relating the determined distance to the co-engagement ratio. The co-engagement ratio, determined based on the predetermined function, decreases as the determined distance increases.

6. The robotic surgical system according to claim 5, wherein, The predetermined functions include: a first function used in response to the primary ontology being the inner ontology and a second function used in response to the primary ontology being the outer ontology.

7. The robotic surgical system according to claim 1, wherein, The memory also stores computer-executable instructions that cause the set of processors to perform the following operations: The user input device receives a relaxation command to relax the medical device. Based on the relaxation command, the primary body is relaxed via the set of instrument manipulators; and The secondary body is relaxed via the set of instrument manipulators based on the relaxation command and the common engagement factor.

8. The robotic surgical system according to claim 7, wherein, The memory also stores computer-executable instructions that cause the set of processors to perform the following operations: It has been determined that the primary body has been relaxed; and In response to determining that the primary body has been relaxed, the secondary body is relaxed via the set of instrument manipulators based on the relaxation command, independent of the common engagement factor.

9. The robotic surgical system according to claim 1, wherein, The memory also stores computer-executable instructions that cause the set of processors to perform the following operations: Receive a command to disable co-engagement via the aforementioned set of user input devices; and In response to receiving the disable co-engagement command, the primary body is engaged via the set of instrument manipulators without engaging the secondary body, based on the engagement command.

10. The robotic surgical system according to claim 1, wherein, Each of the outer body and the inner body includes: i) a group of one or more tendons configured to apply tension for engagement of the respective outer or inner body; and ii) a group of one or more tension monitors configured to monitor tension in a respective one or more tendons. The memory also stores computer-executable instructions that cause the group of processors to perform the following operations: Based on the output received from the corresponding tension monitor, the tension in one of the tendons of at least one of the outer body and the inner body is measured; The measured tension is compared with one or more threshold values; and In response to a measured tension greater than at least one of the thresholds, the engagement applied to at least one of the outer and inner bodies is modified.

11. The robotic surgical system according to claim 10, wherein, The one or more thresholds include a first threshold to a fourth threshold, and the memory also stores computer-executable instructions that cause the set of processors to perform the following operations: In response to the measured tension being greater than the first threshold, the engagement speed is slowed down; In response to the measured tension being greater than the second threshold, the engagement is limited to the maximum engagement value; In response to a measured tension exceeding the third threshold, at least one of the outer and inner bodies is automatically relaxed; and A fault condition is generated in response to the measured tension being greater than the fourth threshold.

12. The robotic surgical system according to claim 1, wherein, The memory also stores computer-executable instructions that cause the set of processors to perform the following operations: The medical device is retracted via the set of user input devices. In response to receiving the retraction command, the medical device is retracted via the set of device manipulators; It is determined that the distance the medical device has retracted is greater than a threshold distance; as well as In response to determining that the medical device has retracted a distance greater than a threshold distance, the medical device is automatically relaxed.

13. The robotic surgical system according to claim 1, wherein, The memory also stores computer-executable instructions that cause the set of processors to perform the following operations: Receive a retraction command for the medical device via the set of user input devices; Determining that the curvature of the medical device is less than a threshold curvature; and In response to determining that the curvature of the medical device is less than the threshold curvature, the medical device is automatically relaxed.

14. The robotic surgical system according to claim 1, wherein, The memory also stores computer-executable instructions that cause the set of processors to perform the following operations: The robotic surgical system receives instructions from the user via the set of user input devices; and The co-engagement factor is determined based on the user's instruction.

Images