Systems and methods for bifurcated navigation control of manipulator carts including within computer-assisted medical systems

Abstract

Systems and methods for bifurcated navigation control of a manipulator cart, including within a computer-assisted medical system. The bifurcated navigation control system defines a path in which the manipulator cart will navigate from an initial position to a target position. The system directs the manipulator cart to navigate along at least a portion of a first portion of a path from an initial position to an intermediate position in a first bifurcated navigation control mode associated with a primary control interface that facilitates operator control of both steering and propulsion. The system further directs the manipulator cart to navigate along at least a portion of a second portion of the path from the intermediate position to the target position in a second bifurcated navigation control mode associated with a second control interface that facilitates the operator to control propulsion without controlling steering. In the first and second bifurcated navigation control modes, the system autonomously controls steering of the manipulator cart while allowing an operator to control propulsion of the manipulator cart.

Description

[0001] This application is a divisional application of Chinese patent application No. 202080021708.9 filed on May 29, 2020 and entitled “System and method for forked navigation control of a manipulator cart included in a computer-assisted medical system”.

[0002] Related applications

[0003] This application claims priority to U.S. Provisional Patent Application No. 62 / 855,558, filed on May 31, 2019, entitled “SYSTEM AND METHOD FOR BIFURCATED NAVIGATION CONTROL OF A MANIPULATOR CART INCLUDED IN A COMPUTER-ASSISTED MEDICAL SYSTEM,” the contents of which are incorporated herein by reference in their entirety. Background Art

[0004] Medical operations, such as various types of surgical and non-surgical procedures, can be performed using a computer-assisted medical system. In some instances, such a computer-assisted medical system can include a manipulator cart having one or more arms (e.g., a robotic arm) configured to manipulate instruments for performing medical operations. For example, the manipulator cart can be positioned near a body being operated on (e.g., a patient's body, a cadaver, a training device, an animal, etc.), and various types of medical operations can be performed on the body through the manipulator cart's arms under the guidance of a practitioner (e.g., a clinician, such as a surgeon, etc.) at a console located outside the operating area. In this way, efficient medical operations can be performed.

[0005] In preparation for such computer-assisted medical procedures, a manipulator cart is typically navigated by a human operator from an initial position (e.g., a position where the manipulator cart is maintained when not in use and / or where the manipulator cart is draped and otherwise prepared for the procedure) to a target position of an operating table positioned proximate to the body to be operated on. Unfortunately, however, a variety of challenges (e.g., low visibility provided to the operator, obstacles in the path, narrow parameters characterizing the target position and target configuration of the manipulator cart, etc.) can make it difficult for the operator to effectively navigate the manipulator cart in an efficient manner. Summary of the invention

[0006] This article describes a system and method for bifurcated navigation control of a manipulator cart included in a computer-assisted medical system. For example, one embodiment is implemented as a system including a memory storing instructions and a processor communicatively coupled to the memory and configured to execute the instructions. For example, the instructions can instruct the processor to define a path in which the manipulator cart included in the computer-assisted medical system will navigate from an initial position to a target position. The instructions can also instruct the manipulator cart to navigate along at least a portion of a first portion of a path extending from the initial position to an intermediate position on the path between the initial position and the target position in a first bifurcated navigation control mode. Additionally, the instructions can also instruct the manipulator cart to navigate along at least a portion of a second portion of a path extending from the intermediate position to the target position in a second bifurcated navigation control mode. In the first bifurcated navigation control mode, the processor can autonomously control the steering of the manipulator cart while allowing an operator to control the propulsion of the manipulator cart using a primary control interface, the primary control interface being configured to facilitate the operator to control both the steering and propulsion of the manipulator cart. In the second bifurcated navigation control mode, the processor can also autonomously control the steering of the manipulator cart while allowing the operator to control the propulsion of the manipulator cart. However, in the second bifurcated navigation control mode, operator control may be performed using a second control interface configured to facilitate operator control of propulsion of the manipulator cart without controlling steering thereof.

[0007] Another exemplary embodiment is implemented as a method performed by a bifurcated navigation control system. For example, the method includes defining a path in which a manipulator cart included in a computer-assisted medical system will navigate from an initial position to a target position. The method further includes guiding the manipulator cart to navigate along at least a portion of a first portion of the path extending from the initial position to an intermediate position on the path between the initial position and the target position in a first bifurcated navigation control mode; and guiding the manipulator cart to navigate along at least a portion of a second portion of the path extending from the intermediate position to the target position in a second bifurcated navigation control mode. When performing portions of the method in the first bifurcated navigation control mode, the bifurcated navigation control system may autonomously control the steering of the manipulator cart while allowing an operator to control propulsion of the manipulator cart using a primary control interface, the primary control interface being configured to facilitate the operator to control both steering and propulsion of the manipulator cart. Additionally, when performing portions of the method in the second bifurcated navigation control mode, the bifurcated navigation control system may autonomously control the steering of the manipulator cart while allowing the operator to control propulsion of the manipulator cart using a second control interface, the second control interface being configured to facilitate the operator to control propulsion of the manipulator cart without controlling its steering.

[0008] Still another exemplary embodiment is implemented by a non-transitory computer-readable medium storing instructions that, when executed, direct a processor of a computing device to perform operations described herein. For example, the instructions may direct a processor to define a path in which a manipulator cart included in a computer-assisted medical system will navigate from an initial position to a target position. The instructions may also direct the manipulator cart to navigate along at least a portion of a first portion of a path extending from an initial position to an intermediate position on the path between the initial position and the target position in a first divergent navigation control mode; and to navigate along at least a portion of a second portion of a path extending from the intermediate position to the target position in a second divergent navigation control mode. In the exemplary first divergent navigation control mode, the instructions may direct the processor to autonomously control the steering of the manipulator cart while allowing an operator to control the propulsion of the manipulator cart using a primary control interface, the primary control interface being configured to facilitate the operator to control both the steering and propulsion of the manipulator cart. In the exemplary second divergent navigation control mode, the instructions may direct the processor to autonomously control the steering of the manipulator cart while allowing the operator to control the propulsion of the manipulator cart using a second control interface, the second control interface being configured to facilitate the operator to control the propulsion of the manipulator cart without controlling its steering. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The accompanying drawings illustrate various embodiments and are part of the specification. The embodiments shown are examples only and do not limit the scope of the present disclosure. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements.

[0010] Figure 1 An exemplary bifurcated navigation control system for bifurcated navigation control of a manipulator cart included within a computer-assisted medical system according to principles described herein is illustrated.

[0011] Figure 2 An exemplary computer-assisted medical system according to the principles described herein is illustrated.

[0012] Figure 3 and Figure 4 Respective example operating rooms are shown in which an example manipulator cart is to be navigated along an example path from an initial position to a target position according to the principles described herein.

[0013] Figure 5 and Figure 6 Illustrated are respective views of an exemplary manipulator cart as the manipulator cart navigates from an initial position and corresponding initial configuration to a target position and corresponding target configuration according to principles described herein.

[0014] Figure 7Illustrated are a number of exemplary navigation control modes that may be employed when navigating a manipulator cart from an initial position to a target position in accordance with the principles described herein.

[0015] Figure 8 Illustrated are exemplary components of a primary control interface that may be employed during navigation of a manipulator cart from an initial position to a target position in accordance with the principles described herein.

[0016] Fig. 9 Illustrative components of a second control interface that may be employed during navigation of a manipulator cart from an initial position to a target position according to the principles described herein are illustrated.

[0017] Fig.10 and 11 Illustrated are exemplary aspects of different exemplary navigations of a manipulator cart from an initial position and corresponding initial configuration to a target position and corresponding target configuration according to the principles described herein.

[0018] Fig.12 An exemplary method for bifurcated navigation control of a manipulator cart included within a computer-assisted medical system according to principles described herein is illustrated. DETAILED DESCRIPTION

[0019] Described herein are systems and methods for bifurcated navigation control of a manipulator cart included in a computer-assisted medical system. For example, to facilitate the use of a manipulator cart to perform an operation, the systems and methods described herein provide a bifurcated navigation control mode that automatically assists an operator in navigating the manipulator cart from an initial position (e.g., a storage position) to a target position (e.g., a position where the manipulator cart is ready to perform an operation). The systems and methods described herein also provide a bifurcated navigation control mode that automatically assists an operator in navigating the manipulator cart from an initial orientation and / or configuration (e.g., a stowed configuration) to a target orientation and / or configuration (e.g., an orientation and / or configuration where the manipulator cart is ready to perform an operation). Examples of operations that can be performed using the implementation of the manipulator cart described herein include medical procedures, such as minimally invasive surgical or non-surgical procedures performed by means of an artificial or natural orifice in the body of a living human patient or another suitable body, which can be living or inanimate, biological or non-biological, natural or artificial, etc. (e.g., including but not limited to the body of an animal, a corpse, a training device, etc.).

[0020] In the bifurcated navigation control mode, the processor autonomously controls the steering of the manipulator cart while allowing the operator to control the propulsion of the manipulator cart using the control interface. For example, the operator can guide the manipulator cart forward and backward at a speed comfortable for the operator while the manipulator cart autonomously steers along the appropriate path. In some examples, different bifurcated navigation control modes associated with different control interfaces can be provided to further facilitate navigation of the manipulator cart to a target destination and / or configuration.

[0021] For example, in one implementation, a bifurcated navigation control system may include or be implemented by a memory storing instructions and a processor communicatively coupled to the memory and configured to execute the instructions to 1) define a path in which a manipulator cart included in a computer assisted medical system is to be navigated from an initial position to a target position, 2) in a first bifurcated navigation control mode, direct the manipulator cart to navigate along at least a portion of a first portion of the path extending from the initial position to an intermediate position on the path between the initial position and the target position, and 3) in a second bifurcated navigation control mode, direct the manipulator cart to navigate along at least a portion of a second portion of the path extending from the intermediate position to the target position.

[0022] In the first and second divergent navigation control modes, the processor may autonomously control the steering of the manipulator cart while allowing the operator to control propulsion of the manipulator cart. However, in the first divergent navigation control mode, the processor may allow the operator to control propulsion using a primary control interface that is configured to facilitate operator control of both steering and propulsion of the manipulator cart (e.g., a standard control interface, such as a handlebar-based control interface integrated into the manipulator cart), while in the second divergent navigation control mode, the processor may allow the operator to control propulsion using a secondary control interface that is different and separate from the primary control interface and that is configured to facilitate operator control of propulsion of the manipulator cart without controlling its steering (e.g., an optional control interface based on external stimuli applied to the manipulator cart arms, gesture-based commands, voice commands, a separate input device (e.g., a separate joystick or control pad, etc.). In this manner, one operator may conveniently direct the navigation of a manipulator cart from different locations (e.g., behind the cart and in front of the cart at different times during navigation), or multiple operators (e.g., a "sterile" operator who is scrubbed and allowed access to sterile fields associated with medical procedures, and a "non-sterile" operator who is not scrubbed and not allowed access to sterile fields) may collaborate to direct the navigation of a manipulator cart in other situations.

[0023] Aspects of the bifurcated navigation control systems and methods described herein are primarily directed to implementations of computer-assisted medical systems, such as minimally invasive surgical systems. However, as will be described in more detail below, it should be understood that the inventive aspects disclosed herein can be embodied and implemented in a variety of ways, including by adopting robotic and non-robotic implementations and implementations. Implementations associated with surgical or other medical systems are merely exemplary and should not be considered to limit the scope of the inventive aspects disclosed herein. For example, any reference to surgical instruments, surgical techniques, and / or other such details related to the surgical context will be understood as non-restrictive, because the instruments, systems, and methods described herein can be used for medical treatment or diagnosis, cosmetic improvement, imaging of human or animal anatomical structures, collecting data from human or animal anatomical structures, setting up or removing systems, training medical or non-medical personnel, etc. (any of which may or may not involve surgical aspects). In other examples, the apparatus, systems, and methods described herein may also be used for procedures performed on or with animals, human cadavers, animal cadavers, portions of human or animal anatomy, tissue removed from a human or animal anatomy (which may or may not be re-implanted into the human or animal anatomy), non-tissue artifacts, training models, etc. In still other examples, the apparatus, systems, and methods described herein may be applied for non-medical purposes, including use in industrial systems, general purpose robotics, teleoperation systems, and / or sensing or manipulation of non-tissue artifacts.

[0024] Additionally, while certain examples described herein involve two separate bifurcated navigation control modes, each of which is associated with a different control interface (e.g., the primary and secondary control interfaces described herein), it will be understood that in certain examples, a single bifurcated navigation control mode and a single control interface (e.g., the primary or secondary bifurcated navigation control modes) may be employed throughout navigation of the manipulator cart.

[0025] The bifurcated navigation control systems and methods described herein can provide a variety of benefits. For example, navigating a manipulator cart via a computer-assisted bifurcated navigation control mode (such as those described herein) can result in more efficient and accurate positioning of the manipulator cart in a safer manner with less risk of equipment damage and in a more timely manner than when a standard navigation control mode (i.e., a navigation control mode in which the operator fully directs the steering and propulsion of the manipulator cart) is employed. Different types of operations may require different cart placements relative to the body to be operated on, and in some instances, the operators responsible for navigating the manipulator cart may not be closely involved in certain aspects of the operation, such that these operators may not fully understand the ideal target placement (e.g., target position, target orientation, target configuration, etc.) of the manipulator cart. Additionally, since every minute in a typical surgical operating room is costly, it is important to lose as little time as possible in navigating the manipulator cart to its target position (and / or target orientation or target configuration, if applicable). As a result, conventional navigation of a manipulator cart to a target location may suffer from either or both suboptimal placement (e.g., a non-ideal or inaccurate final placement of a manipulator cart for a given type of operation) and inefficient placement (e.g., an ideal or non-ideal position that takes longer than necessary to achieve). Accordingly, it is a significant benefit that the systems and methods described herein facilitate manipulator cart navigation in a manner that results in optimal and efficient manipulator cart placement.

[0026] Another exemplary benefit of the systems and methods described herein is that a human operator retains propulsion control for manipulator cart navigation even while the system autonomously handles steering control. It may be impractical or otherwise undesirable to fully automate the navigation of a manipulator cart such that both steering and propulsion are autonomously controlled. For example, it may be desirable (e.g., for efficiency reasons, safety reasons, etc.) for one or more operators to be directly involved in the movement of large, heavy, expensive equipment (e.g., a manipulator cart) at all times. Therefore, by bifurcating steering control of the manipulator cart to be performed autonomously by the system, while retaining propulsion control of the manipulator cart as a task performed by a human operator, effective and efficient navigation and positioning of the manipulator cart can be consistently achieved in a convenient, safe, and cost-effective manner that is partially or completely independent of the operator's specific knowledge of ideal cart placement for a given type of operation.

[0027] Still another benefit comes from the use of different bifurcated navigation control modes associated with different types of control interfaces as described herein (e.g., a primary control interface configured to facilitate operator control of both steering and propulsion of a manipulator cart, while a secondary control interface configured to facilitate operator control of propulsion of the manipulator cart without controlling its steering). In some cases, for example, a non-sterile operator may direct the manipulator cart to navigate from an initial position to an intermediate position at or near the boundary of a sterile area (in which operations will be performed). This may represent the largest portion of the total path that the manipulator cart will traverse, so the non-sterile operator may use the primary control interface to have the option of manually steering the manipulator cart along at least a portion of the path if appropriate (e.g., if the operator so chooses, if the bifurcated navigation control system needs to assist in navigating around a specific obstacle or defining a specific portion of the path, etc.).

[0028] Although the last leg of the path between the intermediate position and the target position in the sterile area may be relatively short, it may be undesirable for a non-sterile operator to guide the navigation of this last leg. In fact, another operator (for example, a sterile operator allowed in the sterile area) may be in a better position to guide the manipulator cart to complete the last leg of the path by only allowing the operator to control the propulsion while the system handles the second control interface of the steering control. For example, the second control interface may require the sterile operator to gently pull the arm of the manipulator cart, perform specific gestures, speak specific voice commands, or otherwise indicate that the manipulator cart is configured to move forward on the predefined path along which the cart travels. Then, once the manipulator cart arrives at the target position (and / or target orientation or target configuration, if applicable), the operation can be performed. The example target orientation and configuration of the manipulator cart is that the manipulator cart faces the target object, and the manipulator arm is positioned in a desired manner. The example target configuration of the kinematic structure of the manipulator cart is one or more joints or connecting rods of those joints or connecting rods of the kinematic structure of the manipulator cart at the target position or orientation, or within the range of the target position or orientation. It will be appreciated that this specific example is provided merely as an illustrative scenario, and that in other examples, the same operator and / or the same control interface may be used for the entire navigation of the manipulator cart along a path.

[0029] Various embodiments will now be described in more detail with reference to the accompanying drawings.The systems and methods described herein may provide one or more of the benefits mentioned above as well as various additional and / or optional benefits that will become apparent from the description below.

[0030] Figure 1An exemplary bifurcated navigation control system 100 ("system 100") for bifurcated navigation control of a manipulator cart included in a computer-assisted medical system is illustrated. As will be described and illustrated in greater detail below, "manipulator cart" as used herein may refer to any robot or other system including one or more manipulators (e.g., manipulator arms, etc.) configured to facilitate performing an operation (e.g., a medical operation, such as a surgical procedure, etc.), which is configured to independently navigate from one location to another, rather than being mounted on, for example, a physical track.

[0031] As shown, system 100 may include, but is not limited to, a storage facility 102 and a processing facility 104 selectively and communicatively coupled to each other. Facilities 102 and 104 may each include or be implemented by hardware and / or software components (e.g., processors, memory, communication interfaces, instructions stored in memory for execution by processors, etc.).

[0032] In some instances, facilities 102 and 104 may be integrated into a single device (e.g., a manipulator cart control system, etc.), while in other instances, facilities 102 and 104 may be distributed among multiple devices and / or multiple locations as may be useful for a particular implementation. For example, in one implementation of system 100, the manipulator cart itself may include one or more built-in processors, data storage devices, sensors, communication interfaces, etc. for implementing system 100. In contrast, in other implementations of system 100, some or all of these components may not be integrated into the manipulator cart itself, but may be implemented on other computing systems as may be useful for a particular implementation (e.g., an edge server, a cloud server, a computing device integrated with other components of a computer-assisted medical system including a manipulator cart, etc.).

[0033] The storage facility 102 may maintain (e.g., store) executable data used by the processing facility 104 to perform any functionality described herein. For example, the storage facility 102 may store instructions 106 that may be executed by the processing facility 104 to perform any functionality described herein. The instructions 106 may be implemented by any suitable application, software, code, and / or other executable data instance. The storage facility 102 may also maintain any data received, generated, managed, used, and / or transmitted by the processing facility 104.

[0034] The processing facility 104 may be configured to perform (e.g., execute instructions 106 stored in the storage facility 102 to perform) a variety of processing functions associated with bifurcated navigation control of a manipulator cart. For example, the processing facility 104 may define a path in which the manipulator cart navigates from an initial position to a target position. The processing facility 104 may also direct the manipulator cart to navigate along at least a portion of a first portion of the path extending from the initial position to an intermediate position on the path between the initial position and the target position in a first bifurcated navigation control mode; and in a second bifurcated navigation control mode, navigate along at least a portion of a second portion of the path extending from the intermediate position to the target position. Similarly, in embodiments having a target orientation and / or target configuration, the processing facility 104 may also direct the manipulator cart from an initial orientation and / or initial configuration to a target orientation and / or target configuration in the first bifurcated navigation control mode, along with guiding the manipulator cart from the initial position to the target position.

[0035] In both the first and second divergent navigation control modes, the processing facility 104 may autonomously control the steering of the manipulator cart while allowing the operator to control the propulsion of the manipulator cart using the control interface. As used herein, controlling the "steering" of the manipulator cart may refer to some or all aspects of navigation control, which involves defining the direction of the guidance motion vector (e.g., the direction in which the wheels of the manipulator cart are pointing, etc.). For example, using a standard automobile as an example, steering control of the automobile may involve control applied to the automobile through a steering wheel. In contrast, controlling the "propulsion" of the manipulator cart may refer to some or all aspects of navigation control, which includes defining the magnitude and / or sign (e.g., positive or negative) of the motion vector (e.g., whether and to what extent the wheels of the manipulator cart are turning in a forward or backward direction). For example, again referring to the automobile analogy, propulsion control of the automobile may be performed through the accelerator pedal, brake, and / or gear shift mechanism (e.g., the car is in "park" mode, "drive" mode, "reverse" mode, etc.).

[0036] Although the bifurcation of steering and propulsion control is common, the first and second bifurcated navigation control modes are different from each other because the first bifurcated navigation control mode is configured to allow the operator to control propulsion using the primary control interface, while the second bifurcated navigation control mode is configured to allow the operator to control propulsion using the secondary control interface. As will be described in more detail below, the primary control interface can be a complete or standard manipulator cart control interface that is configured to facilitate the operator to control both steering and propulsion of the manipulator cart. In contrast, the secondary control interface can be an abbreviated or auxiliary control interface. For example, in some instances, the second control interface can be configured to facilitate the operator to control propulsion of the manipulator cart without controlling its steering. In other instances, the second control interface can be configured to facilitate the operator to control both propulsion and steering of the manipulator cart, but can be used to assist the primary control control interface in other ways (including fewer features than the primary control interface), by being used from the side of the manipulator cart opposite to the primary control interface, or in other suitable ways. For example, the second control interface may be configured for use by an operator in a sterile environment (eg, an operator located in a sterile area on the patient side of the manipulator cart, rather than an operator located in a non-sterile area on the other side of the manipulator cart).

[0037] The processing facility 104 may perform the functions described above and other functions described herein in any suitable manner, as will be described in greater detail below.

[0038] In some embodiments, the system 100 (e.g., the processing facility 104) can be configured to provide bifurcated navigation control of the manipulator cart in real time. As used herein, when a function involves or is based on dynamic, time-sensitive information and is performed while the time-sensitive information remains accurate or otherwise relevant, the function can be said to be performed in real time. Due to processing time, communication latency, and other inherent delays in physical systems, when certain functions are performed immediately and without undue delays - even if they are performed after a small delay (e.g., a delay of up to several seconds, etc.), they can be considered to be performed in real time. As an example of real-time functionality, the processing facility 104 can define a path based on the real-time status of obstacles between an initial position and a target position, and can update the path as the obstacle status changes. As another example of real-time functionality, in some embodiments where there is a target orientation and / or configuration for the manipulator cart, the processing facility 104 can define changes in the cart orientation and / or configuration to achieve the target orientation and / or configuration based on the real-time status of the obstacles.

[0039] The system 100 may be used in a variety of contexts such as a variety of different types of technologies that may be used for specific implementations. For example, the system 100 may be used in a medical context, such as in preparation for a computer-assisted medical procedure in which an operation is performed inside the body of any suitable type as described herein. In other implementations, the system 100 may be used in a medical context that is not surgical in nature (e.g., diagnostic or exploratory imaging without a surgical element) or is not used for treatment or diagnosis (e.g., training or other procedures in which such procedures do not involve treatment). Additionally, in certain embodiments, the system 100 may be used in a non-medical context. For example, the system 100 may be used to navigate other types of large, free-moving objects that may or may not fall into the category of a manipulator cart (as that term is used herein).

[0040] To illustrate an exemplary context in which system 100 may be implemented and employed, an exemplary computer-assisted medical system that implements system 100 and includes a manipulator cart will now be described. The computer-assisted medical system described below is illustrative and not limiting. It should be understood that the bifurcated navigation control system and methods described herein may operate as part of or in conjunction with the computer-assisted medical system described herein, other suitable computer-assisted medical systems that may or may not be surgical systems, and / or other suitable medical and / or non-medical systems as may be useful for a particular implementation.

[0041] Figure 2 An exemplary computer-assisted medical system 200 ("medical system 200") that can be used to perform surgical and / or non-surgical medical procedures is illustrated. As shown, the medical system 200 may include a manipulator cart 202, a user control system 204, and an auxiliary system 206 that are communicatively coupled to each other. The medical system 200 may be used by a medical team to perform a computer-assisted medical procedure or other such operation on the body of a patient 208 or on any other body as may be used for a specific implementation. As shown, the medical team may include a first clinician 210-1 (e.g., a surgeon for a surgical procedure), an assistant 210-2, a nurse 210-3, and a second clinician 210-4 (e.g., an anesthesiologist for a surgical procedure), all of whom may be collectively referred to as "team members 210", and each of whom may control, interact with, or otherwise be a user of the medical system 200. Additional, fewer, or optional team members may exist during a medical procedure as may be used for a specific implementation. For example, for some medical procedures, "clinician 210-1" may not be a doctor. Further, the team composition for non-medical procedures will often vary and will include other combinations of members who assume non-medical roles.

[0042] Although Figure 2A minimally invasive medical procedure, such as a minimally invasive surgical procedure, is illustrated as being performed, but it should be understood that the medical system 200 may also be used to perform open medical procedures or other types of operations that may benefit from the accuracy and convenience of the medical system 200. For example, the medical system 200 may also be used to perform operations such as exploratory imaging operations, simulated medical procedures for training purposes, and / or other operations.

[0043] like Figure 2 As shown, the manipulator cart 202 may include a plurality of manipulator arms 212 (e.g., arms 212-1 to 212-4) to which a plurality of instruments (e.g., surgical instruments, other medical instruments, or other instruments) may be coupled. Each instrument may be implemented by any suitable surgical tool (e.g., a tool having tissue-interacting functionality), a medical tool, an imaging device (e.g., an endoscope), a sensing instrument (e.g., a force sensing instrument), a diagnostic instrument, or the like that may be used, for example, for a surgical procedure on a patient 208, a computer-assisted medical procedure (e.g., by being at least partially inserted into a patient 208 and manipulated to perform a computer-assisted medical procedure on the patient 208). Although the manipulator cart 202 is depicted and described herein as including four manipulator arms 212, it should be appreciated that the manipulator cart 202 may include only a single manipulator arm 212 or any other number of manipulator arms as may be useful for a particular implementation. Additionally, it will be appreciated that in some exemplary systems, certain instruments may not be coupled to or controlled by a manipulator arm, but may be hand-held and manually controlled (e.g., by a surgeon, other clinician, or other medical personnel). For example, certain hand-held devices of this type may be coupled to a Figure 2 Computer-assisted instrumentation of the manipulator arm 212 is shown for use in conjunction with or as an alternative to it.

[0044] The manipulator arm 212 and / or an instrument attached to the manipulator arm 212 may include one or more displacement transducers, orientation sensors, and / or position sensors for generating raw (i.e., uncorrected) kinematic information. One or more components of the medical system 200 may be configured to use the kinematic information to track the instrument (e.g., determine its position) and / or control the instrument.

[0045] During the medical operation, the user control system 204 can be configured to facilitate the clinician 210-1 to control the manipulator arm 212 and the instruments attached to the manipulator arm 212. For surgical procedures, for example, the clinician 210-1 can be a surgeon. The clinician 210-1 can interact with the user control system 204 to remotely move or manipulate the manipulator arm 212 and the instruments. To this end, the user control system 204 can provide the clinician 210-1 with an image (e.g., a high-definition 3D image) of the operating area associated with the patient 208 captured by the imaging device. In some instances, the user control system 204 may include a stereoscopic viewer with two displays, wherein the stereoscopic image of the internal view of the body of the patient 208 generated by the stereoscopic imaging device can be viewed by the clinician 210-1. The clinician 210-1 can use the image to perform one or more procedures, wherein one or more instruments are attached to the manipulator arm 212.

[0046] To facilitate control of the instruments, the user control system 204 may include a set of master controllers. These master controllers may be manipulated by the clinician 210-1 to control the movement of the instruments (e.g., by utilizing robotics and / or teleoperation techniques). The master controllers may be configured to detect a variety of hand, wrist, and finger movements of the clinician 210-1. In this way, the clinician 210-1 may intuitively perform procedures using one or more instruments.

[0047] The auxiliary system 206 may include one or more computing devices configured to perform processing operations of the medical system 200. In such a configuration, the one or more computing devices included in the auxiliary system 206 may control and / or coordinate operations performed by various other components of the medical system 200, such as the manipulator cart 202 and / or the user control system 204. For example, a computing device included in the user control system 204 may send instructions to the manipulator cart 202 via one or more computing devices included in the auxiliary system 206. As another example, the auxiliary system 206 may receive and process image data representing an image captured by an imaging device attached to one of the manipulator arms 212.

[0048] In some instances, the assistance system 206 can be configured to present visual content to a team member 210 who may not be able to access images provided to the clinician 210-1 at the user control system 204. To this end, the assistance system 206 can include a display monitor 214 configured to display one or more user interfaces, one or more images (e.g., 2D images) of an operating area, information associated with the patient 208 and / or a medical procedure, and / or any other content as may be useful for a particular implementation. In some instances, the display monitor 214 can display an image of an interior view of the operating area along with additional content (e.g., graphical content, contextual information, etc.). The display monitor 214 can be implemented by a touch screen display with which the team member 210 can interact (e.g., via touch gestures) to provide user input to the medical system 200, or can be implemented by any other type of display screen as may be useful for a particular implementation.

[0049] As will be described in more detail below, the system 100 may be implemented within a medical system 200 or may operate in conjunction with the medical system 200. For example, in some implementations, the system 100 may be implemented entirely by the manipulator cart 202 or by sensors and / or computing components implemented by one or more other components of the medical system 200.

[0050] Manipulator cart 202, user control system 204, and auxiliary system 206 may be communicatively coupled to one another in any suitable manner. Figure 2 As shown, the manipulator cart 202, the user control system 204, and the auxiliary system 206 can be communicatively coupled via a control line 216, which can represent any wired or wireless communication link as may be useful for a particular implementation. To this end, the manipulator cart 202, the user control system 204, and the auxiliary system 206 can each include one or more wired or wireless communication interfaces, such as one or more local area network interfaces, Wi-Fi network interfaces, cellular interfaces, etc.

[0051] To illustrate specific environments and scenarios in which the system 100 may be employed to perform forked navigation control of a manipulator cart, such as the manipulator cart 202, Figure 3 An exemplary operating room 300 is shown in which the manipulator cart 202 is to be navigated along an exemplary path 304 extending from an initial position 306—initial position to a target position 306—target position toward an operating table 302. The operating room 300 provides an environment in which the medical system 200 is configured to perform a medical procedure on a body positioned on the operating table 302 under the direction of one or more members of a medical operating team (e.g., team member 210).

[0052] Figure 3A relatively small operating room 300 having a relatively simple layout is depicted. Specifically, as shown, location 306—initial location is located relatively close to location 306—target location within operating room 300, and no obstacles are shown between location 306—initial location and 306—target location. As a result, path 304 is shown as a relatively simple path having a single gentle curve.

[0053] System 100 can define positions 306 (e.g., positions 306—initial position and 306—target position), as well as other positions described herein, based on any suitable coordinate system or other formal or informal spatial characteristics. For example, a global coordinate system can be defined relative to operating table 302, a door or center of operating room 300, a storage location of manipulator cart 202, or any other suitable origin, and positions 306 can be defined and analyzed relative to this.

[0054] As shown, position 306—initial position can be a position that is tucked out of the way in a corner of operating room 300. For example, this position can be a storage position for manipulator cart 202 when medical system 200 is not in use (e.g., between medical procedures, when non-computer-assisted medical procedures are being performed on operating table 302, etc.). Position 306—initial position can also or alternatively be a preparation position for manipulator cart 202, where manipulator cart 202 can be draped and / or otherwise sterilized and prepared for entry into sterile field 308 of operating room 300 for procedures to be performed therein.

[0055] Position 306—target position may be a position relatively close to operating table 302. Specifically, position 306—target position may be positioned at a location where manipulator cart 202 will be located during a medical procedure performed on a body, thus making position 306—target position closer to operating table 302 than position 306—initial position. Figure 3As depicted, from a top view, the position 306—the target position—can overlap the operating table 302 because the arm 212 incorporated into the manipulator cart 202 can hover above the operating table 302 when the manipulator cart 202 is in an operative position at the operating table 302. In some instances, the position 306—the target position can be specifically selected by an operator of the manipulator cart 202 (e.g., by selecting a point on a map, by selecting one of a plurality of predetermined positions of the manipulator cart 202 associated with different operations, etc.). However, in other instances, the position 306—the target position can be automatically selected by the system 100. For example, the position 306—the target position can be automatically selected in a manner that takes into account the type of operation to be performed, the photographic input representing the room layout, the cannula position detected or expected on the body, a gesture indicating the location of a person near the target area (e.g., a gesture of a bedside surgical team member), and / or any other criteria as may be useful for a particular implementation.

[0056] Other components of the medical system 200 are also shown as being located in the operating room 300 along with the manipulator cart 202. For example, in this example, the user control system 204 is shown as being statically located in another corner of the operating room 300 (although it should be understood that in some examples, the user control system 204 can be mobile), and the auxiliary system 206 is shown as being similarly moved from an initial storage or preparation position on the side of the operating room 300 to a target position within the sterile field 308 adjacent to the operating table 302. It should be understood that other people and objects not explicitly shown may also be present in the operating room 300, but, for the purposes of this example, it should be understood that there are no significant obstacles on or near the path 304 between the position 306—initial position and the position 306—target position.

[0057] exist Figure 3In the illustrated scenario, positions 306—initial position and 306—target position are relatively close to each other within operating room 300. Thus, path 304 has a clear beginning and a clear end and is completely contained within operating room 300. However, in other instances, it will be appreciated that one or both of the initial position and the target position may not be located in the same room as one another, or that the initial position and / or the target position associated with the path may not have been explicitly specified as the manipulator cart 202 navigates along the path. For example, in one instance, the manipulator cart 202 may be located outside of the operating room 300 (e.g., in a different operating room, in a storage room outside of the operating room 300, etc.), such that the initial position of the manipulator cart 202 is outside of the room 300 and the target position is not specified (in which case the manipulator cart 202 within the room 300 will be draped and sterilized) until the manipulator cart 202 enters the operating room 300. Similarly, it will be appreciated that after an operation at the operating table 302 is completed, the initial position and the target position may be swapped such that the manipulator cart 202 may be rolled back to a storage location or another operating room, etc.

[0058] Figure 4 Another exemplary operating room 400 is shown in which manipulator cart 202 is to be navigated via exemplary path 404 from initial position 406 —initial position to target position 406 —target position near operating table 402 within sterile field 408 toward operating table 402 . Figure 4 The scenario shown in is similar to Figure 3 The scene shown in Figure 3 Each principle described can be similarly applied to Figure 4 However, Figure 4 The operating room 400 is shown as Figure 3 4 is much more complex than the operating room 300 of FIG. 4. For example, a plurality of obstacles 410 (e.g., obstacles 410-1 through 410-4) that must be avoided or otherwise considered in planning the path 404 are located on the ground between the location 406, the initial location, and the location 406, the target location. In addition, the obstacle 412 that is shaded with a different hatch-line pattern than the obstacle 410 will be understood to represent an overhead obstacle that must also be considered in defining the path 404.

[0059] As described above, the system 100 can be configured to define a path 404 from the position 406—initial position to the position 406—target position, thereby allowing the system 100 to autonomously control the steering of the manipulator cart 202 when the operator controls propulsion in a divergent navigation control mode. The path can be defined in any manner as may be useful for a particular implementation. For example, the system 100 can further include (e.g., as part of or in addition to the facilities 102 and 104) at least one sensor, such as a visible light image sensor (e.g., a camera, a video capture device, etc.), an infrared image sensor, a depth sensor (e.g., a time of flight (“TOF”) sensor, a light detection and ranging (“LIDAR”) sensor, an ultrasonic sensor, a radar sensor, a laser rangefinder, etc.), or any other sensor configured to detect features of the natural world in a manner that helps define the path of the manipulator cart 202. Such sensors may be integrated with the manipulator cart 202 itself (e.g., such as by being mounted on the arm 212 or a base or other portion of the manipulator cart 202), or may be integrated with other components of the medical system 200 or otherwise located in the operating room 400 independently of the manipulator cart 202 (e.g., mounted on a wall, attached to the operating table 402, etc.).

[0060] In instances where system 100 includes or is in communication with one or more of these types of sensors, system 100 may perform definition of path 404 by receiving sensor data from at least one sensor and defining the path based on the received sensor data. For example, system 100 may receive image data and / or depth data from one or more vantage points representing the real-time locations of obstacles 410 and / or 412. Thus, system 100 may define path 404 in a manner that attempts to avoid or otherwise appropriately address each obstacle 410 and / or 412 detected by the sensors.

[0061] When the system 100 plans and defines a path 404 in which the manipulator cart 202 navigates from position 406—initial position to position 406—target position, the system 100 may consider a variety of factors. For example, the system 100 may detect an obstacle (e.g., one of obstacles 410 or 412) between position 406—initial position and position 406—target position, determine a mobility state of the obstacle, and consider the mobility state of the obstacle in defining the path 404. As used herein, a “mobility state” associated with an obstacle may refer to characteristics of the obstacle related to how easily the obstacle can be repositioned, how much free space is around the obstacle, whether the movement of the obstacle is limited by a cable connected to the obstacle, how likely the obstacle is to reposition itself (e.g., whether the obstacle is a person who appears to be aware of the manipulator cart 202 and is likely to move away), etc.

[0062] Thus, the consideration of the movability state of easily movable obstacles (e.g., stools, observers, etc.) may be performed differently than the consideration of the movability state of more permanent or immovable obstacles (e.g., an anesthesiologist station set up near an operating table, etc.). For example, easily movable obstacles may be considered by planning a path through the obstacle and then indicating to the operator that the obstacle should be moved out of the way, while more permanent or less conveniently movable obstacles may be considered by planning a path around the obstacle to avoid the obstacle completely. In other examples, certain obstacles may be determined to be likely to move on their own (e.g., people who cross the path but have a clear motion vector indicating that they will not remain on the path for a long time, etc.), and, at least when the obstacle is not in close proximity to the manipulator cart, it may be treated with a lower priority to be avoided or may be completely ignored by the system 100 in defining the path 404. Additionally, certain overhead obstacles (e.g., obstacle 412) may be addressed by lowering the operating platform of the manipulator cart 202 to a boom and attaching the arm 212. By lowering the operating platform in this way, it can pass underneath without having to be planned around overhead obstacles.

[0063] As another exemplary factor that the system 100 may consider when defining the path 404, the system 100 may determine a risk factor after detecting an obstacle between the position 406—initial position and the position 406—target position, and consider the risk factor in defining the path 404. As used herein, a "risk factor" may refer to anything related to an obstacle, a manipulator cart being navigated, a path being navigated, or any other aspect of navigation that may be associated with an adverse consequence. For example, one risk factor may be that if the propulsion speed is too high for a particular curve of the path, there may be a risk that the manipulator cart may tip over, thereby causing damage to the manipulator cart. As another example, a risk factor associated with a specific obstacle may relate to the consequence of the manipulator cart contacting the specific obstacle as the manipulator cart traverses the path, thereby causing potential damage to the manipulator cart or a breach of sterility and / or causing potential damage to the specific obstacle.

[0064] Thus, the risk factor for obstacles that are fragile or breakable, expensive or costly to replace, or sterile and intended to remain that way may be considered differently than the risk factor for obstacles that are less critical to contact with the manipulator cart 202 (e.g., a sterile object that a sterile manipulator arm may bump into). For example, a fragile, expensive, or sterile obstacle may be considered by steering around the obstacle with a relatively wide margin to ensure that the manipulator cart 202 does not contact the obstacle even if the obstacle moves or there is a miscalculation in the navigation path 404. In contrast, an object that would not suffer any serious consequences if contacted by the manipulator cart 202 (e.g., a stool that was just knocked over, etc.) may be considered by steering around the obstacle with a relatively narrow margin or no margin, assuming that there is a lower risk of negative consequences if a minor contact occurs.

[0065] In various examples, the system 100 may take into account the obstacle by steering around it, requesting that the obstacle be moved manually (e.g., projecting a light of one color onto the obstacle to be moved and a light of another color onto the obstacle that does not need to be moved), lowering the operating platform to pass under the obstacle, raising the operating platform to allow an arm connected to the boom to pass over the obstacle, changing the posture of one or more arms (e.g., extending or reducing the extension range of the arm, rotating the arm from one side of the cart to the other side, raising or lowering the arm, etc.), or in any other suitable manner. Additionally, in addition to taking into account obstacles, the system 100 may further take into account other factors that affect the navigation of the manipulator cart 202 along the path 404. For example, the system 100 may consider the width of the manipulator cart 202 - determining the width of the path 404 (e.g., including margins), the turning radius of the manipulator cart 202, the paths that people moving in the operating room 400 tend to take to avoid obstacles, whether and how obstacles such as people move, how surfaces of objects are reconfigured (e.g., distinguishing between solid, monolithic objects, objects with joints or flexible areas that allow bending and shape reconfiguration, objects covered by loose drapes, etc.), and the like.

[0066] In some implementations, the system 100 can be configured to define and provide multiple path options to the operator, thereby allowing the operator to participate in defining the path 404 by selecting one of the path options. Specifically, for example, the system 100's definition of the path 404 can include 1) defining multiple different paths in which the manipulator cart 202 can navigate from the position 406—the initial position to the position 406—the target position, and 2) selecting the path 404 from the multiple different paths and based on the operator's input. Additionally or alternatively, the system 100 can request or accept additional operator input to define or modify the path 404 according to the operator's preferences.

[0067] In some scenarios (e.g. Figure 3 ), it may be possible and desirable for system 100 to define the entire path (or multiple options for multiple complete paths) before manipulator cart 202 begins navigating along the path. However, in other instances, it may be difficult, impractical, impossible, or undesirable to completely define path 404 at the outset in this manner. This may occur for a variety of reasons. For example, obstacles may be dynamically moving (e.g., some move out of the way and others get in the way after manipulator cart 202 has begun navigating). As another example, the sensor may provide different representations (e.g., more comprehensive representations) from different locations along path 404 as different portions of the room become obscured or unobstructed from the sensor's vantage point as the sensor moves. Due to the dynamic and mobile nature of the manipulator cart 202, and in some instances, the dynamic and mobile nature of certain sensors and / or obstacles, the system 100 may be configured to update the path 404 as the manipulator cart 202 navigates along the path 404, to dynamically provide different path options that may be identified or detected during navigation, or to otherwise dynamically change the path 404 while the path 404 is being navigated.

[0068] The system 100 may plan the path 404 based on a predefined map of the operating room 400 accessed by the system 100, the predefined map indicating the layout of the operating room 400 (e.g., including wall placement, ceiling height and overhead obstacle height and layout, location and configuration of permanent obstacles, location and configuration of the operating table 402, etc.). However, the system 100 may also add to this predefined information a more dynamic analysis of temporary or new obstacles, dynamic docking considerations (e.g., detected cannula locations, etc.), etc. Sensors positioned in a manner that can be at least partially controlled by the system 100 may be particularly useful for dynamic path definition. For example, a sensor positioned on a boom or a single arm 212 of a manipulator cart 202 may be configured to continuously scan the operating room 400 to capture new perspectives as the manipulator cart 202 navigates along the path 404. In some examples, the system 100 may raise or lower the operating platform, rotate the boom to which the arm 212 is attached, reposition specific links or joints of the arm 212, or otherwise adjust the manipulator cart 202 to collect sensor data from different perspectives. In some examples, the predefined map of the operating room 400 used by the system 100 may have been generated (or may be generated in real time) by the system 100 using simultaneous localization and mapping ("SLAM") technology or other such technology to build and update a three-dimensional model of the operating room 400.

[0069] Even when the system 100 has a certain degree of control over the sensors, in some instances, the system 100 may not be able to locate and positively identify the position 406—the target position—until the manipulator cart 202 is close to the target position. Therefore, the system 100 may direct the manipulator cart 202 to move from a first position where the position 406—the target position is not detectable by the sensors of the manipulator cart 202 to a second position where the position 406—the target position is detectable by the sensors before defining the path 404. When the manipulator cart 202 is in the second position, the system 100 may determine that the sensors detected the position 406—the target position, and in response to determining that the sensors detected the position 406—the target position, the second position may be designated as the position 406—the initial position (i.e., the position where the path 404 begins).

[0070] Figure 4 A path 404 is shown that the manipulator cart 202 can traverse by always using forward propulsion (i.e., moving in a forward direction). However, it will be understood that in some instances, an implementation of the path 404 may be defined to include at least one portion in which the manipulator cart 202 uses backward propulsion to advance along the path from the initial position to the target position. For example, the turning radius of the manipulator cart 202 may be greater than the turning radius required for the path, and the back-and-forth movement of the manipulator cart 202 while turning the manipulator cart 202 may allow the manipulator cart to achieve a tighter turn. As another example, an implementation of the path 404 may have a blind spot from which the manipulator cart 202 may need to exit (i.e., use backward propulsion to move backward) to traverse the new or modified path 404. In such an instance, the manipulator cart 202 may indicate to the operator that the operator should command propulsion in the opposite direction (e.g., start pulling backward instead of pushing forward) in order to reach the target position. In a further instance, the backward portion of the path 404 may be initially planned into the path when the path is defined.

[0071] The medical system 200 is shown as including only a single manipulator cart 202, such as Figure 3 and 4 As shown, it is assumed that the manipulator cart 202 alone performs manipulation tasks associated with the operations performed at the operating tables 302 and 402. However, it will be understood that in some instances, more than one manipulator cart or a manipulator cart and another instrument component of the medical system 200 can be navigated to the operating table for use in the operation. For example, the medical system 200 may further include in addition to the manipulator cart 202 and Figure 2-4Additional instrument components other than those shown in , such as a second manipulator cart (e.g., a manipulator cart having more, less, or equal manipulator arms as manipulator cart 202), instrument components that move freely on tracks or along a floor, instrument components that move in the air (e.g., a drone, etc.), or any other instrument components as may be useful for a particular implementation. In this example, the definition of the path can be performed to take into account additional paths in which the additional instrument components will navigate from additional initial locations to additional target locations. For example, path 404 can be defined as a path that not only avoids obstacles 410 and 412, but also does not interfere with (or risk interfering with) one or more additional instrument components that may be present in a particular implementation of medical system 200 ( Figure 4 To this end, the definition of path 404 may take into account additional paths based on the location of manipulator cart 202 and additional instrument components, the roles that manipulator cart 202 and additional instrument components play when performing an operation at operating table 402, and the like.

[0072] In addition to positions 406 — initial position and 406 — target position, Figure 4 An intermediate position labeled as position 406-intermediate position is also depicted. As described above, system 100 can be configured to switch from a first divergent navigation control mode associated with the primary control interface to a second divergent navigation control mode associated with the secondary control interface upon reaching position 406-intermediate position. Position 406-intermediate position can be at any suitable location, including at an operator-selected location (e.g., a preselected location, any location at which the operator decides to switch from the primary control interface to the secondary control interface during navigation, etc.) or a system-selected location (e.g., a location at a specific proximity to position 406-target location, etc.).

[0073] In some instances, position 406-intermediate position can be associated with the boundary of sterile area 408, which, as shown, can include position 406-target position but not position 406-initial position. It should be understood that sterile area 408 and other sterile areas described herein can be relatively loosely defined areas that are not as Figure 4 For example, the rear end of manipulator cart 202 may enter the area drawn for sterile area 408 even though the rear end of manipulator cart 202 may not be sterile. Thus, the "boundary" of sterile area 408 is to be understood as not necessarily existing at a precise point or boundary in space, but rather may be associated with any location where a potentially non-sterile operator from a non-sterile portion of the operating room can conveniently hand off to a sterile operator authorized in sterile area 408. Thus, as Figure 4As shown in the example of , position 406-intermediate position can depict the point where the manipulator cart 202 begins to enter the sterile area 408 (i.e., the point where a sterile operator standing at the operating table 402 ready to assist in the operation can be close enough to provide external stimulation to the arm 212 of the manipulator cart 202 to guide the manipulator cart 202 to use the second control interface to the position 406-target position the remaining way, as described in more detail below). Position 406-intermediate position can be associated with the boundary of the sterile area 408 as a way to allow a non-sterile operator (i.e., an operator who has not been scrubbed in order to be allowed in the sterile area 408) to cooperate with the sterile operator in the sterile area 408 to navigate the manipulator cart 202 from position 406-initial position all the way to position 406-target position. Thus, during navigation of the manipulator cart 202 along the path 404, any interactive physical components associated with the primary control interface may not be sterilized (i.e., such that a first operator performing operator controls using the primary control interface does not need to be sterilized), while any interactive physical components associated with the secondary control interface (e.g., the arm 212 used to guide the manipulator cart 202 from position 406 - the intermediate position to position 406 - the target position) may be sterilized (i.e., such that a second operator performing operator controls using the secondary control interface does need to be sterilized).

[0074] In addition to directing the base of the manipulator cart 202 to navigate from the position 406—initial position to the position 406—target position along the path 404, the system 100 may also be similarly configured to navigate and move other components of the manipulator cart 202 according to a predefined or dynamically defined path (e.g., the path 404) in addition to or as an alternative to base navigation. That is, during navigation of the manipulator cart 202 toward the position 406—target position and / or once the base of the manipulator cart 202 has reached the position 406—target position, the system 100 may further direct the manipulator cart 202 to re-orient to the target orientation. Alternatively or additionally, during navigation of the manipulator cart 202 toward the position 406—the target position and / or once the base of the manipulator cart 202 has reached the position 406—the target position, the system 100 may further direct other movable components of the manipulator cart 202 (e.g., the operating platform, the boom, one or more manipulator arms 212, etc.) to reorient, pose, and otherwise reconfigure as part of, in addition to, or as an alternative to navigation of the path 404. For example, in some instances, the initial position 406—the initial position may be associated with an initial configuration of the movable components of the manipulator cart 202, the target position 406—the target position may be associated with a target configuration of the movable components of the manipulator cart 202, and defining the path 404 may further include defining a configuration plan with respect to the path 404 in which the movable components of the manipulator cart 202 are to transition from the initial configuration to the target configuration.

[0075] Although the initial position and target position of the manipulator cart 202 have been used to describe Figure 3 and Figure 4 , however, the described techniques may also be applied to transitioning from an initial orientation to a target orientation of the manipulator cart 202, and from an initial configuration to a target configuration of the manipulator cart.

[0076] To illustrate, Figure 5 and Figure 6 A series of side views and a series of top views are respectively shown of the manipulator cart 202 as it navigates from position 406—an initial position and a corresponding initial orientation and configuration—to position 406—a target position and a corresponding target orientation and configuration associated with path 404. Specifically, as shown in FIG. Figure 5 As shown, a series of snapshots 500 (e.g., snapshots 500-1 to 500-3) depict the manipulator cart 202 from a side view at three different points in time as the manipulator cart 202 traverses the path 404. Similarly, as shown in FIG. Figure 6As shown, a series of snapshots 600 (e.g., snapshots 600-2 through 600-4) depict the manipulator cart 202 from a top view at three different points in time, which are to be understood as overlapping the points in time associated with the snapshot 500. Specifically, snapshot 600-2 is to be understood as depicting a top view of the manipulator cart 202 at the same point in time depicted by the side view of snapshot 500-2, and snapshot 600-3 is to be understood as depicting a top view of the manipulator cart 202 at the same point in time depicted by the side view of snapshot 500-3.

[0077] Various movable components of the manipulator cart 202 are labeled in each of the snapshots 500 and 600. Specifically, the manipulator cart 202 is shown as having a base 502 that repositions the entire manipulator cart 202 as it moves across the floor (e.g., driven on wheels or the like that are not explicitly shown). The manipulator cart 202 is further shown as including an operating platform 504 that can be raised and lowered, and to which is attached an arm 506 that can be extended, retracted, pivoted, etc. One or more manipulator arms 212 are attached to the arm 506 and each can be translated laterally, deployed, brought in, rotated, and / or manipulated in any other manner as may be useful for a particular implementation. It should be understood that other movable components not explicitly shown herein may also be present on the manipulator cart 202 or other manipulator cart implementations.

[0078] In some examples, navigation of the base 502 of the manipulator cart 202 along the path 404 can be planned and executed independently and in time separately from reconfiguration of other movable components, such as the operating platform 504, the boom 506, and / or the arm 212. However, in other examples, such as Figure 5 and Figure 6As shown, the navigation plan and the configuration plan of the manipulator cart 202 can be integrated together so that the navigation of the base 502 along the path 404 can be performed concurrently with the reconfiguration of other movable components. In some instances, there can be a seamless and / or integrated transition between the navigation control of the base 502 and the configuration control of the other movable components. In certain implementations, these types of controls can all be treated the same and can be considered part of the navigation along the path 404. Therefore, the manipulator cart 202 may not be considered to have completed navigation along the path 404 until not only the base 502 has reached the position 406—the target position and oriented at the target orientation, but also the operating platform 504, the arm frame 506, the arm 212, and the other movable components of the manipulator cart 202 have been properly positioned and configured to allow instruments to be connected to the arm 212 and docked with cannulas associated with the body on the operating table 402. In some cases, further steps of configuring the manipulator cart 202 may not be required before the manipulator cart 202 is ready to begin a medical procedure. In other cases, further steps of configuring the manipulator cart 202 (eg, further adjusting the arm 212, adjusting instruments coupled to the arm 212, etc.) may be appropriate before the manipulator cart 202 is ready to begin a medical procedure.

[0079] Snapshot 500-1 depicts lateral movement 508 of base 502 along path 404. Movement, such as movement 508, may ultimately result in manipulator cart 202 moving from position 406—initial position to position 406—target position, as described above. In snapshot 500-1, manipulator cart 202 is shown in an initial configuration, which may be associated with a relatively small footprint of manipulator cart 202 (e.g., a minimum amount of space that manipulator cart 202 may occupy), wherein manipulator cart 202 may be configured when not in use (e.g., when in storage, etc.). As shown, in the initial configuration of snapshot 500-1, operating platform 504 may be fully lowered, boom 506 may be fully retracted, and arm 212 may be stowed and brought in as much as possible. This initial configuration may be useful for navigating certain portions of path 404. For example, as shown, the initial configuration may allow the manipulator cart 202 to pass under the overhead obstacle 412, even though if the operating platform 504 were slightly raised, the manipulator cart 202 would not fit under the overhead obstacle 412, and the obstacle 412 would need to be steered around rather than passed under. Thus, the path 404 may incorporate and rely on the configuration plan of the manipulator cart 202, in the sense that the path 404 may be defined with the requirement that the manipulator cart 202 be in a specific configuration (e.g., the initial configuration) when the manipulator cart 202 passes under the obstacle 412. Similarly, the path 404 may incorporate changes in the orientation of the manipulator cart 202 to facilitate moving the manipulator cart 202 to the position 406—the target position.

[0080] In snapshots 500-2 (providing a side view) and 600-2 (providing a top view), manipulator cart 202 has reached surgical table 402 and will be understood to be at position 406, the target position, such that base 502 has completed all lateral movements (e.g., movement 508) defined for path 404. At this point, it may be desirable to position arm 212 above surgical table 402, however, if manipulator cart 202 is still in the initial configuration of snapshot 500-1, arm 212 is too low to the ground and would contact surgical table 402 if arm 506 were extended. Therefore, as shown in snapshot 500-2, operating platform 504 may be raised via movement 510 to lift arm 212 above surgical table 402.

[0081] Thereafter, snapshots 500-3 (providing a side view) and 600-3 (providing a top view) illustrate that the arm 506 can now be safely extended by movement 512 until the arm 212 is hovering above the operating table 402. For example, the arm 506 can be extended until a laser crosshair associated with an arm 212 associated with an imaging instrument (e.g., an endoscope) is properly aligned with a target cannula that has been inserted into a body (not explicitly shown) on the operating table 402. When this alignment is achieved, the arms 212 can each be docked to a respective cannula and instruments can be attached to each arm 212 and inserted into a respective cannula. Once each arm 212 is docked with its respective cannula and any further steps of adjusting the manipulator cart 202 or setting up the medical system 200 are completed, the medical procedure to be performed by the medical system 200 can begin.

[0082] In some instances, the arms 212 may need to be adjusted (e.g., deployed, rotated, reconfigured, etc.) to properly align with the cannula in the manner described above. For illustration, snapshot 600-4 shows how the arms 212 can be deployed from each other and rotated in movement 602 until each arm is properly aligned and ready to dock with its respective cannula so that the medical operation can begin. Once this final movement is complete, the manipulator cart 202 can not only be located at position 406—the target position, but can also be in a target orientation and / or target configuration associated with the specific medical operation to be performed. Thus, the system 100 can determine and indicate to the operator (e.g., through visual, tactile, auditory, or any other suitable type of feedback) that the navigation and configuration of the manipulator cart 202 is complete.

[0083] As described above, the reconfiguration of one or more movable components of the manipulator cart 202 may be performed separately in time from the movement of the base 502 along the path 404, or may be integrated therewith. For example, in one implementation, raising the operating platform 504 via movement 510 and extending the arm 506 via movement 512 may be performed sequentially after movement 508 is completed and the base 502 stops at the position 406, the target position. However, in other implementations, raising the operating platform 504 via movement 510 and extending the arm 506 via movement 512 may be performed concurrently with movement 508 and before the base 502 reaches the position 406, the target position. For example, movement 510 may be initiated once the obstacle 412 has been cleared (passed underneath), and movement 512 may be initiated once the operating platform 504 has been raised enough so that the arm 212 does not come into contact with the operating table 402 or a body placed thereon. In this way, additional valuable time in the operating room 400 can be conserved because the manipulator cart 202 can be in position (and oriented and / or configured, if applicable) even earlier than the start of the operation. Moreover, as another potential way to save time, in some instances, the respective instruments can be attached to the arm 212 and docked with the cannula while the movement 508 of the base 502 is still in progress. This will be discussed below with respect to Fig.10 and Fig.11 The parallelization of various movements associated with navigation along path 404 in these manners is described and illustrated in greater detail.

[0084] While implementations that include both navigation and reconfiguration of movable components of the manipulator cart 202 have been described above, it should be understood that in certain implementations, the system 100 may be responsible for only navigation or reconfiguration of the manipulator cart, but not both. For example, certain implementations of the system 100 may be configured only to assist an operator in steering the manipulator cart to a target location, or only to assist an operator in steering and orienting the manipulator cart to a target location, and once the base of the manipulator cart is in place, the user may direct the configuration of other movable components of the manipulator cart without assistance from the system 100. As another example, an implementation of the system 100 associated with a manipulator system that is, for example, attached to the bedside of an operating table or configured to move along a physical track, etc., may not benefit from navigation assistance from the system 100. Rather, such a manipulator system may benefit only from assistance from the system 100 in configuring movable components such as an operating platform, an arm frame, and / or a manipulator arm.

[0085] As described above, the system 100 can direct the manipulator cart 202 to navigate in different divergent navigation control modes along different portions of the path 404. For example, as described above, the system 100 can direct the manipulator cart 202 to navigate from the position 406 - initial position to the position 406 - intermediate position in a first divergent navigation control mode associated with the primary control interface, and can direct the manipulator cart 202 to navigate from the position 406 - intermediate position to the position 406 - target position in a second divergent navigation control mode associated with the secondary control interface.

[0086] To illustrate, Figure 7 4 shows a plurality of exemplary navigation control modes that can be employed when manipulator cart 202 navigates from position 406—initial position to position 406—target position. Specifically, Figure 7 A representation of a standard navigation control mode 702 and two different divergent navigation control modes 704 (divergent navigation control modes 704-1 and 704-2) are shown. Figure 7 In each of the navigation control mode representations depicted in FIG, an indication of who or what controls the steering and propulsion of manipulator cart 202 is given by placing labels indicating "automatic system control" (i.e., controlled by system 100) or "operator-directed control" (i.e., controlled by the operator) appropriately in the columns for steering ("steering control") and propulsion ("propulsion control"). Additionally, for Figure 7 For each instance of an operator-directed control in a representation of , an indication is given of which control interface the operator will use to apply the control (e.g., "primary control interface" or "secondary control interface").

[0087] It should be understood that, as will be described in more detail below, at certain times, two different operators may use both the primary and secondary control interfaces simultaneously, and commands from one control interface may take precedence over commands from the other. In some implementations, this may occur within yet another forked navigation control mode 704, while in other implementations, such concurrent use of the primary and second control interfaces may be handled in one of the forked navigation control modes 704-1 or 704-2, or modified versions thereof. Additionally, as indicated by the label indicating "No Control Interface Required," any commands received from an operator via the primary or secondary control interface may be effectively ignored and not used for control because control is being handled by the system 100.

[0088] As shown, the standard navigation control mode 702 is characterized by allowing the operator to control both steering and propulsion of the manipulator cart 202 using a primary control interface. The standard navigation control mode 702 is a control mode that is generally used to navigate the manipulator cart from an initial position and / or orientation to a target position and / or orientation. In contrast, the forked navigation control modes 704 are each characterized by a forked steering and propulsion control, allowing the operator to control only propulsion, while the steering is automatically controlled by the system 100. The difference between the forked navigation control modes 704-1 and 704-2 relates to which control interface is used for operator control. Specifically, the primary control interface can be employed in the forked navigation control mode 704-1, and the secondary control interface can be employed in the forked navigation control mode 704-2.

[0089] The primary control interface may be any suitable control interface configured to facilitate operator control of both steering and propulsion of the manipulator cart 202. Figure 7 As shown, the primary control interface used to provide proportional control in the bifurcated navigation control mode 704-1 can be the same control interface that is conventionally used to provide both steering and propulsion control in the standard navigation control mode 702. For example, in some examples, the primary control interface can include a handlebar-based control interface integrated into the manipulator cart 202, or one or more other suitable control mechanisms (e.g., a steering wheel, a joystick, a touch screen control panel, a remote control, a throttle, etc.).

[0090] To illustrate, Figure 8 Exemplary components of a primary control interface 800 that may be employed during navigation of the manipulator cart 202 from at least the position 406—the initial position to the position 406—the intermediate position are shown. As shown, the primary control interface 800 is implemented as a handlebar-based control interface integrated into the manipulator cart 202 (i.e., built into the manipulator cart 202, e.g., on a side of the manipulator cart 202 opposite the arm 212). Figure 8 A variety of controls are shown that may be included in the primary control interface 800 in a particular implementation. Specifically, the primary control interface 800 is shown to include drive switches 802 (i.e., drive switches 802-1 and 802-2) built into the handlebars that allow an operator to conveniently steer and / or direct the propulsion of the manipulator cart 202. For example, in the standard navigation control mode 702, one or both of the drive switches 802 may be pressed to cause the manipulator cart 202 to be driven forward, and turns may be directed by the user pulling or pushing the handlebars to one side or the other. The handlebars and drive switches 802 may thus be used to steer and direct the propulsion of the manipulator cart 202 along a path, such as the path 404.

[0091] Other controls included in the main control interface 800 can facilitate operator-directed movement of other movable components of the manipulator cart 202. For example, the main control interface 800 includes a boom position knob 804 that can be pushed forward to extend the boom 506 (e.g., as shown in movement 512), pulled back to retract the boom 506, pushed left or right to pivot the boom 506, or turned clockwise or counterclockwise to rotate the arm 212 on the boom 506 (e.g., as shown in movement 602). As another example, the main control interface 800 is shown as including a boom height rocker switch 806 that, when pushed forward, can raise the operating platform 504 (e.g., as shown in movement 510), and when pulled back, can lower the operating platform 504. Additionally, as further shown, the primary control interface 800 may include various buttons 808 (e.g., buttons 808-1 and 808-2) for inputting an emergency stop command (button 808-1), turning the manipulator cart 202 on and off (button 808), or performing other operations as may be useful for a particular implementation. Although for purposes of illustration, the primary control interface 800 may include various buttons 808 (e.g., buttons 808-1 and 808-2) for inputting an emergency stop command (button 808-1), turning the manipulator cart 202 on and off (button 808), or performing other operations as may be useful for a particular implementation. Figure 8 Some specific input controllers are explicitly shown in FIG. 8 , but it should be understood that more or fewer controllers performing similar or different functionality as described above may be included in other implementations of the primary controller interface 800 .

[0092] Together with the controller described above, the main control interface 800 is shown as further including a touch screen 810, which is used to accept other types of input commands (e.g., based on the user's selection of a touch screen panel) and provide output information to the operator. Specifically, the touch screen 810 can provide visual feedback to the operator, while other suitable output mechanisms such as speakers, actuators, LEDs, buzzers, etc. can be used to provide visual and / or other types of feedback (e.g., auditory feedback, tactile feedback, etc.). It should be understood that in some instances, the touch screen 810 may not include a touch panel, but may only be implemented as a display monitor that can output information and does not accept operator input. In still other instances, the main control interface may be implemented exclusively by buttons, knobs and / or other controllers, and may not include any display monitor or touch screen.

[0093] Various types of feedback may be provided to the operator via the touch screen 810 and / or other output mechanisms of the primary control interface 800. For example, the system 100 may provide one or more status indicators to an operator exercising operator control of the manipulator cart 202 via the primary control interface 800. For example, the status indicators may indicate the progress of the manipulator cart 202 along the path 404, whether the manipulator cart 202 has completed navigating the path, and the like. The status indicators may take any form as may be useful for a particular implementation. For example, one status indicator may indicate the percentage of the path 404 that has been traversed and indicate when the manipulator cart 202 has reached the position 406—the target position and / or completed the path 404. As another example, another status indicator may display (e.g., from a top view) a depiction of the path 404 (e.g., including the positions 406—the initial position and 406—the target position, obstacles 410 and / or 412, etc.) to indicate the current position of the manipulator cart 202 on the path 404. In some examples, after the manipulator cart 202 has reached position 406—the target position—and has stopped, the status indicator may continue to indicate that the manipulator cart 202 has not completed the navigation path 404 until the target orientation has been achieved, and / or until each movable component has been properly configured according to the configuration plan.

[0094] Other types of feedback provided via the primary control interface 800 may include the current position of the manipulator cart 202 displayed on a map of the path 404 (e.g., a map indicating turns, obstacles displayed in different colors, etc.), an indication that navigation has stopped abruptly, an indication that navigation has attempted to continue forward after navigation is complete, and / or a 2D photographic image or 3D model image captured or derived by sensors on the manipulator cart 202 (e.g., an image depicting the port and / or cannula being approached). In addition, the primary control interface 800 may provide feedback when the system 100 switches from a dynamically generated path to a complete predefined path (e.g., when the position 406—target position is identified and the path to the position 406—target position and / or to the final target configuration is defined), when the controller switches from one control interface to another, or when the navigation control mode switches (e.g., from a standard navigation control mode to a bifurcated navigation control mode, from one bifurcated navigation control mode to another, etc.). As another example, the primary control interface 800 can provide feedback displaying an area in which the manipulator cart 202 can be parked (e.g., a target location or a range of potential target locations), feedback indicating the range of motion of various movable components of the manipulator cart 202, an indication of when the manipulator cart 202 enters the sterile area 408, an indication of which navigation control mode is currently being used, and / or an indication of the direction in which the manipulator cart 202 is moving or is assumed to be moving according to the path 404.

[0095] Tactile or other feedback may also be provided to signal the operator to give up or take control. For example, when the manipulator cart 202 enters a certain vicinity of the operating table 402, when the manipulator cart 202 enters the sterile area 408, or when it is detected that some other threshold or standard has been met, the primary control interface 800 can provide an indication to the non-sterile operator that the sterile operator will take control (e.g., using the primary or secondary control interface). In some instances, an intermediate position may be indicated where a relay from one operator to another is recommended or required. For example, at a specific location or along a specific length of the path 404, control may be performed by either the primary or secondary control interface, or a specific control interface may be required to advance beyond a specific location.

[0096] Haptic or other feedback may also be provided to the operator of the manipulator cart 202. For example, instead of providing visual feedback via the touch screen 810, the system 100 may use the boom 506, the arm 212, or another such component of the manipulator cart 202 to, for example, point in the direction the manipulator cart 202 is currently turning. It should be understood that certain types of feedback described above may be provided in one or more manners other than those explicitly described herein (e.g., visual, audible, haptic, etc.). Additionally, it should be understood that certain types of feedback described above may be provided not only by the primary control interface 800, but also by a secondary control interface, such as will now be described in more detail.

[0097] Back to Figure 7 , the representation of the forked navigation control mode 704-2 shows that when the system 100 is operating in the forked navigation control mode 704-2, the second control interface may be used for operator-directed control of propulsion of the manipulator cart 202 (rather than the primary control interface 800). The second control interface may be any suitable control interface configured to facilitate operator control of propulsion of the manipulator cart 202 without controlling its steering. In other words, the second control interface may be an auxiliary control interface that is suitable and convenient for use in a forked navigation control mode such as the forked navigation control mode 704-2, but may not be suitable for use in the standard navigation control mode 702. For example, in some instances, the second control interface may not be configured to facilitate operator control of steering of the manipulator cart 202, but may only accept forward propulsion commands (possibly with a desired speed and possibly providing feedback to the operator that forward propulsion is about to begin or is in progress), reverse propulsion commands (possibly with a desired speed and possibly providing feedback to the operator that reverse propulsion is about to begin or is in progress), and stop commands (possibly providing feedback to the operator that propulsion is about to stop or has stopped).

[0098] For example, an exemplary second control interface may be a gesture-based control interface. For example, such a control interface may be associated with an image sensor and may be configured to facilitate control of propulsion of manipulator cart 202 by 1) accessing (e.g., via an image sensor) an image of an operator (e.g., a sterile operator standing in front of manipulator cart 202 near operating table 402) performing a control gesture, 2) determining whether the control gesture performed by the operator is a forward propulsion gesture, and 3) determining whether the control gesture performed by the operator is a backward propulsion command. Because steering is controlled by system 100 in forked navigation control mode 704-2, determining whether the operator is providing a forward or backward propulsion command may be sufficient to allow system 100 to guide manipulator cart 202 to navigate from position 406-intermediate position to position 404-target position.

[0099] The gestures used in the gesture-based control interface can be detected by one or more image capture devices mounted on the manipulator cart 202 or other suitable locations within the operating room 400. The gestures can be predefined as any suitable gestures, such as continuous waving with a hand or finger, continuous waving, etc. In some instances, a clear stop gesture (e.g., a flat palm, etc.) can be defined. Additionally or alternatively, the system 100 can interpret the absence of a continuous forward or backward gesture as a stop command (i.e., an indication that the manipulator cart 202 should not continue forward on the path 404 for the time being). In addition to indicating a basic forward or backward propulsion command, the gesture can be further used to indicate a basic propulsion speed. For example, a relatively fast continuous waving of the hand can be interpreted as a forward propulsion command associated with a relatively fast speed, while a relatively slow continuous waving can be interpreted as a forward propulsion command associated with a relatively slow speed.

[0100] Another exemplary second control interface may be a voice control interface. Such a control interface may include, for example, a microphone, and may be configured to facilitate control of propulsion of the manipulator cart 202 by: 1) capturing (e.g., via a microphone) a voice command provided by an operator (e.g., a sterile operator), 2) determining whether the voice command is a forward propulsion command, and 3) determining whether the voice command is a reverse propulsion command. Similarly, because steering is controlled by the system 100 in the forked navigation control mode 704-2, determining whether the operator is providing a forward or reverse propulsion command may be sufficient to allow the system 100 to guide the manipulator cart 202 to navigate from position 406-intermediate position to position 404-target position.

[0101] The voice commands provided through the voice control interface can be predefined as any suitable voice commands, such as continuous repetition of a word or phrase, etc. In some instances, an explicit stop voice command can be defined. Additionally or alternatively, the system 100 can interpret the absence of any word or any specific voice command as a stop command. In addition to indicating basic forward or reverse propulsion commands, voice commands can be further used to indicate basic propulsion speeds.

[0102] Still another exemplary second control interface that may be employed in the forked navigation control mode 704-2 may be an arm guidance interface, in which an operator guides the manipulator cart 202 by pulling or pushing a specific manipulator arm 212. To implement the arm guidance interface, the manipulator cart 202 may include a set of one or more arms (e.g., the manipulator arms 212 described above), and the second control interface 212 may be implemented using a portion of a specified arm of the set of one or more arms 212. For example, the second control interface may be configured to facilitate controlling propulsion by 1) sensing an external stimulus applied to a portion of the specified arm, and 2) in response to the sensed external stimulus satisfying a set of criteria, interpreting the sensed external stimulus as a propulsion command (e.g., one of a forward or reverse propulsion command).

[0103] The portion of the designated arm used to provide guidance to control the propulsion of the manipulator cart 202 along the path 404 can be any suitable portion of any suitable arm. For example, the portion can include a link and / or joint, or can include multiple links and / or joints of a specific manipulator arm 212 of the manipulator cart 202. In some examples, the implementation of the manipulator cart 202 can include a single manipulator arm, so that the group of one or more arms includes only one manipulator arm 212. In this case, since there is only one choice, it can be clear which arm is used to guide the propulsion control. However, in other examples such as those shown above, the group of one or more arms of the manipulator cart 202 can include multiple arms (e.g., four arms in the examples shown above). In these cases, the designated arm can be the only arm of the multiple arms that is designated for implementing the second control interface. Therefore, the system 100 can be configured to indicate to the operator which arm of the multiple arms is the designated arm. For example, the system 100 can visually distinguish the designated arm from another arm of the multiple arms (or from other arms) so that the designated arm can be identified by the operator as the designated arm included in the second control interface.

[0104] To illustrate, Fig. 9206 , which may be employed during navigation of the manipulator cart 202 from at least the position 406 — the intermediate position — to the position 406 — the target position. In some embodiments, the second control interface 900 may also be employed during orientation and / or configuration of the manipulator cart 202 to a target orientation and / or configuration. More specifically, Fig. 9 A specific designated arm 212—designated arm—of the arm frame 506 attached to the manipulator cart 202 is depicted. The arm 212—designated arm is shown as including a plurality of links 902, including a manipulator link 904 (i.e., a distal link configured to hold a manipulator tool) and a plurality of joints 906. As described above, the links 902 and / or joints 906 (including Fig. 9 Any combination of links 902 and / or joints 906 (not explicitly shown or labeled in the drawings) may form the portion of the arm 212—the designated arm—used by the operator to provide guidance for controlling propulsion of the manipulator cart 202. For example, in some implementations, the operator may pull or push the manipulator links 904 to apply an external stimulus (e.g., force, torque, joint deflection, etc.) to one or more joints 906, which may be sensed by the system 100 and interpreted as a forward propulsion command, a reverse propulsion command, etc.

[0105] The system 100 may designate an arm 212-designated arm based on detecting which arm 212 the operator is applying force on (thereby essentially allowing the operator to designate which arm 212 to use), or may automatically determine which arm 212 will be used and visually distinguish the arm 212 from one or more other arms 212 to indicate to the operator that it is the arm used to provide propulsion control. The arm 212-designated arm may be visually distinguished using any aspect of the color, shape, size, posture, position, movement, or other suitable features of the arm 212-designated arm. As an example, the system 100 may visually distinguish the arm 212-designated arm from one or more other arms that may be present by using one or more visual indicators 908, which may be implemented as a colored light emitting diode ("LED"), etc. For example, when the second control interface 900 is not available (e.g., when the primary control interface 800 is still in use) and / or if the specific arm is not designated as the arm to be used in the second control interface in a specific case, the system 100 may make the indicator 908 a color (e.g., red, orange, etc.). Conversely, in different situations, such as when the second control interface 900 is available and the arm 212—the designated arm—is designated as the arm to be used for the second control interface 900, the system 100 can make the indicator 908 a different color (e.g., green, blue, etc.). In other examples, other indicator characteristics (e.g., LED pulse rate, etc.) can be used instead of or in addition to color to provide similar status indications.

[0106] Employing a visual indicator such as the indicator 908 may be an advantageous way for the system 100 to visually distinguish the arm 212-designated arm because the indicator 908 may be illuminated independently of the pose of the arm 212-designated arm. This independence is advantageous because, in some instances, the arm 212-designated arm may be associated with a target configuration that must be achieved before navigation of the manipulator cart 202 will be completed (i.e., a final pose in which the arm is configured to dock with the cannula) (as with other manipulator arms 212 included on the manipulator cart 202). Thus, when an external stimulus is applied to the arm 212-designated arm, it may be desirable for the arm 212-designated arm to remain unclutched (e.g., rigidly engaged in place at a desired pose using brakes or active servo control of the various joints 906). Additionally or alternatively, the arm 212-designated arm may be configured to compromise its pose to some extent (e.g., allowing the manipulator link 904 to be pulled forward relative to the base 502 of the manipulator cart 202 to a threshold amount), but thereafter restore the same pose by navigating and / or reconfiguring the manipulator cart 202 (e.g., by driving the manipulator cart 202 to the same threshold amount to restore the spatial relationship of the base 502 to the manipulator link 904). Visual differentiation methods that do not require a configuration change of the arm 212-designated arm may advantageously allow the arm 212-designated arm to remain rigid, absorb deflections, and / or otherwise maintain or restore its pose in any of these ways.

[0107] However, despite these potential advantages in certain implementations, it should be understood that in other implementations, the system 100 may visually distinguish the arm 212—the designated arm—in other suitable manners that may be suitable and desirable for such implementations (e.g., manners that may or may not involve the indicator 908). For example, the arm 212—the designated arm—may float in a manner that the other arms 212 do not (i.e., clutch for easy reconfiguration rather than being rigidly held in place), may be presented to the operator in a manner that is different from that presented by the other arms (e.g., by extending in front of the other arms), or in any other manner as may be useful for a particular implementation.

[0108] If the arm 212-designated arm remains substantially rigid and unfastened when the operator applies an external stimulus to the arm 212-designated arm (e.g., by pushing or pulling the manipulator link 904, etc.), the system 100 can be configured to temporarily disable (e.g., when the arm 212-designated arm is used for the second control interface 900) a disconnect clutch feature that generally allows manual reconfiguration of the arm 212-designated arm in a manner similar to the operator intentionally engaging a dedicated clutch-type button. Alternatively, the system 100 can be configured without such a disconnect clutch feature.

[0109] In other examples, rather than disabling the disconnect clutch feature, the system 100 may be configured to differentiate between external stimuli intended to exercise propulsion control of the manipulator cart 202 and external stimuli intended to engage the disconnect clutch feature. This differentiation may be implemented in any suitable manner, including by limiting the clutching features of the arm 212-designated arm compared to the clutching features provided to other arms of the manipulator cart 202. For example, the disconnect clutch may be tighter, or the arm 212-designated arm may need to be forced further out of the normal range to engage the disconnect clutch (rather than providing a propulsion command). In still other examples, the indicator 908 may flash a certain color or pattern to help the operator know which functionality is active, etc.

[0110] Regardless of whether the disconnect clutch is temporarily disabled when the arm 212-designated arm is used for the second control interface 900, or remains enabled but with modified functionality when the arm 212-designated arm is used for the second control interface 900, or is handled in another manner, a variety of safety features typically implemented in conjunction with the clutch mode (e.g., including for disconnecting the clutch) can continue to be leveraged for the arm 212-designated arm. For example, speed limiting, high frequency disturbance suppression, calibrated gravity compensation, friction compensation, etc. can be implemented for the arm 212-designated arm when the arm 212-designated arm is grasped or used to provide propulsion control.

[0111] Any suitable external stimulus provided by the operator can be interpreted as any suitable propulsion command by the system 100. For example, the external stimulus can be a deflection of a portion of the arm 212-specified arm, a force or torque applied to a link 902 or joint 906 of the arm 212-specified arm, or another external stimulus as may be useful for a particular implementation.

[0112] If the system 100 detects that any of these types of external stimuli satisfy a set of criteria, the system 100 may interpret the sensed external stimulus as a propulsion command 910 (e.g., a forward propulsion command 910-1, a backward propulsion command 910-2, or another suitable propulsion command not explicitly shown). For example, the set of criteria may include criteria based on the direction of the external stimulus (e.g., whether the operator is pushing or pulling on a portion of the arm 212-designated arm, whether the stimulus is substantially parallel or antiparallel to the direction of the path 404 or substantially orthogonal to the direction of the path 404, etc.). If the direction of the external stimulus is a substantially forward direction (e.g., if the operator pulls the arm 212-designated arm in a direction substantially parallel to the direction of the path 404), the system 100 may interpret the external stimulus as a forward propulsion command 910-1. Conversely, the system 100 may interpret the external stimulus as a backward propulsion command 910-2 in response to the direction of the external stimulus being a substantially backward direction (e.g., if the operator pushes backward on the arm 212-designated arm in a direction substantially antiparallel to the direction of the path 404). If the external stimulus is not substantially parallel or anti-parallel to the direction of path 404, system 100 may interpret the external stimulus as an attempt by the operator to grasp arm 212—the designated arm—or steer manipulator cart 202 in a direction different from path 404, which may trigger a switch to a standard navigation control mode or may be ignored (e.g., if the second control interface is not configured to facilitate steering).

[0113] In some implementations, the external stimulus is interpreted as a forward propulsion command 910-1 in response to directing the direction of the external stimulus to be substantially forward along the path 404 toward the position 406—the target position, rather than, for example, having a significant component of force applied in a direction substantially orthogonal to the path 404. For example, when the parallel component of the external stimulus (i.e., the component that points to moving the manipulator cart 202 to the position 406—the target position by following the remaining portion of the path 404) is greater than the antiparallel component (i.e., the component that points to the opposite of the parallel component along the portion of the path 404 that the manipulator cart 202 has navigated from the position 406—the initial position), the system 100 may determine that the direction of the external stimulus is directed “substantially forward” along the path 404 toward the position 406—the target position. In other examples, the system 100 may determine that the direction of the external stimulus is directed “substantially forward” along the path 404 toward the position 406—the target position when the parallel component of the external stimulus is greater than a predetermined threshold. Because the arm 212—the designated arm—is not a single entity, but is instead comprised of multiple links 902 and joints 906, there may be situations where an external stimulus applied to one link or joint is opposite in direction or otherwise inconsistent with an external stimulus applied to another link or joint. Therefore, other rules may be formulated to further define what is interpreted as a “substantially forward” external stimulus associated with a specific threshold of a specific link 902 and joint 906. In other examples, this potential ambiguity may be resolved by requiring the user to apply a force to the arm 212—the designated arm—at the clutch button.

[0114] In a similar manner as described above with respect to interpreting propulsion command 910 in response to the direction of the external stimulus being substantially forward along path 404, system 100 may also be configured to not interpret the external stimulus as propulsion command 910 in response to the direction of the external stimulus being substantially orthogonal to path 404 toward position 406—the target position. For example, system 100 may ignore the external stimulus and not interpret the external stimulus as propulsion command 910 if the orthogonal component of the external stimulus is greater than the forward or backward component of the external stimulus, or greater than a predetermined threshold.

[0115] In addition to these direction-based criteria, the system 100 may also consider other exemplary criteria in the set of criteria when determining whether a sensed external stimulus is to be interpreted as a propulsion command 910. For example, the set of criteria may include a magnitude of the external stimulus being greater than a threshold magnitude, a frequency of the external stimulus being less than a threshold frequency (or within a predetermined threshold frequency range), etc. In this way, very low magnitude or low frequency external stimuli (e.g., detectable based on gravity or another source other than an intentional external stimulus applied by an operator) may not be interpreted as a propulsion command 910. Additionally, these criteria may ensure that high frequency external stimuli (e.g., detectable if the arm 212—a designated arm—inadvertently collides with an obstacle) are similarly not misinterpreted as a propulsion command 910 intentionally given by the operator.

[0116] In some instances, an external stimulus applied to the arm 212—a portion of a designated arm—may be interpreted as a basic forward propulsion command 910-1 or a basic backward propulsion command 910-2 that indicates only a propulsion direction and not an operator-desired propulsion speed. However, in other instances, the external stimulus applied by the operator may be interpreted not only as a propulsion command 910 in a specific direction (forward or backward) along the path 404, but also as a propulsion command 910 associated with a specific speed magnitude. Specifically, for example, the system 100 may be configured to sense the magnitude of the sensed external stimulus as part of sensing the external stimulus, and as part of the interpretation of the sensed external stimulus, incorporate the magnitude into the propulsion command 910 so as to indicate both the direction and speed magnitude that will characterize the navigation of the manipulator cart 202 along the path 404. Thus, when the operator applies more force to the second control interface 900 (e.g., when the arm 212—specifying that the arm is configured to remain rigid rather than engage the disconnect clutch, and the operator pulls or pushes harder on the manipulator link 904, as determined by the motor force required to match the operator's effort), the manipulator cart 202 can navigate along the path 404 at a higher rate than when the operator applies less force.

[0117] In some examples, the secondary control interface 900 can be configured to provide a consistent user experience for an operator who has also used or will use the primary control interface 800. For example, the impedance of movement of an operator when pushing and pulling a handle of the primary control interface 800 can be configured to be similar or identical to the impedance of movement that an operator might experience when pulling or pushing an arm 212—a portion of a designated arm—in the secondary control interface 900.

[0118] return Figure 7It should be understood that the primary control interface shown in the standard navigation control mode 702 and the forked navigation control mode 704-1 may be implemented by the primary control interface 800 (or another suitable primary control interface), while the secondary control interface shown in the forked navigation control mode 704-2 may be implemented by the secondary control interface 900 (or another suitable secondary control interface). Thus, the different navigation control modes 702 and 704 (and the corresponding control interfaces associated therewith) may be switched between in any suitable manner and for any suitable reason, such as Figure 7 As shown in the multiple transformations 706 in.

[0119] Before the system 100 can operate in any of the navigation control modes 702 or 704 (i.e., before the manipulator cart 202 can begin moving at all), it must first be determined that the manipulator cart 202 is navigable. The manipulator cart 202 may be navigable (i.e., capable of being navigated, moved, driven, etc.) when the cart is powered and the associated control interfaces are operational, the power drives are operational, no cannulas or instruments are attached to any of the arms 212, the parking feet of the manipulator cart 202 are not deployed, and so on. However, even when the manipulator cart 202 is navigable, the system 100 may check additional factors to determine whether any of the divergent navigation control modes 704 are available for use. For example, the system 100 may determine whether a suitable path can be defined from an initial position to a target position (and, if applicable, from an initial orientation or configuration to a target orientation or configuration). As described above, the entire path (e.g., path 404) may not need to be defined from the outset before the manipulator cart 202 begins traversing the path (i.e., because the path can be dynamically defined and updated while the manipulator cart 202 is moving along the path). However, if the sensors used for pathfinding are blocked or the system 100 determines that no suitable path exists (e.g., because no path actually exists, because a path cannot be found, because a path found is too complex or too narrow, etc.), the system 100 can determine (and indicate to the operator through visual, auditory, tactile, or other suitable prompts) that no divergent navigation control mode is currently available and that the standard navigation control mode will be required for navigation.

[0120] If the system 100 determines that the divergent navigation control modes 704 are available for use, it may be desirable for the manipulator cart 202 to navigate the entire path 404 in the divergent navigation control mode 704 (e.g., starting from the divergent navigation control mode 704-1 and thereafter switching to the divergent navigation control mode 704-2). However, in some cases, specific conditions or events may cause the divergent navigation control mode 704 to become unavailable again during navigation, such that the system 100 may switch from one of the divergent navigation control modes 704 to the standard navigation control mode 702 to allow the operator to control both steering and propulsion of the manipulator cart using the primary control interface. The system 100 may transition from the standard navigation control mode 702 back to one of the divergent navigation control modes 704 once any pathfinding issues are resolved (e.g., once obstacles are cleared, once a target location is identified, once a valid path can be determined, etc.) and / or once the operator so directs.

[0121] In some examples, the system 100 may receive operator input to resist autonomous steering of the manipulator cart in one of the divergent navigation control modes 704. For example, the operator input may be received at the primary control interface or the secondary control interface by the operator exerting a force on the handlebar 212 (of the primary control interface 800) or the arm 212-designated arm (of the secondary control interface 900) that is opposite to the steering control being performed by the system 100. In other examples, the operator input may not involve physically resisting the system steering control, but may involve pressing a button, ceasing to press a button (e.g., a "disable switch"), or selecting another suitable user input mechanism by the operator. In response to any such operator input, the system 100 may transition from guiding navigation along a path in the divergent navigation control mode 704 to guiding navigation along a path in the standard navigation control mode 702. The system 100 may also indicate to the operator (e.g., using a haptic rumble or any other feedback prompt described herein) that the navigation control mode has changed.

[0122] In other instances, the switch from the bifurcated navigation control mode 704 to the standard navigation control mode 702 may be based on conditions other than explicit user input. For example, the system 100 may identify a navigation condition associated with navigation of the manipulator cart 202 along the path 404, and based on the navigation condition, may transition from directing navigation along the path 404 in one of the bifurcated navigation control modes 704 (and indicate the transition to the operator) to directing navigation along the path 404 in the standard navigation control mode 702. The identified navigation condition may be any of a variety of conditions and / or events that the system 100 may detect. For example, the navigation condition may be detection that the operating table 402 has moved from its previous position, thus requiring recalculation of the position 406—the target position and the path 404. In instances where the path 404 is dynamically defined while the manipulator cart 202 is navigating along the path, the navigation condition may be that the path 404 becomes untenable, the system 100 fails to find the next portion of the path 404, the line of sight to the position 406—the target position is lost, a sensor is blocked, etc. In still another example, detection of obstacles and general movement of objects within operating room 400 may cause system 100 to reduce the confidence level of a previously defined path until it is determined that the path must be redefined. For example, system 100 may measure the amount of entropy in the operating room, the level of activity in the operating room, etc., and based on the measurements decide whether to proceed in divergent navigation control mode 704 or require navigation in standard navigation control mode 702 so that a human operator can determine how to best handle the complexity.

[0123] Fig.10 and 11 Different exemplary navigation 1000 are illustrated (in Fig.10 in) and 1100 (in Fig.11 Specifically, as shown, each of navigations 1000 and 1100 depicts various navigation aspects as manipulator cart 202 navigates along path 404 from position 406—initial position (and corresponding initial orientation and / or corresponding initial configuration 1002—initial configuration) to position 406—target position (and corresponding target orientation and / or corresponding target configuration 1002—target configuration). Fig.10 and 11The navigation aspects shown include "base navigation" (i.e., movement of the base 502 of the manipulator cart 202 along the path 404); "operating platform height adjustment" (i.e., raising and / or lowering the operating platform 504 in accordance with a configuration plan associated with the path 404); "boom extension" (i.e., extension and retraction of the boom 506 in accordance with the configuration plan); any of a variety of "misc. boom / arm adjustments" (i.e., pivoting, rotating, deploying and other adjustments of the boom 506 and / or arm 212 in accordance with the configuration plan); and whether navigation is being performed in a "first forked navigation control mode" (e.g., a forked navigation control mode 704-1 using a primary control interface such as the primary control interface 800) or in a "second forked navigation control mode" (e.g., a forked navigation control mode 704-2 using a second control interface such as the second control interface 900).

[0124] like Fig.10 and 11 As shown, path 404 may be considered to incorporate not only navigation of substrate 502 along path 404, but also configuration of various movable components of manipulator cart 202 as described above. Thus, in these examples, path 404 may be considered to have been completely traversed or navigated, not when substrate 502 reaches position 406—the target position, but later when substrate 502 is positioned at position 406—the target position—with the target configuration and / or each of the other movable components is properly configured in configuration 1002—the target configuration. It will be appreciated that in other examples, path 404 may extend only until substrate 502 reaches position 406—the target position, and orientation of substrate 502 toward the target orientation or reconfiguration of other movable components may not be integrated with path navigation in this manner.

[0125] exist Fig.10 and 11 4, various time periods 1004 (e.g., time periods 1004-1 to 1004-4) are called out to help illustrate how certain navigation aspects may or may not overlap as path 404 is navigated. Specifically, time period 1004-1 shows that operating platform 504 may be adjusted (e.g., lowered to avoid an overhead obstacle such as obstacle 412, raised to avoid arm 212 contacting operating table 402, etc.) during the same time period as base 502 is being navigated along path 404. For example, operating platform 504 may begin to rise as soon as manipulator cart passes under obstacle 412. Although not shown in this example, it is similarly understood that in certain cases and implementations, other movable components (e.g., arm frame 506, arm 212, etc.) may also be adjusted while base 502 is being navigated along path 404.

[0126] Time period 1004-2 shows that while the operating platform 504 is being adjusted, the boom 506 can also start to extend. For example, once the operating platform 504 has been raised high enough so that the boom 506 will not contact the operating table 402 or the body located thereon when the boom 506 extends, the configuration plan can cause the boom 506 to start extending.

[0127] Time period 1004-3 shows that during the same time period in which the boom 506 is being extended, the boom 506 can also start to pivot and / or the arm 212 can be rotated, deployed, and / or otherwise adjusted. For example, once the cannulas can be seen using the laser crosshairs set on each arm 212, rotation and other automatic adjustments can start to occur to align each arm 212 with the respective cannulas on the body to which the arm will dock.

[0128] Time period 1004-4 is shown as different in navigation 1000 ( Fig.10 in) and navigation 1100 ( Fig.11 in). Specifically, in navigation 1000, time period 1004-4 shows that there can be no overlap between using the first bifurcated navigation control mode and using the second bifurcated navigation control mode. Navigation 1000 thus represents an implementation or scenario in which the primary control interface (associated with the first bifurcated navigation control mode) and the second control interface (associated with the second bifurcated navigation control mode) are configured for non-concurrent use, such that 1) the primary control interface is used to control advancement during a first time period, 2) the second control interface is used to control advancement during a second time period, and 3) the second time period is after the first time period and does not overlap with the first time period.

[0129] In an instance of this type shown in navigation 1000, in some cases, the operator providing the advancement command on each navigation control interface can be the same person. For example, an operator (e.g., "Operator #1") can use the primary control interface 800 to drive the manipulator cart 202 until reaching position 406 - the intermediate position, and then the same operator can walk to the other side of the manipulator cart 202 (e.g., perhaps after scrubbing or finishing scrubbing to be disinfected) to provide an advancement command using a second control interface such as the second control interface 900. Conversely, in other cases, two different operators ("Operator #1" and "Operator #2") can each be associated with navigation control in different bifurcated navigation control modes. As shown, when no operator applies advancement control during time period 1004-4, there can be no navigation or other advancement of the movable components along path 404 (i.e., it is shown that the base navigation and the adjustment of the operating platform height stop moving during this time period).

[0130] In contrast, in Fig.11In navigation 1100 in FIG. 1 , time period 1004-4 shows that there may be overlap between using the first and second bifurcated navigation control modes. Navigation 1100 thus represents an implementation or scenario in which the primary control interface and the second control interface are configured for concurrent use, such that 1) a first operator ("Operator #1") performs operator-controlled propulsion of manipulator cart 202 using the primary control interface during a first time period, 2) a second operator ("Operator #2") different from the first operator performs operator-controlled propulsion of manipulator cart 202 using the second control interface during a second time period, and 3) the second time period overlaps the first time period.

[0131] During time period 1004-4 in the example of navigation 1100, the system 100 may be considered to be in two bifurcated navigation control modes 704 concurrently, or may be considered to be in a third bifurcated navigation control mode (not in the example of navigation 1100) in which control is performed using two different control interfaces (i.e., the primary control interface and the secondary control interface). Figure 7 ). In some cases, concurrent use of the primary and secondary control interfaces may not cause any problems because the propulsion commands received through the two control interfaces may be compatible, identical, or otherwise consistent. For example, if the propulsion commands are generally consistent (e.g., in the same direction and within a threshold amount of velocity, etc.), an average propulsion command reflecting two compatible commands originating from the two control interfaces may be used. However, in other cases, the propulsion commands provided by these concurrent control interfaces may be different, incompatible, or inconsistent. In some instances, the system 100 may take steps to avoid this situation (e.g., when no command is provided through the primary control interface, only the arm 212 to which the operator applies force—the designated arm, etc.) is presented. However, when this situation does occur, the system 100 may prioritize one of the primary and secondary control interfaces over the other to resolve conflicting propulsion commands. This prioritization may be accomplished in any suitable manner as may be useful for a specific implementation.

[0132] For example, when the first time period overlaps with the second time period (i.e., Fig.11 During time period 1004-4 in FIG. 1004-4 ), the system 100 may prioritize the operator's use of the primary control interface to control propulsion of the manipulator cart 202 over the operator's use of the secondary control interface to control propulsion of the manipulator cart 202. The system 100 may prioritize the primary control interface in this manner because the primary control interface is typically and traditionally the primary method of navigating the manipulator cart 202 and because the primary control interface has greater flexibility to allow the operator to exert control as desired or for any other suitable reason (e.g., to switch to steering the manipulator cart 202, to exert an emergency stop, etc.).

[0133] In other examples, although the first time period overlaps with the second time period (i.e., Fig.11 4-4 in the time period 1004-4), the system 100 may prioritize the operator's use of the secondary control interface to control the propulsion of the manipulator cart 202 over the operator's use of the primary control interface to control the propulsion of the manipulator cart 202. The system 100 may prioritize the secondary control interface in this manner because it may be assumed that the operator associated with the secondary control interface has a better vantage point on the manipulator cart 202 because the manipulator cart 202 ultimately approaches the position 406—the target position—or for any other suitable reason.

[0134] In still other examples, the system 100 can identify navigation conditions associated with navigation of the manipulator cart 202 along the path 404 and identify a basic priority based on the navigation conditions. Specifically, for example, based on the identified navigation conditions, the system 100 can prioritize an operator's use of one of the primary and second control interfaces to control propulsion of the manipulator cart 202 (when the first time period overlaps the second time period) over the other of the primary and second control interfaces. The navigation conditions can be a determination of which operator has better visibility of the path 404 and its surrounding space for a specific segment of the navigation 1100, where the manipulator cart 202 is on the path 404 (e.g., whether the manipulator cart 202 has entered the sterile area 408, etc.), a confidence level of how likely it is that propulsion commands received via the second control interface are intended rather than misinterpreted, or any other navigation conditions as may be useful for a specific implementation.

[0135] exist Fig.10 or Fig.11 , once the manipulator cart 202 reaches position 406—target position (and target orientation and / or configuration 1002—target configuration, if applicable), navigation of path 404 may be considered complete. At this point, an operation (e.g., a surgical procedure or other medical operation) may be performed. Thereafter, once the operation is complete, the same path 404 or a newly defined path may be used in a manner similar to but opposite to that described herein to facilitate navigation of the manipulator cart 202 from position 406—target position (and target orientation and / or configuration 1002—target configuration) back to position 406—initial position (and initial orientation and / or configuration 1002—initial configuration). In this case, the manipulator cart 202 may first be carefully backed away from the operating table and / or exited the sterile field using the secondary control interface, after which the manipulator cart 202 may be directed back to a storage location, etc. using the primary control interface.

[0136] Fig.12 An exemplary method 1200 for bifurcated navigation control of a manipulator cart included in a computer-assisted medical system is illustrated. Fig.12Exemplary operations according to one embodiment are shown, but other embodiments may omit, add, reorder and / or modify Fig.12 Any of the actions shown in . Fig.12 One or more operations illustrated in may be performed by, for example, the bifurcated navigation control system of system 100 , any components included therein, and / or any implementation thereof.

[0137] In operation 1202, a bifurcated navigation control system may define a path in which a manipulator cart included in a computer-assisted medical system may navigate from an initial location to a target location. Operation 1202 may be performed in any manner described herein.

[0138] In operation 1204, the forked navigation control system may direct the manipulator cart to navigate along at least a portion of a first portion of a path extending from an initial position to an intermediate position on the path between the initial position and a target position. For example, the forked navigation control system may direct the manipulator cart to navigate along at least a portion of the portion of the path in a first forked navigation control mode in which the forked navigation control system autonomously controls steering of the manipulator cart while allowing an operator to control propulsion of the manipulator cart. In the first forked navigation control mode, a primary control interface may be used to allow the operator to control propulsion, the primary control interface being configured to facilitate operator control of both steering and propulsion of the manipulator cart. Operation 1204 may be performed in any manner described herein.

[0139] In operation 1206, the bifurcated navigation control system may direct the manipulator cart to navigate along at least a portion of a second portion of the path extending from the intermediate location to the target location. For example, the bifurcated navigation control system may direct the manipulator cart to navigate along at least a portion of the portion of the path in a second bifurcated navigation control mode in which the bifurcated navigation control system also autonomously controls the steering of the manipulator while allowing the operator to control propulsion of the manipulator cart. However, in this second bifurcated navigation control mode, a second control interface may be used to allow the operator to control propulsion of the manipulator cart without controlling its steering. Operation 1206 may be performed in any manner described herein.

[0140] In some examples, a non-transitory computer-readable medium storing computer-readable instructions may be provided according to the principles described herein. When executed by a processor of a computing device, the instructions may direct the processor and / or computing device to perform one or more operations, including one or more operations described herein. Any of a variety of known computer-readable media may be used to store and / or transmit such instructions.

[0141] As referred to herein, non-transitory computer-readable media may include any non-transitory storage medium that participates in providing data (e.g., instructions) that can be read and / or executed by a computing device (e.g., by a processor of a computing device). For example, non-transitory computer-readable media may include, but are not limited to, any combination of non-volatile storage media and / or volatile storage media. Exemplary non-volatile storage media include, but are not limited to, read-only memory, flash memory, solid-state drives, magnetic storage devices (e.g., hard disks, floppy disks, magnetic tapes, etc.), ferroelectric random access memory ("RAM"), and optical disks (e.g., laser discs, digital video disks, Blu-ray discs, etc.). Exemplary volatile storage media include, but are not limited to, RAM (e.g., dynamic RAM).

[0142] In some examples, any of the systems and / or other components described herein may be implemented by a computing device including one or more processors, storage devices, input / output modules, communication interfaces, buses, infrastructure, etc. For example, the storage facility 102 of the system 100 may be implemented by a storage device of a computing device, and the processing facility 104 of the system 100 may be implemented by one or more processors of the computing device. In other examples, the systems and / or other components described herein may be implemented by any suitable non-transitory computer-readable medium storing instructions that, when executed, direct a processor of such a computing device to perform the methods and operations described herein.

[0143] In the foregoing description, various exemplary embodiments have been described with reference to the accompanying drawings. However, it is apparent that various modifications and changes may be made thereto, and additional embodiments may be implemented, without departing from the scope of the invention as set forth in the appended claims. For example, certain features of one embodiment described herein may be combined with or replaced by features of another embodiment described herein. Therefore, the description and drawings are to be regarded as illustrative rather than restrictive.

Claims

1. A computer-assisted medical system, comprising: Manipulator carts; a memory for storing instructions; and a processor communicatively coupled to the memory and configured to execute the instructions to: defining a path in which the manipulator cart will navigate from an initial location to a target location; in a first divergent navigation control mode, directing the manipulator cart to navigate along at least a portion of a first portion of the path extending from the initial position to an intermediate position on the path between the initial position and the target position; and instructing the manipulator cart to navigate along at least a portion of a second portion of the path extending from the intermediate position to the target position in a second divergent navigation control mode; in: in the first bifurcated navigation control mode, the processor autonomously controls steering of the manipulator cart while allowing an operator to control propulsion of the manipulator cart using a primary control interface, the primary control interface being configured to facilitate operator control of both steering and propulsion of the manipulator cart; and In the second bifurcated navigation control mode, the processor autonomously controls the steering of the manipulator cart while allowing an operator to control the propulsion of the manipulator cart using a second control interface, the second control interface being configured to facilitate operator control of the propulsion of the manipulator cart without controlling its steering.

2. The computer-assisted medical system according to claim 1, wherein: The initial position is associated with an initial orientation of the manipulator cart; The target position is associated with a target orientation of the manipulator cart; and Wherein the definition of the path for the manipulator cart to navigate from the initial position to the target position further comprises defining an orientation plan with respect to the path, wherein the manipulator cart is reoriented from the initial orientation to the target orientation.

3. The computer-assisted medical system according to claim 1, wherein: The initial position is associated with an initial configuration of a movable component of the manipulator cart; The target position is associated with a target configuration of the movable component of the manipulator cart; and Wherein the definition of the path for the manipulator cart to navigate from the initial position to the target position further comprises defining a configuration plan with respect to the path, wherein the movable component of the manipulator cart transforms from the initial configuration to the target configuration.

4. The computer-assisted medical system according to claim 1, wherein: The computer-assisted medical system is configured to perform a medical procedure on a body positioned on an operating table within an operating room, under the direction of one or more members of a medical operating team; and The target position is a position in which the manipulator cart is located during performance of the medical procedure on the body and is closer to the operating table than the initial position.

5. The computer-assisted medical system of claim 1 , wherein the definition of the path for the manipulator cart to navigate from the initial position to the target position comprises: detecting an obstacle between the initial position and the target position; determining a mobility status of the obstacle; and The mobility status of the obstacles is taken into account in the definition of the path.

6. The computer-assisted medical system of claim 1 , wherein the definition of the path for the manipulator cart to navigate from the initial position to the target position comprises: detecting an obstacle between the initial position and the target position; determining risk factors associated with the obstacle; and The risk factor of the obstacle is taken into account in the definition of the path.

7. The computer-assisted medical system of claim 1, wherein the processor is further configured to execute the instructions to: prior to definition of the path in which the manipulator cart navigates from the initial position to the target position, directing the manipulator cart to move from a first position in which the target position is not detectable by a sensor of the manipulator cart to a second position in which the target position is detectable by the sensor; determining that the sensor detects the target position when the manipulator cart is in the second position; and In response to determining that the sensor detects the target position, the second position is designated as the initial position.

8. The computer-assisted medical system of claim 1 , wherein the definition of the path by which the manipulator cart will navigate from the initial position to the target position comprises: defining a plurality of different paths in which the manipulator cart can navigate from the initial location to the target location; and The path in which the manipulator cart navigates from the initial location to the target location is selected from the plurality of different paths and based on input from an operator.

9. The computer-assisted medical system of claim 1, wherein the processor is further configured to execute the instructions to update the path while the manipulator cart is navigating along the path.

10. The computer-assisted medical system of claim 1, further comprising at least one sensor selected from the group of sensors consisting of: a visible light image sensor, an infrared image sensor, and a depth sensor; wherein the definition of the path for the manipulator cart to navigate from the initial position to the target position comprises: receiving sensor data from the at least one sensor, and The path is defined based on the received sensor data.

Images