Systems and methods for inserting a robotic assembly into an internal body cavity

Abstract

A system and method for inserting a robotic arm assembly into an internal body cavity is provided. The system inserts the robotic arm assembly into an internal cavity of a subject through a trocar. The system determines when a first articulation joint of the robotic arm exits the trocar and reaches a first position within the internal cavity. The first position indicates that the first articulation joint is free to rotate relative to the trocar. The system allows a hand controller to articulate the first articulation joint within the first volume. The system determines when a second articulation joint of the robotic arm exits the trocar and reaches a second position within the internal cavity. The second position indicates that the second articulation joint is free to rotate relative to the trocar. The system allows a hand controller to articulate the second articulation joint within the second volume.

Description

[Background technology]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 433,972, filed December 20, 2022, the entire contents of which are incorporated herein by reference.

[0002] Surgical robotic systems allow a user (also referred to herein as an "operator" or "user") to perform actions using robotically controlled instruments to perform tasks and functions during a procedure. However, conventional surgical robotic systems are limited by linear robotic arm insertion techniques. If obstacles are encountered within the abdominal cavity during linear robotic arm insertion, the linear robotic arm cannot be easily guided around these obstacles, necessitating the use of additional trocar ports to access the surgical target. Alternatively, additional time must be spent clearing these obstacles with laparoscopic tools before the robotic instruments can freely maneuver within the abdominal cavity. Summary of the Invention

[0003] A surgical robotic system is presented. The surgical robotic system includes a camera assembly, a robotic arm assembly having a first robotic arm and a second robotic arm, a hand controller graspable by a user of the surgical robotic system for controlling the first and second robotic arms and the camera assembly, and a trocar. Each of the first and second robotic arms has a plurality of articulation joints. The surgical robotic system also includes a memory storing one or more instructions and a processor configured or programmed to read the one or more instructions stored in the memory. The processor is operably coupled to the robotic arm assembly, the hand controller, and the camera assembly. The processor is configured to enter an insertion mode that allows a user to insert the camera assembly and the robotic arm assembly through the trocar and into an internal cavity of a subject. The processor is further configured to determine that the first robotic arm is inserted into the trocar. The processor is further configured to determine that a first articulation joint of the first robotic arm exits the trocar and reaches a first articulation joint insertion position within the internal cavity. The first articulation joint insertion position indicates that the first articulation joint is free to rotate relative to the trocar. The processor is further configured to allow a first hand controller of the hand controllers to articulate the first articulation joint within the first volume upon determining that the first articulation joint reaches the first articulation joint insertion position. The processor is further configured to determine that a second articulation joint of the plurality of articulation joints of the first robotic arm exits the trocar and reaches a second articulation joint insertion position within the internal cavity. The second articulation joint insertion position indicates that the second articulation joint is free to rotate relative to the trocar. The processor is further configured to allow the first hand controller to articulate the second articulation joint within the second volume upon determining that the second articulation joint reaches the second articulation joint insertion position within the internal cavity.

[0004] A method for inserting a robot assembly of a surgical robotic system into an internal cavity of a subject through a trocar is presented. The robot assembly includes a robotic arm assembly. The method includes inserting a first robotic arm of the robotic arm assembly through the trocar, the first robotic arm having a plurality of articulation joints. The method further includes determining when a first articulation joint of the plurality of articulation joints exits the trocar and reaches a first articulation joint insertion position within the internal cavity. The first articulation joint insertion position indicates that the first articulation joint is free to rotate relative to the trocar. The method includes enabling, via a first hand controller of the surgical robotic system, articulation of the first articulation joint within a first volume within the internal cavity such that the first articulation joint is articulated by the first hand controller within the first volume. The method further includes determining when a second articulation joint of the plurality of articulation joints exits the trocar and reaches a second articulation joint insertion position within the internal cavity. The second articulation joint insertion position indicates that the second articulation joint is free to rotate relative to the trocar. The method further includes enabling, via the first hand controller, articulation of the second articulation joint within a second volume within the internal cavity such that the second articulation joint is articulated by the first hand controller within the second volume.

[0005] These and other features and advantages of the present invention will be more fully understood by reference to the following detailed description taken in conjunction with the accompanying drawings, in which like reference characters refer to like elements throughout the various views, illustrating the principles of the invention and showing relative dimensions, although not to scale, in which: [Brief explanation of the drawings]

[0006] [Figure 1] FIG. 1 illustrates an exemplary surgical robotic system, according to some embodiments. [Figure 2A]FIG. 1 illustrates an exemplary perspective view of a patient cart including a robotic support system coupled to a robotic subsystem of a surgical robotic system, according to some embodiments. [Figure 2B] FIG. 1 is an exemplary perspective view of an exemplary operator console of the surgical robotic system of the present disclosure, in accordance with some embodiments. [Figure 3A] FIG. 1 illustrates an exemplary side view of a surgical robotic system for performing a surgical procedure within an internal cavity of a subject, according to some embodiments. [Figure 3B] FIG. 3B illustrates an exemplary top view of a surgical robotic system for performing a surgical procedure within the internal cavity of the object of FIG. 3A, according to some embodiments. [Figure 4A] FIG. 1 illustrates an exemplary perspective view of a single robotic arm subsystem, according to some embodiments. [Figure 4B] FIG. 4B is an exemplary side perspective view of a single robotic arm of the single robotic arm subsystem of FIG. 4A, according to some embodiments. [Figure 5] FIG. 1 illustrates an exemplary front perspective view of a camera assembly and a robotic arm assembly, according to some embodiments. [Figure 6A] FIG. 1 is an exemplary perspective view of a left hand controller for use in an operator console of a surgical robotic system, according to some embodiments. [Figure 6B] FIG. 1 is an exemplary perspective view of a right hand controller for use in an operator console of a surgical robotic system, according to some embodiments. [Figure 6C] FIG. 1 is a perspective view of a left hand controller for use in connection with an operator console of a surgical robotic system, according to some embodiments. [Figure 6D] FIG. 1 is a perspective view of a right hand controller for use in connection with an operator console of a surgical robotic system, according to some embodiments. [Figure 6E] FIG. 1 is a perspective view of a left hand controller for use in connection with an operator console of a surgical robotic system, according to some embodiments. [Figure 6F] FIG. 1 is a perspective view of a right hand controller for use in connection with an operator console of a surgical robotic system, according to some embodiments. [Figure 7] 1 is an exemplary graphical user interface of a robot pose view including a view frustum view, a robot pose view, and a camera view of a patient cavity and a pair of robotic arms of a surgical robotic system, according to some embodiments; [Figure 8] 10 is a flowchart illustrating steps for inserting a robotic assembly performed by a surgical robotic system, according to some embodiments. [Figure 9A] 10 illustrates the insertion of a camera assembly through a trocar, according to some embodiments. [Figure 9B] 10 illustrates the insertion of a camera assembly through a trocar, according to some embodiments. [Figure 9C] 10 illustrates the insertion of a camera assembly through a trocar, according to some embodiments. [Figure 9D] 10 illustrates the insertion of a camera assembly through a trocar, according to some embodiments. [Figure 10] FIG. 4C illustrates an exemplary robotic arm assembly in simplified segmented form. [Figure 11A] 14 illustrates articulated robotic arm insertion of a first robotic arm through a trocar, according to some embodiments. [Figure 11B] 14 illustrates articulated robotic arm insertion of a first robotic arm through a trocar, according to some embodiments. [Figure 11C] 14 illustrates articulated robotic arm insertion of a first robotic arm through a trocar, according to some embodiments. [Figure 11D] 14 illustrates articulated robotic arm insertion of a first robotic arm through a trocar, according to some embodiments. [Figure 11E] 14 illustrates articulated robotic arm insertion of a first robotic arm through a trocar, according to some embodiments. [Figure 11F] 14 illustrates articulated robotic arm insertion of a first robotic arm through a trocar, according to some embodiments. [Figure 11G] 14 illustrates articulated robotic arm insertion of a first robotic arm through a trocar, according to some embodiments. [Figure 12A] 13 illustrates articulated robotic arm insertion of a second robotic arm through the trocar of FIG. 12, according to some embodiments. [Figure 12B] 13 illustrates articulated robotic arm insertion of a second robotic arm through the trocar of FIG. 12, according to some embodiments. [Figure 12C] 13 illustrates articulated robotic arm insertion of a second robotic arm through the trocar of FIG. 12, according to some embodiments. [Figure 12D] 13 illustrates articulated robotic arm insertion of a second robotic arm through the trocar of FIG. 12, according to some embodiments. [Figure 12E] 13 illustrates articulated robotic arm insertion of a second robotic arm through the trocar of FIG. 12, according to some embodiments. [Figure 12F] 13 illustrates articulated robotic arm insertion of a second robotic arm through the trocar of FIG. 12, according to some embodiments. [Figure 13A] 1 illustrates articulated robot arm insertion of a robot arm while another robot arm is partially inserted. [Figure 13B] 1 illustrates articulated robot arm insertion of a robot arm while another robot arm is partially inserted. [Figure 14] 10 depicts a graphical user interface showing articulated robotic arm insertion into an internal cavity. [Figure 15] 1 depicts a computation module in more detail, according to some embodiments. [Figure 16] 1 depicts an example of a robotic support system (RSS) with axis and translational positioning elements around a cavity of interest, according to some embodiments. [Figure 17A]17A shows an example of various axes of an RSS, including a side view of the insertion axis (FIG. 17A), according to some embodiments. [Figure 17B] 17B shows an example of the various axes of the RSS, including a side view of the roll axis (FIG. 17B), according to some embodiments. [Figure 17C] 17A-17C show an example of the various axes of an RSS, including a side view of the pitch axis (FIG. 17C), according to some embodiments. [Figure 17D] 17D shows an example of the various axes of the RSS, including a top view of the yaw axis (FIG. 17C), according to some embodiments. [Figure 18] 1 depicts a flowchart illustrating steps for repositioning a portion of a robot assembly to avoid an obstacle, according to some embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0007] During insertion of a robotic arm through a trocar into a cavity, such as the abdominal cavity, there may be sensitive tissue and / or obstacles that need to be avoided. Conventional insertion techniques are limited in their ability to remove sensitive tissue and / or obstacles. Laparoscopic or other conventional techniques often require the removal of sensitive tissue and / or obstacles prior to the insertion instrument or robotic arm.

[0008] As taught herein, articulated robotic arm insertion is a technique that allows a surgical robotic system to fully or partially insert a robotic arm of a robotic arm assembly into a cavity, such as the abdominal cavity, while avoiding and / or removing potential obstacles or sensitive tissue. The articulated robotic arm insertion process taught herein allows a user (e.g., a surgeon) to articulate or otherwise move the joints of the robotic arm as the joints remove the distal end of a trocar. This allows the user to adjust a portion of the robotic arm, such as the hand portion or forearm portion, or the upper arm portion, joint by joint during the robotic arm insertion process, e.g., to remove an obstacle such as sensitive tissue. The articulated robotic arm insertion process taught herein allows the user to precisely control the robotic arm position as each joint of the robotic arm removes the distal end of a trocar.

[0009] The articulated robotic arm insertion process taught herein provides the user with an increased range of dexterity during robotic arm insertion through a trocar by leveraging the articulated joints (e.g., wrist joint, elbow joint, and shoulder joint) of each robotic arm to allow maneuverability around obstacles or sensitive tissue. This increased mobility during the robotic assembly insertion process allows the user to guide each or both inserted portions of the robotic arms (e.g., a full arm or partial arms) through a single trocar port to a corresponding target location by allowing the user to navigate the inserted portions of the robotic arms around obstacles. This saves valuable time that would otherwise be spent clearing obstacles before inserting the robotic arms or setting up additional trocar ports to approach the target from other locations and orientations.

[0010] The articulated robotic arm insertion process taught herein also allows for the user to vary the yaw or pitch, or both, of the inserted trocar relative to the insertion axis of the robotic support system (RSS). In this way, during the robotic assembly insertion process, the user can utilize two additional degrees of freedom to navigate through or around delicate tissue and / or obstacles. As described in more detail below, one or more volumes within the cavity can be defined based on the articulated segments of the robotic arm. The ability to vary the yaw or pitch, or both, of the inserted trocar relative to the insertion axis of the RSS allows the user to reposition the plunge location as well as the volume within the internal cavity to facilitate insertion of the robotic arm through or around sensitive tissue and / or obstacles.

[0011] In some embodiments, starting with the insertion of the camera assembly, the user can control the movement of the camera assembly through the trocar and into a cavity, e.g., the abdominal cavity. Following this, the user can sequentially move each robotic arm of the robotic arm assembly through the trocar, past the camera assembly, and around the obstacle. The user can constantly control the camera orientation and plunge position so that the user can observe the progress of the robotic arm insertion. The sequential structure of articulated robotic arm insertion taught herein can focus on a single robotic arm at a time. This can allow for precise focus on the maneuverability of each robotic arm and can reduce the cognitive load on the user by allowing the user to limit their attention to the camera or one of the two robotic arms during the insertion sequence. A sequential insertion process also has the advantage of supporting smaller trocars, thus reducing the incision on the patient and therefore reducing the trauma experienced by the patient. It should be understood that the camera assembly and robotic arm assembly, or the robotic arms of the robotic arm assembly, can be fully or partially inserted in any order or in any particular order. For example, a robotic arm can be followed by a camera assembly, which can then be followed by another robotic arm. The robotic arm may be inserted during the insertion process of the camera assembly (e.g., the camera assembly is partially inserted) or during the insertion process of another robotic arm (e.g., the robotic arm is partially inserted).

[0012] In some embodiments, the articulating robotic arm insertion process, as taught herein, allows the surgical robotic system to track the progress of the articulating robotic arm insertion during the articulating robotic arm insertion process and determine when the distal end of the trocar is removed from each articulation joint. In some embodiments, to determine sufficient clearance of the robotic arm joint past the distal end of the trocar, a point far enough inside the trocar can be selected according to the physical radius of the articulation joint so that the articulation joint moves so that it no longer impacts the trocar, as described in more detail below. In some embodiments, the presently disclosed surgical robotic system can use visual on-screen prompts to guide the user through the articulating robotic arm insertion process, as the user controls each robotic component using a combination of hand controllers and foot pedals, as described in more detail below.

[0013] Before providing additional specifics of articulated robotic arm insertion with respect to FIGS. 8-15, a surgical robotic system in which some embodiments may be employed is described below with respect to FIGS.

[0014] While various embodiments have been taught and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions will occur to those skilled in the art without departing from the invention. It will be understood that various alternatives to the embodiments taught herein may be employed.

[0015] As used in this specification and claims, the singular forms "a," "an," and "the" include plural references unless the context clearly dictates otherwise. It will be further understood that the terms "comprises" and / or "comprising," or "include" and / or "including," as used herein, indicate the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. The term "and / or," as used herein, includes any and all combinations of one or more of the associated listed items.

[0016] Unless specifically stated or apparent from the context, the term "about" as used herein is understood to mean within normal tolerances in the art, for example, within two standard deviations of the mean. "About" may be understood to mean within 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, 0.5%, 0.1%, 0.05%, or 0.01% of the stated value. Unless otherwise apparent from the context, all numerical values ​​provided herein are modified by the term "about."

[0017] Although exemplary embodiments are described herein or in the documents incorporated by reference as employing multiple units to perform exemplary processes, it is understood that the exemplary processes may also be performed by one or more of the modules. Additionally, it is understood that the term controller / controller may refer to a hardware device, including a memory and a processor, specifically programmed to perform processes according to some embodiments described herein. In some embodiments, the memory is configured to store modules, and the processor is specifically configured to execute the modules to perform one or more processes described further below. In some embodiments, multiple different controllers or controllers, or multiple different types of controllers or controllers, may be employed to perform one or more processes. In some embodiments, different controllers or controllers may be implemented in different portions of a surgical robotic system.

[0018] Surgical Robot System Some embodiments may be employed with a surgical robotic system. A system for robotic surgery may include a robotic subsystem. The robotic subsystem includes at least a portion, which may also be referred to herein as a robot assembly, that may be inserted into a patient via a trocar through a single incision point or site. The portion inserted into the patient via the trocar is small enough to be deployed in vivo at a surgical site and is sufficiently maneuverable when inserted into the body so that it can be moved within the body to perform various surgical procedures at multiple different points or sites. Herein, the portion inserted into the body that performs a functional task may be referred to as a surgical robot module, surgical robot module, or robot assembly. A surgical robot module may include multiple different sub-modules or parts that may be separately inserted into a trocar. A surgical robot module, surgical robot module, or robot assembly may include multiple separate robotic arms that are deployable within a patient along different or separate axes. These multiple separate robotic arms may be collectively referred to herein as a robotic arm assembly. Additionally, a surgical camera assembly may also be deployed along a separate axis. The surgical robot module, surgical robot module, or robot assembly may also include a surgical camera assembly. Thus, a surgical robot module or robot assembly employs multiple distinct components, such as a pair of robotic arms and a surgical or robotic camera assembly, each of which is deployable along a different axis and separately operable, steerable, and movable. Robotic arms and camera assemblies that are disposable along separate operable axes are referred to herein as split-arm (SA) architectures. The SA architecture is designed to simplify and increase the efficiency of robotic surgical instrument insertion through a single trocar at a single insertion site, while also assisting in the deployment of the surgical instruments into a surgical-ready state and their subsequent removal through the trocar. As an example, surgical instruments can be inserted through the trocar to access a patient's abdominal cavity and perform surgery in vivo.In some embodiments, a variety of surgical instruments may be used or employed, including, but not limited to, robotic surgical instruments and other surgical instruments known in the art.

[0019] The systems, devices, and methods taught herein may be incorporated into and / or used in conjunction with, for example, robotic surgical devices and related systems taught in U.S. Patent No. 10,285,765 and PCT Patent Application No. PCT / US2020 / 39203, and / or camera assemblies and systems taught in U.S. Patent Application Publication No. 2019 / 0076199, and / or systems and methods for exchanging surgical tools in implantable surgical robotic systems taught in PCT Patent Application No. PCT / US2021 / 058820, the entire contents and teachings of which are incorporated herein by reference. A surgical robot module forming part of the present invention can, in some embodiments, form part of a surgical robotic system that includes a user workstation including appropriate sensors and displays, and a robotic support system (RSS) for interacting with and supporting the robotic subsystem of the present invention. The robotic subsystem, in some embodiments, includes a motor and a surgical robot module including one or more robotic arms and one or more camera assemblies. The robotic arm and camera assembly may form part of a single-support axis robotic system, a split-arm (SA) architecture robotic system, or other arrangements. The robotic support system may provide multiple degrees of freedom so that the robotic module can be maneuvered to a single position or multiple different positions within a patient. In one embodiment, the robotic support system may be attached directly to a surgical table or to the floor or ceiling within an operating room. In another embodiment, attachment is achieved by various fastening means, including, but not limited to, clamps, screws, or combinations thereof. In other embodiments, the structure may be freestanding. The robotic support system may attach a motor assembly coupled to a surgical robot module, including the robotic arm assembly and camera assembly. The motor assembly may include gears, motors, drivetrains, electronics, and the like for powering the components of the surgical robot module.

[0020] The robotic arm assembly and camera assembly are capable of multiple degrees of freedom of movement. According to some embodiments, when the robotic arm assembly and camera assembly are inserted into a patient through a trocar, they are capable of movement in at least the axial, yaw, pitch, and roll directions. The robotic arm of the robotic arm assembly is designed to incorporate and employ multiple degrees of freedom of movement with an end effector attached to its distal end that corresponds to the user's wrist area or joint. In other embodiments, the working end (e.g., end effector end) of the robotic arm is designed to incorporate and use or employ other robotic surgical instruments, such as, for example, the surgical instruments described in U.S. Patent Application Publication No. 2018 / 0221102, the entire contents of which are incorporated herein by reference.

[0021] Similar numerical identifiers are used throughout the figures to refer to the same elements.

[0022] 1 is a schematic diagram of a surgical robotic system 10 in which aspects of the present disclosure may be employed according to some embodiments of the present disclosure. The surgical robotic system 10 includes an operator console 11 and a robotic subsystem 20 according to some embodiments.

[0023] The operator console 11 includes a display 12, an image computation module 14, which may be a three-dimensional (3D) computation module, a hand controller 17 having a sensing and tracking module 16, and a computation module 18. Additionally, the operator console 11 may include a foot pedal array 19 including a plurality of pedals. The image computation module 14 may include a graphical user interface 39. The graphical user interface 39, the controller 26, or the image rendering device 30, or both, may render one or more images or one or more graphical user interface elements on the graphical user interface 39. For example, pillar boxes associated with modes of operating the surgical robotic system 10 or any of the various components of the surgical robotic system 10 may be rendered on the graphical user interface 39. Live video footage captured by a camera assembly 44 may also be rendered on the graphical user interface 39 by the controller 26 or the image rendering device 30.

[0024] The operator console 11 may include a visualization system 9 including a display 12, which may be any selected type of display for displaying information, images, or video generated by the image computation module 14, the computation module 18, and / or the robotic subsystem 20. The display 12 may include or form part of, for example, a head-mounted display (HMD), an augmented reality (AR) display (e.g., an AR display or AR glasses combined with a screen or display), a screen or display, a two-dimensional (2D) screen or display, a three-dimensional (3D) screen or display, and the like. The display 12 may also include an optional sensing and tracking module 16A. In some embodiments, the display 12 may include an image display for outputting images from a camera assembly 44 of the robotic subsystem 20.

[0025] The hand controller 17 is configured to sense the movement of the operator's hand and / or arm to operate the surgical robotic system 10. The hand controller 17 may include a sensing and tracking module 16, circuitry, and / or other hardware. The sensing and tracking module 16 may include one or more sensors or detectors that sense the movement of the operator's hand. In some embodiments, the one or more sensors or detectors that sense the movement of the operator's hand are disposed within the hand controller 17, which is grasped or engaged by the operator's hand. In some embodiments, the one or more sensors or detectors that sense the movement of the operator's hand are coupled to the operator's hand and / or arm. For example, sensors of the sensing and tracking module 16 may be coupled to regions of the hand and / or arm, such as the fingers, wrist region, elbow region, and / or shoulder region. In some embodiments, additional sensors may also be coupled to the operator's head and / or neck region. In some embodiments, the sensing and tracking module 16 may be external and coupled to the hand controller 17 via electrical components and / or mounting hardware. In some embodiments, the optional sensor and tracking module 16A may sense and track movement of one or more of the operator's head, at least a portion of the operator's head, the operator's eyes, or the operator's neck based at least in part on imaging of the operator, in addition to or instead of a single or multiple sensors attached to the operator's body.

[0026] In some embodiments, the sensing and tracking module 16 may employ sensors coupled to the operator's torso or any other body part. In some embodiments, the sensing and tracking module 16 may employ, in addition to sensors, an inertial momentum unit (IMU) having, for example, an accelerometer, a gyroscope, a magnetometer, and a motion processor. The addition of a magnetometer allows for reducing sensor drift around the vertical axis. In some embodiments, the sensing and tracking module 16 also includes sensors placed within surgical equipment such as gloves, surgical scrubs, or a surgical gown. The sensors may be reusable or disposable. In some embodiments, the sensors may be disposed external to the operator, such as in a fixed location in a room such as an operating room. The external sensors 37 may generate external data 36 that may be processed by the computing module 18 and thus employed by the surgical robotic system 10.

[0027] The sensors generate position and / or orientation data indicative of the position and / or orientation of the operator's hands and / or arms. The sensing and tracking modules 16 and / or 16A may be utilized to control the movement (e.g., changes in position and / or orientation) of the camera assembly 44 and robotic arm assembly 42 of the robotic subsystem 20. The tracking and position data 34 generated by the sensing and tracking modules 16 may be communicated to the computation module 18 for processing by at least one processor 22.

[0028] The computing module 18 can determine or calculate the position and / or orientation of the operator's hands or arms, and in some embodiments, the operator's head, from the tracking and position data 34 and 34A and communicate the tracking and position data 34 and 34A to the robotic subsystem 20. The tracking and position data 34, 34A can be processed by the processor 22 and stored, for example, in the storage 24. The tracking and position data 34 and 34A can also be used by the controller 26, which can responsively generate control signals to control the movement of the robotic arm assembly 42 and / or the camera assembly 44. For example, the controller 26 can change the position and / or orientation of at least a portion of the camera assembly 44, at least a portion of the robotic arm assembly 42, or both. In some embodiments, the controller 26 can also adjust the pan and tilt of the camera assembly 44 to follow the movement of the operator's head.

[0029] The robotic subsystem 20 can include a robotic support system (RSS) 46 having a motor 40 and a trocar 50 or trocar mount, a robotic arm assembly 42, and a camera assembly 44. The robotic arm assembly 42 and camera assembly 44 can form part of a single support axis robotic unit as taught and described in U.S. Pat. No. 10,285,765, or can form part of a split-arm (SA) architecture robotic system as taught and described in PCT Patent Application No. PCT / US2020 / 039203, both of which are incorporated herein by reference in their entireties.

[0030] The robotic subsystem 20 can employ multiple distinct robotic arms deployable along different or separate axes. In some embodiments, the camera assembly 44, which can employ multiple distinct camera elements, can also be deployed along a common, separate axis. Thus, the surgical robotic system 10 can employ multiple distinct components, such as a pair of distinct robotic arms and camera assemblies 44 deployable along different axes. In some embodiments, the robotic arm 42 and camera assembly 44 are independently operable, steerable, and movable. The robotic subsystem 20, including the robotic arm assembly 42 and camera assembly 44, is disposable along separate operable axes and is referred to herein as an SA architecture. The SA architecture is designed to simplify and increase the efficiency of insertion of robotic surgical instruments through a single trocar at a single insertion point or site, while also assisting in the deployment of the surgical instruments to a surgical-ready state and subsequent removal of the surgical instruments through the trocar 50, as further described below.

[0031] The RSS 46 may include a motor 40 and a trocar 50 or trocar mount. The RSS 46 may further include a support member supporting the motor 40 coupled to a distal end thereof. The motor 40 may be coupled to each of the camera assembly 44 and the robotic arm assembly 42. The support member may be configured and controlled to move one or more components of the robotic subsystem 20 linearly or in any other selected direction or orientation. In some embodiments, the RSS 46 may be freestanding. In some embodiments, the RSS 46 may include a motor 40 coupled at one end to the robotic subsystem 20 and coupled at an opposite end to an adjustable support member or element.

[0032] The motors 40 can receive control signals generated by the controller 26. The motors 40 can include gears, one or more motors, drive trains, electronics, and the like for powering and driving the robotic arm assembly 42 and the camera assembly 44, individually or together. The motors 40 can also provide mechanical power, electrical power, mechanical communications, and electrical communications to the robotic arm assembly 42, the camera assembly 44, and / or the RSS 46 and other components of the robotic subsystem 20. The motors 40 can be controlled by the computing module 18. Thus, the motors 40 can generate signals to control one or more motors that can in turn control and drive the robotic arm assembly 42, including, for example, the position and orientation of each robotic joint of each robotic arm, and the camera assembly 44. The motors 40 can further provide translational or linear degrees of freedom that are primarily utilized to insert and remove each component of the robotic subsystem 20 through the trocar 50. The motors 40 can also be employed to adjust the insertion depth of each robotic arm 42 of the robotic arm assembly as it is inserted into the patient 100 through the trocar 50.

[0033] The trocar 50 is a medical device that, in some embodiments, may consist of a prong (which may be a metal or plastic, sharp or non-bladed tip), a cannula (essentially a hollow tube), and a seal. The trocar 50 may be used to position at least a portion of the robotic subsystem 20 within an internal cavity of a subject (e.g., a patient) and may withdraw gases and / or fluids from the body cavity. The robotic subsystem 20 may be inserted through the trocar 50 to access the patient's body cavity and perform surgery in vivo. In some embodiments, the robotic subsystem 20 may be supported, at least in part, by the trocar 50 or a trocar mount with multiple degrees of freedom so that the robotic arm assembly 42 and the camera assembly 44 may be maneuvered into a single position or multiple different positions within the patient. In some embodiments, the robotic arm assembly 42 and the camera assembly 44 may move relative to the trocar 50 or the trocar mount with multiple different degrees of freedom so that the robotic arm assembly 42 and the camera assembly 44 may be maneuvered into a single position or multiple different positions within the patient.

[0034] In some embodiments, the RSS 46 may further include an optional controller for processing input data from one or more of the system components (e.g., the display 12, the sensing and tracking module 16, the robotic arm assembly 42, the camera assembly 44, and the like) and for generating control signals in response thereto. The motor 40 may also, in some embodiments, include a storage element for storing data.

[0035] In some embodiments, and in some modes of operation, the robotic arm assembly 42 can be controlled to follow the reduced motion or movement of an operator's arm and / or hand, as sensed by associated sensors. The robotic arm assembly 42 includes a first robotic arm including a first end effector at a distal end of the first robotic arm and a second robotic arm including a second end effector disposed at a distal end of the second robotic arm. In some embodiments, the robotic arm assembly 42 can have portions or regions that can be associated with motions associated with the operator's shoulder, elbow, and wrist joints and fingers. For example, a robotic elbow joint can track the position and orientation of a human elbow, and a robotic wrist joint can track the position and orientation of a human wrist. The robotic arm assembly 42 can also have associated therewith an end region that can terminate in an end effector that tracks the motion of one or more of the operator's fingers, such as, for example, the index finger when a user pinches the index finger and thumb together. In some embodiments, the robotic arm assembly 42 can follow the movement of the operator's arms in some control modes, while the virtual chest of the robotic assembly may remain stationary (e.g., in an instrument control mode). In some embodiments, the position and orientation of the operator's torso is subtracted from the position and orientation of the operator's arms and / or hands. This subtraction allows the operator to move their torso without moving the robotic arms. Further disclosure of control of the movement of individual arms of a robotic assembly is provided in International Patent Applications WO 2022 / 094000 A1 and WO 2021 / 231402 A1, each of which is incorporated herein by reference in its entirety.

[0036] The camera assembly 44 is configured to provide the operator with image data 48, such as a live video feed of the procedure or surgical site, and to allow the operator to operate and control cameras forming part of the camera assembly 44. In some embodiments, the camera assembly 44 can include one or more cameras (e.g., a pair of cameras) whose optical axes are axially spaced a selected distance, known as the inter-camera distance, to provide a stereoscopic view or image of the surgical site. In some embodiments, the operator can control the camera movement through hand movement, either via a sensor coupled to the operator's hand or via a hand controller 17 grasped or held by the operator's hand, thus enabling the operator to obtain a desired view of the surgical site in an intuitive and natural manner. In some embodiments, the operator can additionally control the camera movement through movement of the operator's head. The camera assembly 44 is movable in multiple directions relative to the direction of view, including, for example, yaw, pitch, and roll. In some embodiments, the stereoscopic camera components can be configured to provide a natural and comfortable user experience. In some embodiments, the inter-axial distance between the cameras can be modified to adjust the depth of the surgical site perceived by the operator.

[0037] Image or video data 48 generated by camera assembly 44 may be displayed on display 12. In embodiments in which display 12 includes an HMD, the display may include an embedded sensing and tracking module 16A that acquires raw orientation data in the yaw, pitch, and roll directions of the HMD, as well as position data in Cartesian space (x, y, z) of the HMD. In some embodiments, position and orientation data for the operator's head may be provided via a separate head tracking module. In some embodiments, sensing and tracking module 16A may be used to provide supplemental position and orientation tracking data for the display instead of, or in addition to, the HMD's embedded tracking system. In some embodiments, operator head tracking is not used or employed. In some embodiments, an image of the operator may be used by sensing and tracking module 16A to track at least a portion of the operator's head.

[0038] 2A depicts an exemplary robot assembly 20, also referred to herein as a robot subsystem, of a surgical robotic system 10 integrated into or mounted on a mobile patient cart, according to some embodiments. In some embodiments, the robotic subsystem 20 includes an RSS 46, which in turn includes motors 40, a robotic arm assembly 42 having an end effector 45, a camera assembly 44 having one or more cameras 47, and may also include a trocar 50 or a trocar mount.

[0039] 2B depicts an example of an operator console 11 of the presently disclosed surgical robotic system 10, according to some embodiments. The operator console 11 includes a display 12, a hand controller 17, and one or more additional controllers, such as a foot pedal array 19, for control of the robotic arm assembly 42, the camera assembly 44, and other aspects of the system.

[0040] 2B also depicts the left and right hand controller subsystems 23A, 23B of the operator console. The left hand controller subsystem 23A includes and supports the left hand controller 17A, and the right hand controller subsystem 23B includes and supports the right hand controller 17B. In some embodiments, the left hand controller subsystem 23A can releasably connect or engage with the left hand controller 17A, and the right hand controller subsystem 23B can releasably connect or engage with the right hand controller 17A. In some embodiments, the connections can be both physical and electronic, such that the left and right hand controller subsystems 23A, 23B can receive signals from the left and right hand controllers 17A, 17B, respectively, including signals conveying input received from user selections on buttons or touch input devices of the left or right hand controller 17A, 17B.

[0041] Each of the left hand controller subsystem 23A and the right hand controller subsystem 23B can include components that enable a range of motion for the respective left hand controller 17A and right hand controller 17B, such that the left hand controller 17A and the right hand controller 17B can translate or displace in three dimensions, and can additionally move in the roll, pitch, and yaw directions. Additionally, each of the left hand controller subsystem 23A and the right hand controller subsystem 23B can register the movement of the respective left hand controller 17A and right hand controller 17B in each of the aforementioned directions and can send signals providing such movement information to the processor 22 (as shown in FIG. 1 ) of the surgical robotic system 10.

[0042] In some embodiments, each of the left hand controller subsystem 23A and the right hand controller subsystem 23B may be configured to receive and connect to or engage a different hand controller (not shown). For example, hand controllers with different configurations of buttons and touch input devices may be provided. In addition, hand controllers with different shapes may be provided. The hand controllers may be selected for compatibility with a particular surgical robotic system or a particular surgical robotic procedure, or may be selected based on operator preference with respect to buttons and input devices or with respect to the shape of the hand controller to provide greater comfort and ease to the operator.

[0043] FIG. 3A schematically depicts a side view of a surgical robotic system 10 performing surgery within an internal cavity 104 of an object 100, according to some embodiments and for some surgical procedures. FIG. 3B schematically depicts a top view of a surgical robotic system 10 performing surgery within an internal cavity 104 of an object 100. The object 100 (e.g., a patient) is positioned on an operating table 102 (e.g., a surgical operating table 102). In some embodiments and for some surgical procedures, an incision is made in the patient 100 to gain access to the internal cavity 104. A trocar 50 is then inserted into the patient 100 at a selected location to provide access to the internal cavity 104 or surgical site. The RSS 46 can then be maneuvered into position on the patient 100 and trocar 50. In some embodiments, the RSS 46 includes a trocar mount that is attached to the trocar 50. The camera assembly 44 and the robotic arm assembly 42 are coupled to the motor 40 and may be inserted individually and / or sequentially through the trocar 50 into the patient 100, and thus into the internal cavity 104 of the patient 100. While the camera assembly 44 and the robotic arm assembly 42 may include some portions that remain outside the subject's body during use, references to inserting the robotic arm assembly 42 and / or the camera assembly 44 into the internal cavity of the subject and disposing the robotic arm assembly 42 and / or the camera assembly 44 within the internal cavity of the subject refer to the portions of the robotic arm assembly 42 and the camera assembly 44 that are intended to be within the internal cavity of the subject during use. The sequential insertion method has the advantage of supporting smaller trocars, thus allowing for smaller incisions to be made in the patient 100, thus reducing trauma experienced by the patient 100. In some embodiments, the camera assembly 44 and the robotic arm assembly 42 may be inserted in any order or in a specific order. In some embodiments, the camera assembly 44 may be followed by a first robotic arm 42A of the robotic arm assembly 42, followed by a second robotic arm 42B of the robotic arm assembly 42, all of which may be inserted into the trocar 50 and thus into the internal cavity 104.Once inserted into the patient 100 , the RSS 46 can manually or automatically move the robotic arm assembly 42 and camera assembly 44 to the surgical site controlled by the operator console 11 .

[0044] Further disclosure regarding controlling the movement of individual arms of a robotic arm assembly is provided in International Patent Applications WO 2022 / 094000 A1 and WO 2021 / 231402 A1, each of which is incorporated herein by reference in its entirety. 4A is a perspective view of the robotic arm subassembly 21, according to some embodiments. The robotic arm subassembly 21 includes a robotic arm assembly 42, an end effector 45 having an instrument tip 120 (e.g., monopolar scissors, a needle driver / holder, a bipolar grasper, or any other suitable tool), and a support tube 122 that supports the robotic arm assembly 42. A distal end of the support tube 122 is coupled to the robotic arm assembly 42, and a proximal end of the support tube 122 is coupled to a housing 124 of the motor 40 (as shown in FIG. 2A). At least a portion of the support tube 122 can be external to the internal cavity 104 (as shown in FIGS. 3A and 3B). At least a portion of the support tube 122 can be inserted into the internal cavity 104 (as shown in FIGS. 3A and 3B). The support tube 122 can be inserted through a trocar (e.g., along the longitudinal axis of the trocar). The support tube 122 can be configured to provide mechanical and structural support to the trocar and the positioning element and wrist element. The support tube can be configured to provide power or electrical control signals (e.g., via electrical cables) to the positioning element, wrist element, camera for performing ablation, and / or the like, including combinations and / or multiples thereof. According to one or more embodiments described herein, the support tube can be configured to provide a separate coaxial cable or the like for video captured by the camera.

[0045] 4B is a side view of the robotic arm assembly 42. According to some embodiments, the robotic arm assembly 42 includes a shoulder joint 126 that forms a virtual shoulder, an elbow joint 128 having a position sensor 132 (e.g., a capacitive proximity sensor) that forms a virtual elbow, a wrist joint 130 that forms a virtual wrist, and an end effector 45. In some embodiments, the shoulder joint 126, elbow joint 128, and wrist joint 130 can include a series of hinge and revolute joints to provide seven positionable degrees of freedom for each arm, as well as one additional grasping degree of freedom for the end effector 45. In some embodiments, the surgical robotic system 10 has nine degrees of freedom overall.

[0046] FIG. 5 illustrates a perspective front view of a portion of a robotic assembly 20 configured for insertion into a patient's internal body cavity. The robotic assembly 20 includes a first robotic arm 42A and a second robotic arm 42B. The two robotic arms 42A and 42B, in some embodiments, can define, or at least partially define, a virtual chest 140 of the robotic assembly 20. In some embodiments, the virtual chest 140 (depicted as a dotted triangle) can be defined by a chest plane extending between a first pivot point 142A of the most proximal joint of the first robotic arm 42A (e.g., shoulder joint 126), a second pivot point 142B of the most proximal joint of the second robotic arm 42B, and a camera imaging center point 144 of the camera 47. A pivot center 146 of the virtual chest 140 is at the center of the virtual chest 140.

[0047] In some embodiments, sensors within one or both of the first robotic arm 42A and the second robotic arm 42B can be used by the surgical robotic system 10 to determine a change in location in three-dimensional space of at least a portion of each or both of the robotic arms 42A and 42B. In some embodiments, sensors within one or both of the first robotic arm 42A and the second robotic arm 42B can be used by the surgical robotic system 10 to determine a location in three-dimensional space of at least a portion of one robotic arm relative to a location in three-dimensional space of at least a portion of the other robotic arm.

[0048] In some embodiments, the camera assembly 44 is configured to acquire images that enable the surgical robotic system 10 to determine its relative position in three-dimensional space. For example, the camera assembly 44 can include multiple cameras, at least two of which are laterally offset from one another relative to an imaging axis, and the system can be configured to determine distances to features within an internal body cavity. Further disclosure regarding surgical robotic systems including camera assemblies and associated systems for determining distances to features can be found in International Patent Application Publication No. 2021 / 159409, entitled "System and Method for Determining Depth Perception In Vivo in a Surgical Robotic System," published August 12, 2021, and incorporated herein by reference in its entirety. Information about the distances to features and information about the optical properties of the cameras can be used by the system to determine relative locations in three-dimensional space.

[0049] According to some embodiments, Figure 6A depicts a left hand controller 201, and Figure 6B depicts a right hand controller 202. The left hand controller 201 and the right hand controller 202 each include a contoured housing 210, 211, respectively. Each contoured housing 210, 211 includes a top surface 212a, 213a, an inner side surface 212b, 213b adjacent the top surface, an outer side surface (not visible in these views) facing away from the inner side surfaces 212b, 213b, and a bottom surface (not visible in these views) facing away from the top surfaces 212a, 213a.

[0050] In some embodiments, each hand controller 201, 202 includes a mounting assembly 215, 216, respectively. The mounting assemblies 215, 216 can be used to directly or indirectly mount the respective hand controller 201, 202 to a user console of a surgical robotic system. In some embodiments, the mounting assembly 215 defines a hole 217, which can be a countersink, configured to receive a screw or bolt to connect the left hand controller 201 to the user console.

[0051] In some embodiments, such as those depicted in FIGS. 6A and 6B , the hand controller includes two control levers, three buttons, and one touch input device. As described herein, embodiments may feature other combinations of touch input devices, buttons, and levers, or subsets thereof. The embodiment shown as the left hand controller 201 features a first control lever 221 and a second control lever 222. Similarly, the right hand controller 202 includes a first control lever 223 and a second control lever 224. In some embodiments, the first control lever 221 is engaged with the second control lever 222 via one or more gears (not shown) such that a user's depression of the first control lever 221 causes a reciprocating motion in the second paddle 222, and vice versa. In other embodiments, the first control lever 221 and the second control lever 222 can be configured to operate independently. In embodiments employing reciprocating motion of first and second control levers, the hand controller may employ only one signal indicative of the deflection of the first and second levers. In embodiments in which the first and second control levers operate independently, the hand controller may employ a first signal indicative of the deflection of the first control lever and a second signal indicative of the deflection of the second control lever.

[0052] In some embodiments, the first control lever 221, 223 and the second control lever 222, 224 can be contoured to receive a user's thumb and / or finger. In some embodiments, the first control lever 221, 223 extends from or beyond an outer side of the respective contoured housing 210, 211, and the second control lever 222, 224 extends from or beyond an inner side 212b, 212c of the respective contoured housing. Deflection or depression of the first control lever 221, 223 and the second control lever 222, 224 for each hand controller 210, 211 is configured to generate a signal that the surgical robotic system uses as an input to control a tool or instrument tip at the distal end of a robotic arm of the surgical robotic system (e.g., to open / close a grasper / jaw opening at the instrument tip). For example, depressing the first and second control levers can change the angle of the jaws of a grasper at the distal end of each robotic arm. In some embodiments, the end effectors, tools, or instruments are used to retract tissue, drive needle drivers, grasp items (e.g., mesh, sutures, needles), or pick up such items within a body cavity when the items are dropped, and deliver energy (e.g., to cut or coagulate) via an electrosurgical unit (ESU).

[0053] In some embodiments, the housing of the hand controller can be contoured. For example, in FIGS. 6A and 6B , the contoured housing 210, 211 includes a rounded shape. In some embodiments, the housing can be shaped to have a contour that matches the contour of at least a portion of the thumb of a user's hand. In some embodiments, the contoured housing 210, 211, the first control lever 221, 223, and the second control lever 222, 224 can each be shaped to comfortably and ergonomically receive a respective user's hand. In some embodiments, the hand controller housing, the hand controller levers, the hand controller buttons, and / or one or more touch input devices can have shapes and / or positions on the hand controller to accommodate different palm sizes and finger lengths.

[0054] The left hand controller 201 also includes a first button 231, a second button 232, and a third button 233. Similarly, the right hand controller 202 also includes a first button 234, a second button 235, and a third button 236. As taught herein, each button can provide one or more inputs that can be mapped to various different functions of a surgical robotic device to control a surgical robotic system, including a camera assembly and a robotic arm assembly. In embodiments, inputs received via the first button 231 of the left hand controller 201 and the first button 234 of the right hand controller 202 can control a clutch feature. For example, engaging the first buttons 231, 234 activates a clutch, allowing an operator to move the left hand controller 201 or right hand controller 202, respectively, without causing any movement of the robotic arm assemblies (e.g., the first robotic arm, the second robotic arm, and the camera assembly) of the surgical robotic system. When the clutch is activated for a hand controller, movement of the respective right or left hand controller is not translated into movement of the robot assembly. In some embodiments, the clutch is activated by the operator engaging the hand controller input (e.g., tapping or pressing a button), and the clutch is disengaged or the clutch mode is exited by the operator re-engaging (e.g., tapping or pressing a button again). In some embodiments, the clutch is activated when the operator engages the hand controller input (e.g., tapping or pressing a button and holding the button), the clutch remains active as long as the input is active, and the clutch is disengaged when the operator no longer engages the hand controller input (e.g., releases the button).Activating the clutch or entering the clutch mode of the hand controllers allows the operator to reposition each hand controller (e.g., repositioning the left hand controller 201 within its range of motion and / or repositioning the right hand controller 202 within its range of motion) without causing movement of the robotic arm assembly itself.

[0055] A second button 232 on the left hand controller 201 can provide input to control a pivot function of the surgical robotic device. An operator who engages (e.g., presses and holds) the second button 232 on the left hand controller 201 can engage a pivot function or pivot mode that reorients the chest of the robotic arm assembly to position the camera at a midpoint between the instrument tips. The pivot function can be activated with a short tap or held down to continuously track the instrument tip as it moves, according to some embodiments.

[0056] The second button 235 on the right hand controller 202 can provide inputs for entering and exiting a menu mode, in which a menu is displayed on the display 12 of the surgical robotic system 10. The operator can activate the menu mode by pressing the second button 235 once and disengage the menu function by pressing the second button 235 again. When the menu mode is engaged, the operator can navigate the menu using the left and / or right hand controllers to select options within the menu. For example, the first touch input device 242 on the right hand controller 202 can be used to navigate the menu and select menu items in some embodiments. During the menu mode, robotic movement can be suspended in response to movement of the left or right hand controller 201 or 202. In some embodiments, the third button 233 on the left hand controller and the third button on the right hand controller can provide inputs for engaging or disengaging an instrument control mode of the surgical robotic system. Movement of at least one of the one or more hand controllers while in the instrument mode causes a corresponding movement in a corresponding robotic arm of the robotic assembly.

[0057] The left-hand controller 201 further includes a touch input device 241. Similarly, the right-hand controller 202 further includes a touch input device 242. In embodiments, the touch input devices 241, 242 may be scroll wheels, as shown in Figures 6A and 6B. Other touch input devices that may be employed include, but are not limited to, rocker buttons, joysticks, pointing sticks, touchpads, trackballs, trackpoint nubs, etc.

[0058] The touch input devices 241, 242 can receive input through several different forms of engagement by an operator. For example, if the touch input devices 241, 242 are scroll wheels, the operator can press or click the first touch input device 241, 242, scroll the first touch input device 241, 242 backward or forward, or both.

[0059] In some embodiments, scrolling the first touch input device 241 of the left hand controller 241 forward can activate a zoom-in function to enlarge the field of view provided by the camera assembly of the surgical robotic system and displayed to the operator, and scrolling the first touch input device 241 backward can activate a zoom-out function to reduce the field of view provided by the camera assembly of the surgical robotic device and displayed to the operator, or vice versa. In embodiments, the zoom function can be mechanical or digital. In some embodiments, the zoom function can be partly mechanical and partly digital (e.g., mechanical zoom for one zoom range and mechanical zoom plus digital zoom for another zoom range).

[0060] In some embodiments, clicking or pressing the first touch input device 241 can engage a scan mode of the surgical robotic system. When in scan mode, movement of at least one of the left hand controller 201 or the right hand controller 202 causes a corresponding change in the orientation of the camera assembly of the robotic assembly without changing the position or orientation of any of the robotic arms of the surgical robotic system. In another embodiment, pressing and holding the first touch input device 241 can activate the scan mode, and releasing the first touch input device 241 can exit the scan mode of the surgical robotic system. In some embodiments, exiting scan mode returns the camera to the orientation it had when it entered scan mode. In some embodiments, a feature can be provided to lock the orientation when exiting scan mode (e.g., to change the "horizon" line).

[0061] In some embodiments, when the left elbow menu item is selected in menu mode, the first touch input device 241 of the left hand controller 201 can be used to select the direction and degree of left elbow bias. As used herein, elbow bias refers to the degree to which the virtual elbow of the robot arm is above or below the neutral or default position.

[0062] In some embodiments, when in menu mode, an operator can select options within a menu by navigating the menu using the left-hand controller and / or the right-hand controller. For example, when in menu mode, the touch input device 242 (e.g., a scroll wheel) on the right-hand controller provides a series of inputs for traversing the displayed menu and selecting items within the displayed menu. For example, by scrolling forward on the touch input device 242, the operator can move up the menu, and by scrolling backward on the touch input device 242, the user can move down the menu, or vice versa. In embodiments, by clicking the first touch input device 242, the operator can make a selection within the menu.

[0063] In some embodiments, the touch input device 242 of the right hand controller 202 can be used to control the right elbow bias when the right elbow bias menu item is selected.

[0064] In some embodiments, the various button and touch input device functions described above with respect to the left hand controller may instead be assigned to the right hand controller, and vice versa.

[0065] 6A also shows a schematic diagram of a foot pedal array 19 having a first foot pedal 251 and a second foot pedal 252 for receiving operator input. As shown in FIG. 6A, in some embodiments, the first foot pedal 251 engages a camera control mode, also described herein as a view control mode, an image framing control mode, or a camera framing control mode, of the surgical robotic system, and the second foot pedal 252 engages a navigation control mode of the surgical robotic system.

[0066] In some embodiments, when the camera control mode is activated, for example using the foot pedal 251, movement of the left hand controller 201 and / or right hand controller 202 by the operator can provide input that is interpreted by the system to control the movement and orientation of the camera assembly of the surgical robotic system while keeping the position of the instrument tip of the robotic arm of the robotic arm assembly constant.

[0067] In some embodiments, when the cruise-control mode is activated, for example, using the foot pedal 252, the left hand controller 201 and the right hand controller 202 can be used to move the robotic arm assemblies of the surgical robotic system in a manner such that the distal tips of the robotic arms direct or guide the movement of the chest of the robotic arm assemblies through an internal body cavity. In the cruise-control mode, the position and orientation of the camera assembly, the chest, or both are automatically adjusted to maintain the field of view of the camera assembly aimed at the tips (e.g., a point between the tip(s) at the distal end of a first robotic arm and the tip(s) at the distal end of a second robotic arm). This can be described as the camera assembly being pinned to the chest of the robotic arm assemblies and automatically following the tips. Further details regarding the cruise-control mode are provided below.

[0068] 6C and 6D depict another embodiment according to the present disclosure, featuring a left hand controller 1001 and a right hand controller 1002. The left hand controller 1001 includes a contoured housing 1010, and the right hand controller 1002 includes a contoured housing 1011. Each contoured housing 1010, 1011 includes a top surface 1012a, 1013a, an inner side surface 1012b, 1013b adjacent the top surface, an outer side surface (not visible in these views) facing away from the inner side surfaces 1012b, 1013b, and a bottom surface (not visible in these views) facing away from the top surfaces 1012a, 1013a.

[0069] Each hand controller 1001, 1002 includes a mounting assembly 1015, 1016, respectively. The mounting assemblies 1015, 1016 can be used to directly or indirectly mount each of the hand controllers 1001, 1002 to a surgeon's console of a surgical robotic system. The mounting assembly 1015 includes an opening 1017, and the mounting assembly 1016 defines an opening 1018. The openings 1017, 1018 can be countersunk openings configured to receive screws or bolts for connecting the respective hand controllers 1001, 1002 to the surgeon's console. The mounting assembly 1015 includes a button 1004, and the mounting assembly 1016 includes a button 1005. The buttons 1004, 1005 provide inputs for toggling between insertion and removal of one or more of the robotic arms 42A, 42B and the camera assembly 44. For example, button 1004 can be used to insert or remove the first robotic arm 42A, and button 1005 can be used to insert or remove the second robotic arm 42B. In some embodiments, buttons 1004, 1005 do not actually control the insertion or removal of the camera assembly 44, but allow the operator to enter an insertion or removal mode. The actual process of the camera assembly 44 can be controlled by other user elements.

[0070] Each of the left hand controller 1001 and the right hand controller 1002 also includes a first button 1031, 1034, a second button 1032, 1035, and a touch input device 1041, 1042 (e.g., a joystick or scroll wheel), respectively. In each hand controller 1001, 1002, the first button 1021, 1034, the second button 1032, 1035, and the touch input device 1041, 1042 are disposed on or on the top surface 1012a, 1013a of the housing 1010, 1011. In some embodiments, the first button 1021, 1034, the second button 1032, 1035, and the touch input device 1041, 1042 are disposed on or on a portion of the top surface 1012a, 1013a that protrudes from the top surface. For each hand controller 1001, 1002, a lever (not visible in this view) extends from its respective outer side (not visible in this view). In some embodiments, a different mechanism may be used for grip input on the hand controller. For example, in some embodiments, instead of or in addition to a single or multiple levers, the hand controller may include at least one "pistol trigger" type button that can be pulled to close and released to open.

[0071] The left hand controller 1001 includes a first paddle 1021 and a second paddle 1022. Similarly, the right hand controller 1002 includes a first paddle 1023 and a second paddle 1024. In some embodiments, the first paddle 1021, 1023 is engaged with the second paddle 1022, 1024 of each hand controller 1001, 1002 via one or more gears (not shown) such that a user's depression of the first paddle 1021, 1023 causes a reciprocating motion in the second paddle 1022, 1024, and vice versa. In another embodiment, the first paddle 1021, 1023 and the second paddle 1022, 1024 of each hand controller may be configured to operate independently. In embodiments employing reciprocating motion of the first and second paddles, the hand controllers 1001, 1002 may employ some form of signal or other indicator to indicate the deflection of the first paddles 1021, 1023 and the second paddles 1022, 1024. In embodiments in which the first and second paddles operate independently, the hand controllers 1001, 1002 may employ a first signal or other indicator to indicate the deflection of the first paddles 1021, 1023 and a second signal or other indicator to indicate the deflection of the second paddles 1022, 1024.

[0072] In some embodiments, the first paddle 1021, 1023 and the second paddle 1022, 1024 may be contoured to receive a user's thumb and / or finger. In some embodiments, the first paddle 1021, 1023 extends from or beyond an outer side of the respective contoured housing 1010, 1011, and the second paddle 1022, 1024 extends from or beyond an inner side 1012b, 1013b of the respective contoured housing. Deflection or depression of the first paddle 1021, 1023 and the second paddle 1022, 1024 for each hand controller 1010, 1011 is configured to trigger a signal that the surgical robotic system uses as an input to control a tool or instrument tip (e.g., open / close a grasper / jaw opening at the instrument tip) at the distal end of the robotic arm of the surgical robotic system. For example, depressing the first paddle 1021, 1023 and the second paddle 1022, 1024 may change the angle of the jaws of the graspers at the distal end of the respective robotic arms. In some embodiments, the end effectors, tools, or instruments are used to retract tissue, drive needle drivers, grasp items (e.g., mesh, sutures, needles), or pick up such items within a body cavity when the items are dropped, and deliver energy (e.g., to cut or coagulate) via an electrosurgical unit (ESU).

[0073] In some embodiments, each of the first paddles 1021, 1023 and second paddles 1022, 1024 may have a loop for receiving a user's thumb and / or finger, as further described with respect to Figures 6E and 6F. In some embodiments, parameters (e.g., length, angle, finger ergonomics, and the like) of each of the first paddles 1021, 1023 and second paddles 1022, 1024 may be adjusted.

[0074] The contoured housings 1010, 1011 can be configured to comfortably and ergonomically fit with the corresponding hands of an operator. An operator can engage each hand controller 1001, 1002 by placing the thumb of each hand on the second paddle 1022, 1024, positioning the index or middle finger of each hand on or above the protruding portion of the top surface 1013 a, 1013 a on which the first button 1021, 1034, second button 1032, 1035, and touch input device 1041, 1042 are disposed, and by positioning at least the middle or ring finger of each hand on or above the first paddle 1021, 1024.

[0075] While various exemplary embodiments described herein assign certain functions to certain buttons and to certain touch input devices, one of ordinary skill in the art in light of this disclosure will understand that which functions are assigned to which buttons and touch input devices may vary in different embodiments. Furthermore, one of ordinary skill in the art in light of this disclosure will understand that additional functions not explicitly described herein may be assigned to some buttons and some touch input devices in some embodiments. In some embodiments, one or more functions may be assigned to a foot pedal of a surgical robotic system that includes one or more hand controllers described herein.

[0076] As an example, a set of functions that may be controlled by the left hand controller 1001 and the right hand controller 1002 for some embodiments of the present technology will now be described.

[0077] For the left hand controller 1001, pressing or pressing and holding the first button 1004 may trigger a signal used to engage the insertion or removal of the left robotic arm assembly and / or camera assembly of the surgical robotic system. Pressing or pressing and holding the first button 1031 may trigger a signal used to control a clutch function of the left hand controller of the surgical robotic system. Pressing or pressing and holding the second button 1032 may trigger a signal to engage or disengage a camera control mode of the surgical robotic system. Scrolling forward on the touch input device 1041 may activate a zoom-in function to enlarge the view provided by the camera assembly of the surgical robotic system and displayed to the operator, and scrolling backward on the first touch input device 1041 may provide a zoom-out function to reduce the view provided by the camera assembly of the surgical robotic device and displayed to the operator, or vice versa. Scrolling the touch input device 1041 may trigger a signal used to select left elbow bias when the elbow hinge function is activated using a menu.

[0078] For the right hand controller 1002, pressing or pressing and holding the first button 1005 may trigger a signal used to engage the insertion or removal of the right robotic arm assembly and / or camera assembly of the surgical robotic system. Pressing or pressing and holding the first button 1034 may trigger a signal used to control a clutch function of the right hand controller of the surgical robotic system. Clicking or pressing the second button 1035 may engage a scan mode of the surgical robotic system. When in scan mode, movement of at least one of the left hand controller 1001 or the right hand controller 1002 causes a corresponding change in the orientation of the camera assembly of the robotic assembly without changing the position or orientation of any of the robotic arms of the surgical robotic system. In another embodiment, pressing and holding the second button 1035 may activate the scan mode, and releasing the second button 1035 may exit the scan mode of the surgical robotic system. In some embodiments, disabling scan mode returns the camera to the orientation it had when scan mode was entered. In some embodiments, a feature may be provided for locking the orientation (e.g., for changing the "horizontal" line) when exiting scan mode. Scrolling the touch input device 1042 may trigger signals used to traverse a menu or highlight a portion of a menu when the menu is displayed or when menu mode is active. Pressing the touch input device 1042 may trigger signals used to select a highlighted portion or function on a menu when the menu is displayed. Scrolling the touch input device 1042 may generate a signal used to select right elbow bias when the elbow bias function is activated using the menu. Scrolling forward on the touch input device 1042 may move up the menu, scrolling backward on the touch input device 1042 may move down the menu, or vice versa. Clicking the first touch input device 1042 may make a selection within a menu.

[0079] 6E and 6F depict another embodiment according to the present disclosure, featuring a left hand controller 1001′ and a right hand controller 1002′. Compared to the hand controllers 1001, 1002 of FIGS. 6C and 6D, some buttons on the hand controllers 1001′, 1002′ are the same button type but have different functions. For example, the second button 1035′ on the right hand controller 1002′ may trigger a signal used to turn a menu on or off. Compared to the hand controllers 1001, 1002 of FIGS. 6C and 6D, some buttons on the hand controllers 1001′, 1002′ may have different button types and / or different functions. For example, the touch input device 1041′ on the left hand controller 1001′ may have a three-way switch button type. Toggling or holding the touch input device 1041′ in the center may trigger a signal used to engage or disengage a scan mode of the surgical robotic system. Toggling the touch input device 1041′ forward may activate a zoom-in function to enlarge the view provided by the camera assembly of the surgical robotic system and displayed to the operator, and toggling the first touch input device 1041′ backward may activate a zoom-out function to reduce the view provided by the camera assembly of the surgical robotic device and displayed to the operator, or vice versa. Toggling the touch input device 1035′ upward may trigger a signal used to traverse a menu when a menu is displayed or a menu mode is active. The touch input device 1042′ of the right hand controller 1002′ may have a three-way switch button type. Toggling the touch input device 1042′ by pressing the touch input device 1035′ may trigger a signal used to traverse a menu or highlight a portion of a menu when a menu is displayed or a menu mode is active. Toggling the touch input device 1042′ forward may move the menu up, and toggling the touch input device 1042′ backward may move the menu down, or vice versa.Clicking on the first touch input device 1042' may trigger a signal used to select a highlighted portion or function on a menu when the menu is displayed. In some embodiments, toggling the touch input device 1042' may trigger a signal used to select a right elbow bias when the elbow bias function is activated using a menu. Compared to the hand controllers 1001, 1002 of FIGS. 6C and 6D , the hand controllers 1001', 1002' may have first paddles 1021', 1023' and second paddles 1022', 1024' coupled to finger loops 1061, 1062, 1063, 1064, respectively. Each finger loop may be a Velcro type. In some embodiments (not illustrated), each finger loop may be a hook type. Deflection or depression of the first paddle 1021′, 1023′ and the second paddle 1022′, 1024′ is configured to trigger a signal to control a tool or instrument tip (e.g., open / close a grasper / jaw opening at the instrument tip) at the distal end of a robotic arm of the surgical robotic system. For example, depressing the first paddle 1021′, 1023′ and the second paddle 1022′, 1024′ can change the angle of the jaws of a grasper at the distal end of the respective robotic arm. In some embodiments, the end effector, tool, or instrument is used to separate tissue, drive a needle driver, grasp an item (e.g., mesh, suture, needle), or pick up such an item within a body cavity when the item is dropped, and deliver energy (e.g., to cut or coagulate) via an electrosurgical unit (ESU). 6C and 6D, the first buttons 1031′, 1034′ may have a slider button type, and sliding the first buttons 1031′, 1034′ may trigger a signal used to control the clutch function of the corresponding hand controller of the surgical robot system.

[0080] 7 illustrates a graphical user interface 39 that is formatted to include a left pillarbox 198 and a right pillarbox 199 to the left and right, respectively, of a live video feed 168 of a patient's cavity. The graphical user interface 39 may be overlaid on the live video feed 168. In some embodiments, the live video feed 168 is formatted by the controller 26 to accommodate the left pillarbox 198 and the right pillarbox 199. In some embodiments, the live video feed 168 may be displayed on the display 12 at a predetermined size and location on the display 12, and the left pillarbox 198 and right pillarbox 199 may be displayed on either side of the live video feed 168 at a particular size based on the remaining area on the display 12 that is not occupied by the live video feed 168. The graphical user interface 39 includes several different graphical user interface elements, which are described in more detail below.

[0081] The robotic arms 42B and 42A are also visible in the live video feed. The left pillarbox 198 may include a state identifier 173, for example, an engaged or disengaged state identifier associated with the instrument tip 120 of the robotic arm 42B. The “engaged” state identifier 173 indicates that the user's left hand and arm are engaged with the left hand controller 201, and therefore the instrument tip 120 is also engaged. The “disengaged” state identifier 173 indicates that the user's left hand and arm are not engaged with the hand controller 201, and therefore the instrument tip 120 is also disengaged. When the user's left hand and arm are disengaged from the left hand controller 201, the surgical robotic system 10 can be completely disengaged; that is, the surgical robotic system 10 may remain on but is unresponsive until the user's hand re-engages the hand controller. The instrument tip 120 may be represented by a graphical symbol 179 containing the name of the instrument tip 120, allowing the user to see what type of end effector or instrument tip is currently in use. 7, the instrument tip 120, represented by iconographic symbol 179, is a bipolar grasper. Notably, the present disclosure is not limited to the bipolar graspers or scissors shown in FIG.

[0082] Similarly, the right pillar box 199 may include a state identifier 175 associated with the instrument tip 120 of the robotic arm 42A, such as an engaged or disengaged state. In some embodiments, based on the state of the end effector, the graphical user interface may also provide a visual representation of the state in addition to text. For example, the end effector iconography may be "grayed out" or less prominent when it is disengaged.

[0083] The state identifier 175 may be "engaged," indicating that the user's right hand and arm are engaged with the right hand controller 202, and therefore the instrument tip 120 is engaged. Alternatively, the state identifier 175 may be "disengaged," indicating that the user's right hand and arm are not engaged with the right hand controller 202, and therefore the instrument tip 120 is disengaged. The instrument tip 120 may be represented by a graphical symbol 176 that includes the name of the instrument tip 120, allowing the user to identify what type of end effector or instrument tip is currently in use. In FIG. 7 , the instrument tip 120 represented by the graphical symbol 176 is a monopolar scissors. Notably, the present disclosure is not limited to the monopolar scissors shown in FIG. 7 .

[0084] The left pillarbox 198 may also include a robot pose view 171. The robot pose view 171 includes a simulated view of the robot arms 42B and 42A, the camera assembly 44, and the support arm, thereby allowing the user to obtain a third-person view of the robot arm assembly 42, the camera assembly 44, and the robot support system 46. A simulated view of the robot arms 42B and 42A is represented by a pair of simulated robot arms 191 and 192. A simulated view of the camera assembly 44 is represented by a simulated camera 193. The robot pose view 171 also includes a simulated camera view associated with the patient cavity or portion of the cavity, which represents the placement or location of the pair of robot arms 151 and 172 relative to the frustum 151. More specifically, the camera view may be the field of view of the camera assembly 44, which is equivalent to the frustum 151.

[0085] The right pillarbox 199 may also include a robot pose view 172 that includes a simulated view of the robot arms 42B and 42A, the camera assembly 44, and the support arm, thereby allowing the user to obtain a third-person view of the robot arm assembly 42, the camera assembly 44, and the support arm. The simulated view of the robot arms 42B and 42A is a pair of simulated robot arms 165 and 166. The simulated view of the camera assembly 44 is represented by a simulated camera 193. The robot pose view 172 also includes a simulated camera view associated with the patient cavity or portion of the cavity, which is the placement or location of the pair of robot arms 165 and 166 relative to a frustum 167. More specifically, the camera view may be the field of view of the camera, which is the frustum 167. The robot pose view 172 provides elbow height awareness and situational awareness, especially when maneuvering in a face-up / face-down configuration.

[0086] Situational awareness can be characterized as a way of understanding certain robotic elements with respect to time and space when the robotic arms 42A and 42B are inside the patient cavity. For example, as shown in robot pose view 171, the elbow of simulated robotic arm 192 is bent downward, thereby providing the user with the ability to know how the elbow of the actual robotic arm 42A is actually oriented and positioned within the patient cavity. Note that due to the positioning of camera assembly 44 relative to robotic arms 42A and 42B, the full length of robotic arms 42A and 42B may not be visible in live video footage 168. As a result, the user may not be able to visualize how the robotic arms 42A and 42B are oriented and positioned within the patient cavity. Simulated robotic arms 165 and 166 and simulated robotic arms 191 and 192 provide the user with situational awareness of at least the position and orientation of the actual robotic arms 42A and 42B within the patient cavity.

[0087] There may be two separate views (robot pose view 171 and robot pose view 172) from two different perspectives on each side of the graphical user interface 39 rendered on the display 12. The robot pose views 171 and 172 are automatically updated to stay centered on the trocar 50 while keeping the robot arms 42A and 42B in view. The robot pose views 171 and 172 also provide spatial awareness to the user.

[0088] Spatial awareness may be characterized as the placement or location of robot arms 42A and 42B when viewed in robot pose views 171 and 172 relative to other objects within the cavity and the cavity itself. Robot pose views 171 and 172 provide the user with the ability to determine where the actual robot arms 42A and 42B are located within the cavity by viewing simulated robot arms 191 and 192 in robot pose view 171 and simulated robot arms 165 and 166 in robot pose view 172. For example, robot pose view 171 illustrates the position and location of simulated robot arms 191 and 192 relative to frustum 151. Robot pose view 171 depicts simulated robot arms 191 and 192 relative to frustum 151 from a side view of the support arm and the simulated robot arms 191 and 192 attached to the support arm. This particular robot pose provides the user with the ability to better verify access to anatomical features within the cavity.

[0089] The robot pose view 172 can also provide the user with the ability to better see how close the actual robot arms 42A and 42B are to each other or how far apart they are from each other. Furthermore, the robot pose view 172 can also illustrate where the actual robot arms 42A and 42B may be positioned or located relative to the patient's cavity to the left and right of the robot arms 42A and 42B, thereby providing the user with spatial awareness of where the robot arms 42A and 42B are within the cavity and where they are relative to anatomical features within the cavity. As mentioned above, because the full length of the robot arms 42A and 42B is not visible in the live video feed 168, the simulated robot arms 165 and 166 can provide the user with spatial awareness of how close or far apart the actual robot arms 42A and 42B are to each other. The view provided by the robot pose view 172 is as if the user were looking into the interior of the cavity. The robot pose view 172 provides the user with spatial awareness to know how close the virtual elbows 128 are to each other and how close the actual robot arms 42A and 42B are when the user manipulates the right hand controller 202 and the left hand controller to bring the virtual elbows 128 closer together. For example, when the user manipulates the left hand controller 201 and the right hand controller 202 to straighten the robot arms 42A and 42B, the simulated robot arms 166 and 165 become parallel to each other and the distance between the elbows of the simulated robot arms 165 and 166 decreases. Conversely, when the user manipulates the left hand controller 201 and the right hand controller 202 to bend the robot arms 42A and 42B and move the virtual elbows 128 of the robot arms 42A and 42B further apart, the simulated robot arms 166 and 165 become less parallel to each other and the distance between the elbows of the simulated robot arms 165 and 166 increases. The robot pose views 171 and 172 provide the user with spatial awareness during the surgical procedure, as the live video feed 168 does not provide visualization of the entire length of the robotic arms 42A and 42B.

[0090] 7, simulated robotic arms 191 and 192 are shown within the field of view of camera assembly 14 associated with frustum 151, which provides the user with situational and spatial awareness of where robotic arms 42B and 42A are located or positioned within a captured portion of the patient's actual cavity. The camera view associated with robot pose view 171 is a simulated view of robotic arms 42B and 42A as if the user were viewing an actual view of robotic arms 42B and 42A from a side view within the patient's cavity. As noted above, the camera view can be the field of view of camera assembly 44, which is frustum 167. That is, robot pose view 171 provides the user with a side view of simulated robotic arms 191 and 192, which are simulated views corresponding to robotic arms 42B and 42A, respectively.

[0091] In some embodiments, the graphical user interface 39 can display live video feed 168 from a single viewpoint that includes the field of view of the cavity and the robotic arms 42B and 42A for different areas within the cavity as shown in Figure 7. As a result, the user may not always be able to determine how the virtual elbows 128 of the robotic arms 42B and 42A are positioned because the camera assembly 44 may not always include video feed of the virtual elbow 128 of the robotic arm 42B and video feed of the elbows of the robotic arms 42A, and therefore the user may not be able to determine how to adjust the right hand controller 202 and the left hand controller 201 if they wish to operate within the patient's cavity. Because the left situation awareness camera view panel includes a simulated view of the entire length of the robot arms 191 and 192, the simulated view of the robot arm 42B (robot arm 191) and the simulated view of the robot arm 42A (robot arm 192) provide a user with a perspective that allows the user to determine the positioning of the virtual elbows 128 of the robot arms 42A and 42B. Because the simulated view of the robot pose view 171 includes views of the virtual elbows 128 of the robot arms 191 and 192, the user can adjust the positioning of the robot arms 42B and 42A by manipulating the left hand controller 201 and the right hand controller 202 and carefully watching how the robot arms 191 and 192 move in accordance with the manipulation of the left hand controller 201 and the right hand controller 202.

[0092] The graphical user interface 39 can include a robot pose view 172 with a frustum 167, which is the field of view of the camera assembly 44 associated with a portion of the patient's cavity, and robot arms 165 and 166 with a simulated camera 158 and a simulated robot support arm supporting the robot arms 165 and 166.

[0093] 7 , simulated robotic arms 165 and 166 are shown within frustum 167, which represents the location and orientation of robotic arms 42B and 42A within the patient's actual cavity. The view shown in robot pose view 172 is a simulated view of robotic arms 42B and 42A as if the user were looking at robotic arms 42B and 42A from a top-down view within the patient's cavity. That is, robot pose view 172 provides the user with a top-down view of simulated robotic arms 165 and 166, which are simulated views corresponding to robotic arms 42B and 42A, respectively. The top-down view provides the user with the ability to maintain a certain level of situational awareness of robotic arms 42B and 42A as the user performs a procedure within the cavity. The view of simulated robot arm 165 corresponding to robot arm 42B and the view of simulated robot arm 166 corresponding to robot arm 42A provide a top-view perspective that allows the user to determine the positioning of robot arm 42B and robot arm 42A, because robot pose view 172 includes a simulated top-down view of robot arms 165 and 166, camera 158, and the support arms of the robot assembly. The simulated field of view of camera assembly 44, represented schematically by frustum 167, includes a top-down view of simulated robot arms 165 and 166, so that the user can adjust the positioning of robot arm 42B and robot arm 42A by manipulating left hand controller 201 and right hand controller 202 and noting how simulated robot arms 165 and 166 move forward or backward within the portion of the cavity within frustum 167 in accordance with the manipulation of left hand controller 201 and right hand controller 202.

[0094] The simulated views of the robot arms 42B and 42A in the robot pose views 171 and 172 are automatically updated to keep the robot arms 42B and 42A in view while remaining centered over the trocar 50. In some embodiments, this can be achieved based on one or more sensors from the sensing and tracking module 16 on the robot arms 42B and 42A, which provide information to the right hand controller 202 and the left hand controller 201. The sensors can be encoders or Hall effect sensors or other suitable sensors.

[0095] Inserting the Robot Assembly The articulating robotic arm insertion taught herein can be employed with any of the surgical robotic systems taught above, or any other suitable surgical robotic system. Additionally, some embodiments taught herein can be employed with semi-robotic endoscopic surgical systems that are only partially robotic.

[0096] Articulated robotic arm insertion (also referred to as the "primary methodology") can be understood with reference to the embodiments depicted in Figures 8-15 described below. For convenience, unless otherwise noted, like reference numerals will be used to refer to similar features of the various embodiments shown in the drawings.

[0097] Figure 8 is a flowchart illustrating steps 300 for inserting a robotic assembly performed by the surgical robotic system 10 of the present disclosure. Figure 8 is a flowchart based on a robotic arm having three articulation joints, a wrist joint, an elbow joint, and a shoulder joint. Nevertheless, the process associated with the flowchart of Figure 8 is applicable to robotic arms having four or more articulation joints.

[0098] In step 302, the surgical robotic system 10 enters an insertion mode, allowing a user to insert the camera assembly 44 and robot arm assembly 42 of the robot assembly 20 through the trocar 50 into the internal cavity of the subject. In some embodiments, the user can control the foot pedal array 19 to enter the insertion mode. In some embodiments, the user can control one of the hand controllers 201 / 202 to enter a menu mode, and the surgical robotic system 10 can display a menu on the display 12. The user can control the appropriate hand controller 201 / 202 to select the insertion mode on the menu. In some embodiments, the surgical robotic system 10 determines that the instrument tip 120 is properly positioned (e.g., via data obtained from sensors associated with the instrument tip 120 or via user input), and then the surgical robotic system 10 automatically enters the insertion mode. It should be understood that the user can control one or both of the hand controllers or one or both of the foot pedals to enter the insertion mode.

[0099] In step 304, the user via the surgical robotic system 10 inserts the camera assembly 44 through the trocar 50. An example is described with respect to Figures 9A-9D.

[0100] In step 306, the user, via the surgical robotic system 10, controls the camera assembly 44 to reach a desired camera view. For example, after the camera assembly 44 reaches a camera insertion point within an internal cavity, as described with respect to FIGS. 9A-9D , the surgical robotic system 10 can automatically deploy the camera assembly 44 to face forward within the internal cavity, such as the abdominal space (e.g., camera deployed state). If the current camera view is not sufficient, the surgical robotic system 10 allows the user to adjust the position and orientation of the camera assembly 44 via pitch and yaw control of the camera support arm to reach the desired camera view.

[0101] In step 308, the surgical robotic system 10 determines that the first articulation joint 130 of the plurality of articulation joints has exited the trocar 50 and reached a first articulation joint insertion position within the internal cavity. The first articulation joint insertion position indicates that the first articulation joint has passed through the second end 53 of the trocar 50 and is free to rotate relative thereto. Figures 11A-11D and the associated text provide more details regarding the articulation robotic arm insertion of the first articulation joint 130 as taught herein.

[0102] In step 310, the surgical robotic system 10 enables articulation of the first articulation joint 130 within the first volume 414 within the internal cavity such that the first articulation joint 130 can be articulated within the first volume 414 by one or both of the hand controllers 201 / 202. In some embodiments, the surgical robotic system 10 automatically enables articulation of the first articulation joint 130, for example, after reaching a minimum insertion point. In some embodiments, a user of the surgical robotic system 10 enables articulation of the first articulation joint 130 via the hand controllers 201 / 202 or foot pedals 19, or graphical user interface 39, after reaching a minimum insertion point. Figures 11D and 14 and their associated text provide more details regarding articulation of the first articulation joint 130 as taught herein.

[0103] In step 312, the surgical robotic system 10 determines that a second articulation joint 128 of the plurality of articulation joints has exited the trocar 50 and passed through the second end 53 of the trocar 50 within the internal cavity to a second articulation joint insertion position. The second articulation joint insertion position indicates that the second articulation joint 128 is free to rotate relative to the second end 53 of the trocar 50. Figures 11E and 11F and the associated text provide more details regarding articulating robotic arm insertion of the second articulation joint 128 as taught herein.

[0104] In step 314, the surgical robotic system 10 enables articulation of the second articulation joint 128 within the second volume 424 within the internal cavity such that the second articulation joint 128 can be articulated by one or both of the hand controllers 201 / 202 within the second volume 424. In some embodiments, the surgical robotic system 10 automatically enables articulation of the second articulation joint 128, for example, after reaching a minimum insertion point. In some embodiments, a user of the surgical robotic system 10 enables articulation of the second articulation joint 128 via the hand controllers 201 / 202 or foot pedals 19, or graphical user interface 39, after reaching a minimum insertion point. Figures 11F and 14 and their associated text provide more details regarding articulation of the second articulation joint 128 as taught herein.

[0105] In step 316, the surgical robotic system 10 determines that the third articulation joint 126 of the plurality of articulation joints has exited the trocar 50 and passed through the second end 53 of the trocar 50 within the internal cavity to reach a third articulation joint insertion position. The third articulation joint insertion position indicates that the third articulation joint 126 is free to rotate relative to the trocar 50. Figures 11G and 14 and their associated text provide more details regarding articulating robotic arm insertion of the third articulation joint 126 as taught herein.

[0106] In step 318, the surgical robotic system 10 enables articulation of the third articulation joint 128 within the internal cavity such that the third articulation joint 126 can be articulated by one or both of the hand controllers 201 / 202 within the internal cavity. In some embodiments, the surgical robotic system 10 automatically enables articulation of the third articulation joint 126, for example, after reaching a minimum insertion point. In some embodiments, a user of the surgical robotic system 10 enables articulation of the third articulation joint 126 via the hand controllers 201 / 202 or foot pedals 19, or graphical user interface 39, after reaching a minimum insertion point.

[0107] In step 320, the surgical robotic system 10 determines that the first robotic arm 42A is fully inserted into the internal cavity. Figures 11G and 14 and the associated text provide more details regarding articulated robotic arm insertion as taught herein.

[0108] In step 322, the surgical robotic system 10, either automatically or via input from a user, one of the hand controllers 201 / 202, foot pedals 19, or graphical user interface 39, enables full articulation of the first robotic arm 42A. Examples are described with respect to Figures 11G and 14, and their associated text provides details regarding articulated robotic arm insertion as taught herein.

[0109] In step 324, if the surgical robotic system 10 includes a second robotic arm 42B and the user desires to use the second robotic arm 42B, the user via the surgical robotic system 10 inserts the second robotic arm 42B of the robotic arm assembly 42 into the internal cavity through the trocar 50; otherwise, the surgical robotic system 10 can proceed to step 326. Articulated robotic arm insertion of the second robotic arm 42B can repeat steps 308-322. Examples are described with respect to Figures 12-14 and their associated text, which provide more details regarding articulated robotic arm insertion of the second robotic arm 42B as taught herein.

[0110] In step 326, the surgical robotic system 10 determines that the insertion process is complete. For example, if the surgical robotic system 10 can determine that both the camera assembly and the robotic arm assembly are fully inserted into the desired location within the cavity (e.g., an articulation joint insertion position, a target position, or the like), the surgical robotic system 10 can determine that the insertion process is complete. The surgical robotic system 10 can also allow the user to exit the insertion mode using an on-screen menu, a foot pedal, and / or a hand controller. One skilled in the art will understand that some or all of the above steps may be repeated depending on the number of articulation joints included in the robotic arm.

[0111] If at any point during the process of performing steps 302-324 it is determined that the articulation of the first and / or second robotic arms is not sufficient to avoid obstacles and / or sensitive tissue, the operator may take further steps to reposition the first and / or second volumes within the internal cavity, as described below in connection with Figures 16-18.

[0112] 9A-9D illustrate the insertion of the camera assembly 44 through the trocar 50, according to some embodiments. As shown in FIGS. 9A-9C, the camera assembly 44 is inserted into the trocar 50 at the first end 52 of the trocar 50 (e.g., using one or both of the hand controllers 201 / 202 and / or the foot pedal 19). For example, a user can control one or both of the hand controllers 201 / 202 to select an insertion mode on a displayed menu and use the hand controllers 201 / 202 to control the position and orientation of the camera assembly 44 within the internal cavity. In some embodiments, the user can use the foot pedal 19 to select the insertion mode such that the hand controllers 201 / 202 can control the position and orientation of the camera assembly 44 within the trocar 50 or the internal cavity. As shown in FIG. 9D, the camera assembly 44 exits the second end 53 of the trocar 50 and is positioned at a camera insertion position P camera The camera insertion position P camera is the trocar position P trocarThe minimum insertion position may be a position that allows a distance 314 between the camera assembly 44 (e.g., the center or proximal portion of the trocar 50) and the position of the camera assembly 44 that meets a distance threshold (e.g., greater than or equal to the distance threshold). The distance threshold indicates that the camera assembly 44 has passed the second end 53 of the trocar 50 to reach the minimum distance, allowing the camera assembly 44 to freely rotate relative to the trocar 50. When the camera assembly 44 has successfully moved to the minimum insertion position, the camera assembly 44 may automatically deploy to face forward into an internal cavity (e.g., camera deployed state), such as the abdominal cavity. If the camera view is not satisfactory after automatic deployment to face forward, the user can control the foot pedal 19 or other control to enter a camera view mode 251, allowing the user to use the hand controller 201 / 202 to control the position and orientation of the camera assembly 44, as well as the pitch and yaw of the camera assembly 44 to reach a desired camera view. Once the camera is deployed, a scan mode may be automatically or manually initiated to capture an overview of the internal cavity. This allows the user to determine whether any features within the cavity are obstructing access to the surgical site and whether such features need to be avoided or cut before fully inserting one or both of the robotic arms.

[0113] To facilitate explanation of the articulated robotic arm insertion process taught herein, FIG. 4B has been simplified and redrawn to help illustrate the hand length, forearm length, and upper arm length of the robotic arm of the robotic arm assembly 42. As shown in FIG. 10 , the articulated joints along the robotic arm of the robotic arm assembly 42 are spaced as follows: The robotic arm assembly 42 includes an end effector 45 having a shoulder joint 126, an elbow joint 128, a wrist joint 130, and an instrument tip 120. In some embodiments, the shoulder joint 126, the elbow joint 128, and the wrist joint 130 may include a series of hinge and revolute joints to provide seven positionable degrees of freedom for each robotic arm 42, as well as one additional grasping degree of freedom for the end effector 45. The hand length 402 extends from the instrument tip 120 to the wrist joint 130. The forearm length 404 extends from the wrist joint 130 to the elbow joint 128. The upper arm length 406 extends from the elbow joint 128 to the shoulder joint 126 .

[0114] 11A-11G illustrate articulated robotic arm insertion of a first robotic arm 42A through a trocar 50, according to some embodiments. As shown in FIGS. 11A and 11B, a camera assembly 44 is inserted into the trocar 50 and positioned at a camera insertion position P cameraAfter reaching the trocar 50 (shown in FIG. 9D ), if the view of the camera assembly 44 indicates that the articulated robotic arm insertion process can continue, the first robotic arm 42A is inserted into the trocar 50 (e.g., using the hand controllers 201 / 202 and / or foot pedal 19). In some embodiments, the user can be alerted when it is possible to begin inserting the robotic arm 42. For example, the display 12 or the graphical user interface 39 can output an alert (e.g., a visual or audio output) indicating that the user can begin inserting the robotic arm 42. As another example, the surgical robotic system 10 can alert the user using sound and / or haptic effects. The user can select one of the robotic arms (e.g., the first robotic arm 42A) and begin inserting the first robotic arm 42A using the hand controllers 201 / 202 to control the position and orientation of the first robotic arm 42A through the trocar 50 and into the internal cavity. In some embodiments (not shown), the user can insert the first robotic arm 42A before the camera assembly 44 is inserted into the internal cavity. For example, a user can control one or both of the hand controllers 201 / 202 to select an insertion mode on a displayed menu and control one or both of the hand controllers 201 / 202 to control the position and orientation of the first robotic arm 42A into the internal cavity through the trocar 50. In some embodiments, a user can control the foot pedal 19 to enter the insertion mode.

[0115] As shown in FIG. 11C, wrist joint 130A exits trocar 50 and is inserted into the internal cavity at wrist joint insertion position P wrist-joint Wrist joint insertion position P wrist-joint indicates that the wrist joint 130A has fully passed through the second end 53 of the trocar 50 and is free to rotate relative to the second end 53 of the trocar 50. Wrist joint insertion position P wrist-joint is the trocar position P trocarand the position of wrist joint 130A satisfies a wrist joint motion distance threshold (e.g., greater than or equal to the wrist joint motion distance threshold), indicating that wrist joint 130A has passed second end 53 of trocar 50 to reach a minimum distance that allows wrist joint 130A to freely rotate relative to trocar 50.

[0116] As shown in FIG. 11D, the wrist joint 130A is inserted at the wrist joint insertion position P wrist-joint Upon reaching wrist joint insertion position P, articulation of wrist joint 130A is enabled automatically or via an action taken by a user, for example, using one of the hand controllers (e.g., hand controller 201 or 202 shown in FIG. 6B). Articulation of wrist joint 130A at this point in the articulated robot arm insertion process is limited to a first volume 414A within the internal cavity. Wrist joint 130A is articulatable within first volume 414A by one or both of hand controllers 202 / 202 (e.g., via rotation translation, pitch, yaw, roll, or other suitable motion that changes the position and / or orientation of wrist joint 130A). First volume 414A is located at wrist joint insertion position P. wrist-jointWhen inserted into or passed through the first volume 414A, the first volume 414A defines a volume accessible by wrist articulation. The first volume 414A is determined by the hand length 402 and the forearm length 404 (as shown in FIG. 10 ). For example, the first volume 414A may be a cylinder with a radius of the hand length 402 and a height of the forearm length 404, e.g., Pi * hand length * hand length * forearm length. A user can use the wrist joint 130A and end effector 45A to manipulate tissue 416 within the first volume 414A to clear a way for further arm insertion and / or to avoid an obstacle (e.g., an abdominal obstacle). In a similar manner, a user can articulate the wrist joint 130A to move the end effector 45A past an obstacle (e.g., an abdominal obstacle) within the first volume 414A. In some embodiments, once the wrist joint 130A and end effector 45A are articulated, the wrist joint 130A may not translate further except for insertion, but the end effector 45A may be moved to match the orientation of the hand controller 201 / 202, which is controlled open or closed by the user.

[0117] As shown in FIG. 11E, the user can decide to continue inserting the robotic arm 42A, in which case the elbow joint 128A exits the second end 53 of the trocar 50 and reaches the elbow joint insertion position P within the internal cavity. elbow-joint Elbow joint insertion position P elbow-joint indicates that elbow joint 128A has fully passed through second end 53 of trocar 50 and can freely rotate relative to trocar 50. Elbow joint insertion position P elbow-joint is the trocar position P trocarand a position of elbow joint 128A that satisfies an elbow joint distance threshold (e.g., equal to or greater than an elbow joint motion distance threshold). The elbow joint motion distance threshold indicates that elbow joint 128A has passed second end 53 of trocar 50 to reach a minimum distance that allows elbow joint 128A to freely rotate relative to trocar 50. Once the minimum distance is reached, articulation of elbow joint 128A may be enabled automatically by robotic surgical system 10 or may be enabled by a user of robotic surgical system 10.

[0118] 11F, articulation of elbow joint 128A occurs within a second volume 424A within the internal cavity such that elbow joint 128A can be articulated by one or both of hand controllers 201 / 202 (e.g., via rotation translation, pitch, yaw, roll, or other suitable motion that changes the position and / or orientation of elbow joint 128A) within second volume 424A. Second volume 424A is defined by elbow joint insertion position P elbow-jointInserted to or beyond defines a volume accessible for wrist articulation and elbow articulation. Second volume 424A is defined by hand length 402, forearm length 404, and upper arm length 406 (as shown in FIG. 10 ). For example, second volume 424A can be a combination of a cylinder with radius equal to the sum of hand length 402 and forearm length 404 (referred to as the “elbow reach”) and the height of upper arm length 402, and a hemisphere extending beyond the cylinder. The hemisphere can have elbow reach as its radius. Second volume 424A can be calculated by (Pi * elbow reach * elbow reach * upper arm length) + (Pi * 2 / 3 * elbow reach * elbow reach * elbow reach). A user can use one or both of the hand controllers 201 / 202 to control the elbow joint 128A, the wrist joint 130A, and the end effector 45A to manipulate tissue 416 within the second volume 424A to clear a way for further arm insertion and / or to avoid an obstacle 416 (e.g., an abdominal obstacle). In a similar manner, a user can articulate the elbow joint 128A and / or the wrist joint 130A to steer the end effector 45A past an obstacle (e.g., an abdominal obstacle) within the second volume 424A. In some embodiments, the wrist joint 130A can translate laterally further along the insertion portion as far as the forearm length 404. Articulation of the elbow joint 128A and the wrist joint 130A can be used to position the end effector 45A (e.g., the position and orientation of the end effector 45A) in accordance with movement of one or both of the hand controllers 201 / 202 held by the user, including, for example, opening and closing the end effector 45A. In some embodiments, inserting the robotic arm 42A through the wrist joint 130A and up to the shoulder joint 126A may be sufficient for the user to perform a desired task.

[0119] As shown in FIG. 11G, the user may decide to continue inserting the robotic arm 42A, in which case the shoulder joint 126A will exit the second end 53 of the trocar 50 and be positioned at a shoulder joint insertion position P within the internal cavity. shoulder-jointShoulder joint insertion position P shoulder-joint is the trocar position P trocar and a position of the shoulder joint 126A that satisfies a shoulder joint distance threshold (e.g., greater than or equal to a shoulder joint motion distance threshold). The shoulder joint motion distance threshold indicates that the shoulder joint 126A has passed the second end 53 of the trocar 50 to reach a minimum distance that allows the shoulder joint 126A to freely rotate relative to the trocar 50.

[0120] When the shoulder joint 126A rotates freely relative to the trocar 50, the surgical robotic system 10 can determine that the first robotic arm 42A is fully inserted within the internal cavity and can determine that the first robotic arm 42A is capable of full articulation. For example, full articulation of the first robotic arm 42A, including the shoulder joint 126A, elbow joint 128A, wrist joint 130A, end effector 45A, and / or other components of the first robotic arm 42A, is enabled via one or both of the hand controllers 201 / 202 within the internal cavity such that a user can use one or both of the hand controllers 201 / 202 to control the first robotic arm 42A to manipulate tissue 416 or avoid obstacles 416.

[0121] 12A-12G illustrate articulated robotic arm insertion of the second robotic arm 42B through the trocar 50, according to some embodiments. The articulated robotic arm insertion process of the second robotic arm 42B is similar to the articulated robotic arm insertion process of the first robotic arm 42A, as shown in FIGS. 11A-11G. After the first robotic arm 42A is inserted into the internal cavity, the user can begin articulated robotic arm insertion of the second robotic arm 42B. In some embodiments, the robotic arms 42 can be fully or partially inserted in any order or in a specific order, as described with respect to FIGS. 13A and 13B.

[0122] As shown in FIGS. 12A-12C, the wrist joint 130B of the second robotic arm 42B exits the trocar 50 and reaches a wrist joint insertion position P' of the second robotic arm 42B within the internal cavity. wrist-joint Wrist joint insertion position P' wrist-joint indicates that the wrist joint 130B has fully passed through the second end 53 of the trocar 50 and is free to rotate relative to the second end 53 of the trocar 50. Wrist joint insertion position P' wrist-joint can be determined based on a wrist joint movement distance threshold, as described above with respect to FIG. 12C. While the first robotic arm 42A is fully articulated, articulation of the wrist joint 130B can be enabled via one of both hand controllers 201 / 202 within a first volume 414B associated with the second robotic arm 42B. The first volume 414B can be calculated as described above with respect to FIG. 11D.

[0123] As shown in FIGS. 12D and 12E, the user can decide to continue inserting the robotic arm 42B, in which case the elbow joint 128B of the second arm 42B exits the second end 53 of the trocar 50 and reaches an elbow joint insertion position P' of the second robotic arm 42B within the internal cavity. elbow-joint The elbow joint insertion position P' is reached. elbow-joint indicates that elbow joint 128B has fully passed through second end 53 of trocar 50 and can freely rotate relative to trocar 50. Elbow joint insertion position P' elbow-joint can be determined based on an elbow joint movement distance threshold, as described above with respect to Figure 11F. Articulation of the elbow joint 128B can be enabled via one or both of the hand controllers 201 / 202 within a second volume 424B associated with the second robotic arm 42B. The second volume 424B can be calculated as described above with respect to Figure 11F.

[0124] As shown in FIG. 12F, the user may decide to continue inserting the robotic arm 42B, in which case the shoulder joint 126B will exit the second end 53 of the trocar 50 and reach a shoulder joint insertion position P' of the second robotic arm 42B within the internal cavity. shoulder-joint Shoulder joint insertion position P' shoulder-joint indicates that the shoulder joint 126B has fully passed through the second end 53 of the trocar 50 and is free to rotate relative to the second end 53 of the trocar 50. Shoulder joint insertion position P' shoulder-joint can be determined based on the shoulder joint motion distance threshold, as described above with respect to Figure 11G. When the shoulder joint 126B is free to rotate relative to the trocar 50, the second robotic arm 42B is fully inserted into the internal cavity.

[0125] In some embodiments, surgical robotic system 10 may include three or more robotic arms. Each of the remaining robotic arms may be inserted into the internal cavity using a similar insertion process as robotic arm 42. Surgical robotic system 10 may determine that the insertion process is complete when all of robotic arms 42 and / or camera assemblies 44 are fully inserted into the desired location within the internal cavity. Surgical robotic system 10 may operate display 12 to output one or more selectable menu items that allow the user to exit the insertion mode.

[0126] In some embodiments, the camera assembly 44 and the robot arm assembly 42, or the robot arms of the robot arm assembly 42, can be fully or partially inserted in any order or in a specific order. For example, the second robot arm 42B can be followed by the camera assembly 44, which can then be followed by the first robot arm 42A. A robot arm 42 can be inserted during the insertion process of the camera assembly 44 (e.g., the camera assembly 44 is partially inserted) or during the insertion process of another robot arm 42 (e.g., the robot arm 42 is partially inserted). Examples are shown in FIGS. 13A and 13B.

[0127] 13A and 13B illustrate articulated robotic arm insertion while another robotic arm is partially inserted. A first robotic arm 42A or a second robotic arm 42B can be partially inserted into the internal cavity while the other robotic arm is also partially inserted. As shown in FIG. 13A, a user can determine which robotic arm 42 to insert and which joints of the robotic arm 42 to articulate; in this case, elbow joint 128A, wrist joint 130A, and wrist joint 130B are inserted into the internal cavity and exit the second end 53 of the trocar 50 to an elbow joint insertion position P. elbow-joint (For example, as shown in Figure 11E), wrist joint insertion position P wrist-joint (e.g., as shown in FIG. 11C), and wrist joint insertion position P' wrist-joint (e.g., as shown in FIG. 12B ). Articulation of elbow joint 128A, wrist joint 130A, and wrist joint 130B occurs in an internal cavity such that these joints can be articulated by one or both of hand controllers 201 / 202 (e.g., via rotation translation, pitch, yaw, roll, or other suitable motion that changes the position and / or orientation of elbow joint 128A, wrist joint 130A, end effector 45A, wrist joint 130B, and end effector 45B). A user can use one or both of hand controllers 201 / 202 to control elbow joint 128A, wrist joint 130A, end effector 45A, wrist joint 130B, and / or end effector 45B to manipulate tissue 416 to clear a way for further arm insertion and / or avoid obstacles 416 (e.g., abdominal obstacles). In a similar manner, the user can articulate elbow joint 128A, wrist joint 130A, and / or wrist joint 130B to steer one or both of end effectors 45 through an obstacle (e.g., an abdominal obstacle).

[0128] As shown in FIG. 13B, the user can determine which robotic arm 42 to insert and which joint of the robotic arm 42 to articulate, in this case elbow joint 128A, wrist joint 130A, elbow joint 128B, and wrist joint 130B are inserted into the internal cavity and exit the second end 53 of the trocar 50 to an elbow joint insertion position P elbow-joint (For example, as shown in Figure 11E), wrist joint insertion position P wrist-joint (For example, as shown in Figure 11C), elbow joint insertion position P' elbow-joint (e.g., as shown in FIG. 12D), and wrist joint insertion position P' wrist-joint (e.g., as shown in FIG. 12B ). Articulation of elbow joint 128A, wrist joint 130A, elbow joint 128B, and wrist joint 130B occurs in an internal cavity such that these joints can be articulated by one or both of the hand controllers 201 / 202 (e.g., via rotation translation, pitch, yaw, roll, or other suitable motion that changes the position and / or orientation of elbow joint 128A, wrist joint 130A, end effector 45A, elbow joint 128B, wrist joint 130B, and end effector 45B). A user can use one or both of the hand controllers 201 / 202 to control elbow joint 128A, wrist joint 130A, end effector 45A, elbow joint 127B, wrist joint 130B, and / or end effector 45B to manipulate tissue 416 at different locations to clear a way for further arm insertion and / or avoid obstacles 416 (e.g., abdominal obstacles). In a similar manner, the user can articulate elbow joint 128A, wrist joint 130A, elbow joint 128B, and / or wrist joint 130B to steer one or both of the end effectors 45 through an obstacle (e.g., an abdominal obstacle).

[0129] In some embodiments, as described herein, at any time during the articulated robotic arm insertion process for each robotic arm 42, the camera assembly 44 can be moved and reoriented so that the user can view the insertion of the arm and adjust the insertion path as needed. For example, during the insertion of the first robotic arm 42A as described with respect to Figures 11A-11G, or the insertion of the second robotic arm 42B as described with respect to Figures 12A-12F, the user can control one or both of the foot pedal 19 or hand controller 201 / 202 to enter a camera control mode to change the position and / or orientation of the camera assembly 44 to obtain a desired camera view.

[0130] In some embodiments, as the robotic arm 42 advances through the trocar 50, a visual representation ("pose view") of the robotic arm 42 may indicate that the robotic arm joints can be articulated by the hand controllers 201 / 202 to change position and orientation. In some embodiments, the visual representation is color-coded to indicate which of the robotic arm joints are free to rotate within the internal cavity. The user can use the hand controllers 201 / 202 to move the robotic arm joints indicated by the visual representation as needed to avoid abdominal obstacles. An example is described below with reference to FIG. 14 .

[0131] 14 depicts the graphical user interface 39 when a robotic arm has been inserted into the internal cavity as taught herein. The graphical user interface 39 displays an image 168 (e.g., live video feed) captured by the camera assembly 44 along with a camera view. The image 168 includes the robot arm 42 having the end effector 45, the wrist joint 130, and the elbow joint 128 (partially captured by the image 168). The graphical user interface 39 also includes a robot pose view 172 having a simulated robot arm 165 corresponding to the robot arm 42 and a simulated camera 158 corresponding to the camera assembly 44. The simulated robot arm 165 includes a simulated shoulder joint 426 corresponding to the shoulder joint 126, a simulated elbow joint 428 corresponding to the elbow joint 128, a simulated wrist joint 430 corresponding to the wrist joint 130, and a simulated end effector 432 corresponding to the end effector 45. The robotic post view 172 is color-coded green to indicate that the robotic arm 42 and camera assembly 44 may be freely manipulated. The robotic arm 42 may be controlled to manipulate tissue 416 to clear the way for further arm insertion and / or to avoid an obstruction 416 (e.g., an incarcerated hernia or the like).

[0132] FIG. 15 schematically depicts the computing module 18 in more detail. The computing module 18 can be used to perform one or more steps of the methods provided by the exemplary embodiments. The computing module 18 includes one or more non-transitory computer-readable media for storing one or more computer-executable instructions or software for implementing the exemplary embodiments. The non-transitory computer-readable media may include, but are not limited to, one or more types of hardware memory, non-transitory tangible media (e.g., one or more magnetic storage disks, one or more optical disks, one or more USB flash drives), and the like. For example, the memory 1506 included in the computing module 18 can store computer-readable and computer-executable instructions or software for implementing the exemplary embodiments. The computing module 18 also includes a processor 22 and associated cores 1504 for executing the computer-readable and computer-executable instructions or software stored in the memory 1506, as well as other programs for controlling the system hardware. The processor 22 can be a single-core processor or a multi-core (1504) processor.

[0133] The memory 1506 may include computer system memory or random access memory, such as DRAM, SRAM, EDO RAM, and the like. The memory 1506 may also include other types of memory, or combinations thereof. A user may interact with the computing module 18 through a display 12, such as a touchscreen display or computer monitor capable of displaying a graphical user interface (GUI) 39. The display 12 may also display other aspects, transducers, and / or information or data associated with the exemplary embodiments. The computing module 18 may include other I / O devices for receiving input from a user, such as a keyboard or any suitable multi-point touch interface 1508, and a pointing device 1510 (e.g., a pen, stylus, mouse, or trackpad). The keyboard 1508 and pointing device 1510 may be coupled to the visual display device 12. The computing module 18 may include other appropriate conventional I / O peripherals.

[0134] The computing module 18 may also include one or more storage devices 24, such as a hard drive, CD-ROM, or other computer-readable medium, for storing data and computer-readable instructions, applications, and / or software that implement the exemplary operations / steps of the surgical robotic system 10 taught herein, or portions thereof, which may be executed to generate the graphical user interface 39 on the display 12. The exemplary storage device 24 may also store one or more databases for storing any appropriate information needed to implement the exemplary embodiments. The databases may be updated by a user to add, delete, or update one or more items in the databases, or automatically at any appropriate time. The exemplary storage device 24 may store one or more databases 1526 for storing provisioned data and other data / information used to implement the exemplary embodiments of the systems and methods taught herein.

[0135] Computing module 18 may include a network interface 1512 configured to interface with one or more networks, e.g., a LAN, a WAN, or the Internet, through various connections, including, but not limited to, a standard telephone line, a local area network (LAN) or wide area network (WAN) link (e.g., 802.11, T1, T3, 56kb, X.25), a broadband connection (e.g., ISDN, Frame Relay, ATM), a wireless connection, a controller area network (CAN), or a combination of any or all of the above, via one or more network devices 1520. Network interface 1512 may include an internal network adapter, a network interface card, a PCMCIA network card, a card bus network adapter, a wireless network adapter, a USB network adapter, a modem, or any other device suitable for interfacing computing module 18 to any type of network with which it is capable of communicating and performing the operations taught herein. Furthermore, computing module 18 may be any computer system, such as a workstation, desktop computer, server, laptop, handheld computer, tablet computer (e.g., an iPad® tablet computer), mobile computing or communication device (e.g., an iPhone® communication device), or other form of computing or communication device capable of communication and having sufficient processor power and memory capacity to perform the operations taught herein.

[0136] Computing module 18 may run any operating system 1516, such as any version of the Microsoft® Windows® operating system, different releases of Unix and Linux® operating systems, any version of MacOS® for Macintosh computers, any embedded operating system, any real-time operating system, any open source operating system, any proprietary operating system, any operating system for mobile computing devices, or any other operating system capable of running on a computing device and performing the operations described herein. In some embodiments, operating system 1516 may run in native mode or in an emulated mode. In some embodiments, operating system 1516 may run on one or more cloud machine instances.

[0137] The computing module 18 may also include an antenna 1530 that may transmit wireless transmissions to and receive wireless transmissions from a radio frequency (RF) front end.

[0138] Secondary techniques related to repositioning the working volume by altering the pitch and / or yaw of the RSS 46 will now be described. It should be understood that altering the position of the RSS 46 during a secondary technique may affect the insertion of the articulated robotic arm (i.e., the primary technique) described above. FIG. 16 illustrates one embodiment of the RSS 46 with axis and translational positioning elements about a cavity, such as the abdomen of a subject (e.g., a patient) supported by a surgical table. The RSS 46 can facilitate positioning and insertion of a robotic arm as described herein into a body cavity (e.g., the abdomen) of a subject (e.g., a patient) at a trocar pivot point during laparoscopic surgery. The RSS 46 can include a motor unit (e.g., motor 40) configured to control movement of a support tube (e.g., along the RSS insertion axis 1602 and / or the RSS roll axis 1604, for example) to insert the robotic arm into the patient's cavity during laparoscopic surgery. The insertion may be guided by an insertion rail, the movement of which may be controlled by the RSS 46 (e.g., along the RSS roll axis 1604, along the RSS pitch axis 1606, and / or along the RSS yaw axis 1608). According to one or more embodiments described herein, the movement controlled by the RSS 46 may be on any combination of the RSS roll axis 1604, pitch axis 1606, and / or RSS yaw axis 1608. For example, in some cases, the RSS 46 provides control along the pitch axis 1606 and / or the RSS yaw axis 1608 without providing control along the RSS roll axis 1604. Other combinations of roll / pitch / yaw control are possible in other embodiments.

[0139] The roll degree of freedom may be rotation of the locator element about the longitudinal axis of the trocar (or another parallel axis) via the support tube 122. This may allow the orientation of the locator element to be adjusted for the operator's desired comfort.

[0140] The yaw degree of freedom may be rotation of the locator element via the support tube 122 about an axis perpendicular to the longitudinal axis of the trocar, typically perpendicular to the ground, which may cause the locator element to move left or right (relative to the trocar) and slightly adjust its orientation.

[0141] The pitch degree of freedom may be rotation of the locator element via the support tube 122 about an axis perpendicular to the longitudinal axis of the trocar, typically parallel to the ground. This may move the locator element up or down (relative to the trocar) and slightly adjust its orientation. As in both manual and robotic laparoscopic surgery, both the yaw and pitch degrees of freedom may involve rotation of the trocar relative to the patient, which may result in temporary stretching of the patient's abdominal wall and surrounding tissue.

[0142] 17A-17D illustrate one embodiment of various axes of the RSS 46, including a side view of the insertion axis (FIG. 17A), a side view of the roll axis (FIG. 17B), a side view of the pitch axis (FIG. 17C), and a top view of the yaw axis (FIG. 17D). FIG. 17A provides a side view of the insertion axis (e.g., RSS insertion axis 1602), including an internal insertion axis 1602a (inside the subject's body cavity) and an external insertion axis 1602b (outside the subject's body cavity). FIG. 17B provides a side view of the roll axis (e.g., RSS roll axis 1604), including an internal roll axis 1604a (inside the subject's body cavity) and an external roll axis 1604b (outside the subject's body cavity). FIG. 17C provides a side view of the pitch axis (e.g., RSS pitch axis 1606), including an internal pitch axis 1606a (inside the subject's body cavity) and an external pitch axis 1606b (outside the subject's body cavity). FIG. 17D provides a top view of the yaw axes (eg, RSS yaw axis 1608), including an internal yaw axis 1608a (inside the subject's body cavity) and an external yaw axis 1608b (outside the subject's body cavity).

[0143] The RSS 46 can change the positioning and orientation of the robot arm by making adjustments to one or more of the RSS roll axis 1604, the RSS pitch axis 1606, and / or the RSS yaw axis 1608. When the RSS roll axis 1604 and / or the RSS pitch axis 1606 are adjusted, the working volumes (e.g., the first volume 414 (see FIGS. 11D and 12C) and / or the second volume 424 (see FIGS. 11F and 12E)) move relative to the subject. More specifically, during an insertion process as described herein, it may be desirable to change the orientation of the initial RSS insertion axis 46 to reposition the first volume 414 (see FIGS. 11D and 12C) and / or the second volume 424 (see FIGS. 11F and 12E) within the subject's internal cavity 104. The first volume 414 and / or the second volume 424 can be repositioned within the internal cavity 104 of the subject 100 relative to the initial RSS insertion axis 1602. For example, if sensitive tissue or an obstacle (e.g., an organ, a tumor, scar tissue, a foreign body, another surgical tool, and / or the like including a combination and / or plurality thereof) is encountered within the internal cavity 104 during an insertion process as taught herein, the working volume of one or more of the robotic arms or the camera can be repositioned, for example, by repositioning the first volume 414 and / or the second volume 424 via altering the RSS roll axis 1604, the RSS pitch axis 1606, and / or the RSS yaw axis 1608.

[0144] As an example, a pitch change along the RSS pitch axis 1606 and / or a yaw rotation along the RSS yaw axis 1608 may be implemented on the RSS 46. The pitch change and / or yaw rotation are implemented relative to an insertion point (e.g., at the point where the trocar 50 enters the subject 100). The trocar 50 at the insertion point acts as a pivot point for the arms of the robotic arm assembly 42 for pitch and yaw movement. As a result, pitch and yaw movement of the support tube 122 of the robotic arm assembly 42 outside the subject 100 is inversely realized within the internal cavity of the subject 100. More specifically, when the RSS 46 changes the initial RSS insertion axis 1602 by adjusting the external pitch axis 1606b and / or the external yaw axis 1608b, a corresponding inverse movement occurs within the cavity 104 of the subject 100 relative to the internal pitch axis 1606a and / or the internal yaw axis 1608a. For example, if the RSS 46 pitches up the RSS insertion axis 1602 outside of the target 100 by adjusting the external pitch axis 1606b, the camera assembly 44 and one or more arms of the robot arm assembly 42 will pitch down within the internal cavity 104 of the target 100 because the pitch adjustment is inversely achieved by the internal pitch axis 1606a. Other motions are similar; for example, if the RSS 46 yaws the RSS insertion axis 1602 to the right outside of the target 100 by adjusting the external yaw axis 1608b, the camera assembly 44 and one or more arms of the robot arm assembly 42 will yaw to the left within the internal cavity 104 of the target 100 because the yaw adjustment is inversely achieved by the internal yaw axis 1608b. Providing external motion (e.g., pitch rotation and / or yaw rotation) of the RSS 46 increases the working volume within the internal cavity 104 of the target 100 in which one or more of the camera assembly 44 and / or end effector 45 can be repositioned without additional insertion points. Relocation is further described with reference to FIG.

[0145] FIG. 18 depicts a flowchart illustrating steps 1800 for repositioning a portion of a robot assembly to avoid an obstacle, according to some embodiments.

[0146] In step 1802, the surgical robotic system 10 enters an insertion mode that allows a user to insert the camera assembly 44 and one or more robotic arms of the robot arm assembly 42 of the robot assembly 20 through the trocar 50 using the RSS 46 to position the camera assembly 44 and one or more robotic arms of the robot arm assembly 42 in the internal cavity 104 of the subject 100. In some embodiments, the user can control the foot pedal array 19 to enter the insertion mode. In some embodiments, the user can control one of the hand controllers 201 / 202 and / or 261 / 262 to enter a menu mode, and the surgical robotic system 10 can display a menu on the display 12. The user can control the appropriate hand controller 201 / 202 and / or 261 / 262 to select the insertion mode on the menu. In some embodiments, the surgical robotic system 10 determines that the instrument tip 120 is properly placed (e.g., via data obtained from a sensor associated with the instrument tip 120 or via user input), and then the surgical robotic system 10 automatically enters the insertion mode. It should be understood that the user can control one or both of the hand controllers or one or both of the foot pedals to enter the insertion mode.

[0147] In step 1804, the user, via the surgical robotic system 10, begins insertion of the camera assembly 44 and / or one or more robotic arms 42 of the robotic arm assembly 42 through the trocar 50. An example is described with respect to Figures 9A-9D. The RSS 46 performs insertion along the RSS insertion axis 1602, as shown in Figure 16.

[0148] In step 1806, a user via surgical robotic system 10 controls camera assembly 44 and / or one or more robotic arms 42 of robotic arm assembly 42 within an interior volume (e.g., first volume 414 and / or second volume 424) of internal cavity 104 of subject 100 by articulating one or more articulation joints as described herein. For example, one or more of steps 308-322 can be performed in step 1806.

[0149] In step 1808, the surgical robotic system 10 determines whether to reposition an internal volume (e.g., first volume 414 and / or second volume 424). For example, this determination may include determining whether sensitive tissue or obstacles (e.g., organs, tumors, scar tissue, foreign objects, another surgical tool, and / or the like including combinations and / or pluralities thereof) are encountered within the internal cavity 104 of the subject 100. For example, images captured by the camera assembly 44 may depict sensitive tissue or obstacles (e.g., organs, tumors, scar tissue, foreign objects, another surgical tool, and / or the like including combinations and / or pluralities thereof). The obstacles may be detected manually by the surgeon, automatically by the surgical robotic system 10 (e.g., using a machine learning model trained to detect obstacles using images), and / or the like including combinations and / or pluralities thereof.

[0150] If, in step 1810, it is determined to reposition the first or second volume (e.g., if an obstacle is detected and cannot be avoided solely by articulation of one or more robotic arms, if a herniation is encountered and can be accommodated, and / or the like, including a combination and / or plurality thereof), the first and / or second volume is repositioned by performing at least one of a pitch change of the RSS pitch axis 1606 of the RSS 46 and / or a yaw rotation of the RSS yaw axis of the RSS 46. By repositioning the first and / or second volumes, the obstacle can be avoided or the effect of the obstacle can be reduced. For example, as described herein, the first volume 414 and / or the second volume 424 can be repositioned relative to the internal cavity 104 of the subject 100. For example, when an obstacle is encountered within the internal cavity 104 (in step 1808), the first and / or second volumes can be repositioned by performing a pitch change and / or a yaw rotation of the RSS 46 described herein, which causes a repositioning of the first volume 414 and / or the second volume 424. As an example, the RSS can be used to perform a yaw rotation and / or a pitch change relative to the insertion axis, as described with reference to FIGS. 17C and 17D . The yaw rotation and / or pitch change are performed along the RSS yaw axis 1608 and / or the RSS pitch axis 1606, respectively, about the insertion point (e.g., at the point where the trocar 50 enters the object 100). Thus, the trocar 50 at the insertion point acts as a pivot for the RSS pitch axis 1606 and the RSS yaw axis 1608. As a result, the movement of the RSS pitch axis 1606 and the RSS yaw axis 1608 outside the object 100 is conversely realized within the internal cavity of the object 100. For example, when the RSS 46 causes the insertion axis 1602 to pitch up outside the target 100 along the external pitch axis 1606b, the camera assembly 44 pitches down within the internal cavity 104 of the target 100 along the internal pitch axis 1606a.Other motions are similar, for example, if the RSS 46 causes the insertion axis 1602 to yaw to the right outside the target 100 along the external yaw axis 1608b, the camera assembly 44 will yaw to the left within the target 100's internal cavity 104 along the internal yaw axis 1608a. Providing external motion (e.g., pitch and / or yaw rotation) of the RSS 46 provides a volume within the target 100's internal cavity 104 in which the camera assembly 44 and / or end effector 45 can be repositioned without additional insertion points.

[0151] In accordance with one or more embodiments described herein, the RSS 46 (or a portion thereof) may be repositioned to avoid obstacles before and / or after selectively activating one or more articulation joints for articulation as described herein, for example, to perform one or more of steps 308-326 of FIG. 8 described herein.

[0152] It should be understood that steps 1806, 1808, and 1810 can be performed repeatedly. For example, multiple obstacles may be encountered as various articulation joints are activated. For example, during step 1806, a first obstacle may be encountered in the first volume 414. Steps 1808 and 1810 may then be performed to reposition the first volume 414 to avoid the first obstacle. Additional articulation joints may be activated, and a second obstacle may be encountered in the second volume 424. In such a case, steps 1808 and 1810 may be performed again to reposition the second volume 424 to avoid the second obstacle.

Claims

1. 1. A surgical robotic system, comprising: a camera assembly; a robotic arm assembly having a first robotic arm and a second robotic arm, each of the first and second robotic arms having a plurality of articulated joints; a hand controller holdable by a user of the surgical robotic system to control the first and second robotic arms and the camera assembly; A trocar and a memory storing one or more instructions; a processor configured or programmed to read the one or more instructions stored in the memory, the processor: entering an insertion mode that allows the user to insert the camera assembly and robotic arm assembly through the trocar into an internal cavity of a subject; determining that the first robotic arm is inserted into the trocar; determining when a first articulation joint of the plurality of articulation joints of the first robotic arm exits the trocar and reaches a first articulation joint insertion position within the internal cavity, where the first articulation joint is free to rotate relative to the trocar; upon determining that the first articulation joint has reached the first articulation joint insertion position, a first one of the hand controllers allows the first articulation joint to articulate within a first volume; determining when a second articulation joint of the plurality of articulation joints of the first robotic arm exits the trocar and reaches a second articulation joint insertion position within the internal cavity, wherein the second articulation joint is free to rotate relative to the trocar; the first hand controller is operably coupled to the robotic arm assembly, the hand controller, and the camera assembly such that upon determining that the second articulation joint has reached the second articulation joint insertion position within the internal cavity, the first hand controller allows the second articulation joint to articulate within a second volume.

2. the processor: determining that the second robotic arm is inserted into the trocar; determining when a first articulation joint of the second robotic arm exits the trocar and reaches a second robotic arm first articulation joint insertion position within the internal cavity, indicating that the first articulation joint of the second robotic arm is free to rotate relative to the trocar; 10. The surgical robotic system of claim 1, wherein a second one of the hand controllers is further configured or programmed to read the one or more instructions stored in the memory to permit articulation of the first articulation joint of the second robotic arm within a third volume upon determining that the first articulation joint of the second robotic arm reaches the first articulation joint insertion position of the second robotic arm.

3. the processor:

3. The surgical robotic system of claim 2, further configured or programmed to read the one or more instructions stored in the memory to allow the user to insert the second robotic arm upon determining that the first robotic arm is fully inserted into the internal cavity.

4. the processor: determining that the second robotic arm is fully inserted into the internal cavity; 3. The surgical robotic system of claim 2, further configured or programmed to read the one or more instructions stored in the memory to determine that an insertion process is complete.

5. the surgical robotic system includes a display, and the processor:

5. The surgical robotic system of claim 4, further configured or programmed to read the one or more instructions stored in the memory that operate the display to output one or more selectable menu items that allow the user to exit the insertion mode.

6. the processor: determining that the first robotic arm is fully inserted into the internal cavity; 10. The surgical robotic system of claim 1, wherein the first hand controller is further configured or programmed to read the one or more instructions stored in the memory that allow the first robotic arm to fully articulate.

7. the processor: determining that the camera assembly is inserted through the trocar; determining that the camera assembly has exited the trocar and reached a camera insertion position within the internal cavity; the hand controller allows for control of the position and orientation of the camera assembly within the internal cavity; 10. The surgical robotic system of claim 1, further configured or programmed to read the one or more instructions stored in the memory that operate the display to output images captured by the camera assembly.

8. the processor:

10. The surgical robotic system of claim 1, further configured or programmed to read the one or more instructions stored in the memory to operate the display to output a visual representation indicating that the first articulation joint or the second articulation joint can be articulated by the first hand controller to change a position and orientation.

9. The surgical robotic system of claim 8 , wherein the visual representation is color-coded to indicate which of the plurality of articulation joints are free to rotate within the internal cavity.

10. 10. The surgical robotic system of claim 1, wherein the first robotic arm or the second robotic arm comprises a wrist hinge joint, an elbow hinge joint, a shoulder hinge joint, and an end effector.

11. 11. The surgical robotic system of claim 10, wherein the first volume is determined by a radius of a hand length and a height of a forearm length, the hand length being the length between the tip of the end effector and the wrist hinge joint, and the forearm length being the length between the wrist hinge joint and the elbow hinge joint.

12. 12. The surgical robotic system of claim 11, wherein the second volume is determined by a radius of the sum of the forearm length and the hand length and a height of an upper arm length, the upper arm length being the length between the elbow hinge joint and the shoulder hinge joint.

13. 1. A method for inserting a robotic assembly of a surgical robotic system through a trocar into an internal cavity of a subject, the robotic assembly including a robotic arm assembly, the method comprising: inserting a first robotic arm of the robotic arm assembly through the trocar, the first robotic arm having a plurality of articulation joints; determining when a first articulation joint of the plurality of articulation joints exits the trocar and reaches a first articulation joint insertion position within the internal cavity, indicating that the first articulation joint is free to rotate relative to the trocar; enabling, via a first hand controller of the surgical robotic system, articulation of the first articulation joint within a first volume within the internal cavity such that the first articulation joint is articulated within the first volume by the first hand controller; determining when a second articulation joint of the plurality of articulation joints exits the trocar and reaches a second articulation joint insertion position within the internal cavity, wherein the second articulation joint is free to rotate relative to the trocar; and and enabling, via the first hand controller, articulation of the second articulation joint within a second volume within the internal cavity such that the second articulation joint is articulated by the first hand controller within the second volume.

14. inserting a second robotic arm of the robotic arm assembly through the trocar; determining when a first articulation joint of the second robotic arm exits the trocar and reaches a second robotic arm first articulation joint insertion position within the internal cavity, indicating that the first articulation joint of the second robotic arm is free to rotate relative to the trocar; and 14. The method of claim 13, further comprising: enabling, via a second hand controller of the surgical robotic system, articulation of the first articulation joint of the second robotic arm within a first volume associated with the second robotic arm.

15. The method of claim 14 , wherein inserting the second robotic arm occurs after the first robotic arm is fully inserted into the internal cavity.

16. determining when a second articulation joint of the second robotic arm exits the trocar and reaches a second articulation joint insertion position of the second robotic arm within the internal cavity, the second articulation joint insertion position indicating that the second articulation joint of the second robotic arm is free to rotate relative to the trocar; enabling articulation of the second articulation joint of the second robotic arm within a second volume associated with the second robotic arm via the second hand controller; determining when a third articulation joint of the second robotic arm exits the trocar and reaches a third articulation joint insertion position of the second robotic arm within the internal cavity, the third articulation joint insertion position indicating that the third articulation joint of the second robotic arm is free to rotate relative to the trocar; determining that the second robotic arm is fully inserted into the internal cavity; determining that the insertion process is complete; and 15. The method of claim 14, further comprising: outputting on a display one or more selectable menu items that allow the user to exit the insert mode.

17. determining when a third articulation joint of the plurality of articulation joints exits the trocar and reaches a third articulation joint insertion position within the internal cavity, wherein the third articulation joint insertion position indicates that the third articulation joint is free to rotate relative to the trocar; determining that the first robotic arm is fully inserted into the internal cavity; 14. The method of claim 13, further comprising: enabling full articulation of the first robotic arm via the first hand controller.

18. inserting a camera assembly of the robotic assembly through the trocar; The method of claim 13 , further comprising controlling the camera assembly to achieve a desired camera view.

19. 14. The method of claim 13, further comprising outputting a visual representation of the first robotic arm on a display, the visual representation indicating that the first articulation joint or the second articulation joint can be articulated by the first hand controller to change position and orientation.

20. 20. The method of claim 19, wherein the visual representation is color-coded to represent which of the plurality of articulation joints are free to rotate within the internal cavity.

21. The method of claim 13 , wherein the first volume is less than the second volume.

22. 14. The method of claim 13, wherein the first articulation joint is articulated by the first hand controller to manipulate tissue within the first volume.

23. 14. The method of claim 13, wherein the first and second articulation joints are articulated by the first hand controller to manipulate tissue within the second volume.

24. 15. The method of claim 14, wherein the first and second articulation joints of the first robotic arm and the first articulation joint of the second robotic arm are articulated by the first and second hand controllers to manipulate tissue within the internal cavity.

25. detecting an obstruction within the internal cavity of the subject during the insertion of the first robotic arm of the robotic arm assembly through the trocar; The method of claim 13 , further comprising: repositioning at least a portion of the robot assembly to avoid the obstacle.

26. 26. The method of claim 25, wherein the repositioning comprises performing a yaw rotation of the robotic arm assembly, the yaw rotation being performed about an insertion axis.

27. 26. The method of claim 25, wherein the repositioning includes performing a pitch rotation of the robotic arm assembly, the pitch rotation being performed about an insertion axis.

28. 1. A method for repositioning at least a portion of a robotic assembly of a surgical robotic system to avoid an obstacle, comprising: entering an insertion mode of the surgical robotic system to allow a user to insert a camera assembly and a robotic arm assembly of the robotic assembly through a trocar into an internal cavity of a subject; beginning to insert the camera assembly and the robotic arm assembly through the trocar; controlling the camera assembly and the robotic arm assembly by articulating one or more articulation joints within an interior volume of the interior cavity of the object; determining whether to reposition the interior volume; in response to determining to reposition the interior volume, repositioning the interior volume by performing at least one of a pitch change of a pitch axis of the robot assembly and a yaw rotation of a foot axis of the robot assembly.

29. 29. The method of claim 28, wherein the yaw rotation is performed about an insertion axis.

30. 29. The method of claim 28, wherein the pitch rotation is performed relative to an insertion axis.

31. 30. The method of claim 28, wherein determining whether to reposition the internal volume comprises determining whether an obstacle is encountered within the internal cavity of the object, and wherein the repositioning is performed to avoid the obstacle.

32. 30. The method of claim 28, wherein determining whether an obstacle is encountered is based at least in part on images captured by the camera.

Images