Cooperative surgical system having a coupling mechanism removably attachable to a surgical instrument - Patent Application 20070122997

Abstract

Described herein is a collaborative robotic system that can be used to assist in laparoscopic surgical procedures. The collaborative robotic system allows surgeons to use commercially available surgical tools while providing the benefits associated with surgical robotics. Advantageously, surgical tools can be seamlessly coupled to the robotic arm using a disposable coupler, while the reusable portion of the robotic arm remains in a sterile drape. Furthermore, the collaborative robotic system can operate in multiple modes, improving usability and safety, while allowing the surgeon to position instruments directly with the instrument handle and then maintain the desired position of the instrument using the robotic arm.

Description

[Technical Field]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Patent Application No. 18 / 318,699, filed May 16, 2023, U.S. Patent Application No. 18 / 057,191, filed November 18, 2022, U.S. Provisional Patent Application No. 63 / 378,434, filed October 5, 2022, and European Patent Application No. 22306496.5, filed October 5, 2022, the entire contents of each of which are incorporated herein by reference. This application is also related to U.S. Patent Application No. 17 / 815,885, filed July 28, 2022, now U.S. Patent No. 11,504,197, which is a continuation of PCT Patent Application No. PCT / IB2022 / 052989, filed March 30, 2022, and claims priority to European Patent Application No. 21306904.0, filed December 22, 2021; European Patent Application No. 21306905.7, filed December 22, 2021; European Patent Application No. 21305929.8, filed July 5, 2021; and European Patent Application No. 21305417.4, filed March 31, 2021, the entire contents of each of which are incorporated herein by reference. This application is also related to U.S. Patent Application No. 18 / 480,360, filed October 3, 2023, U.S. Patent Application No. 18 / 331,060, filed June 7, 2023, and U.S. Patent Application No. 17 / 816,958, filed August 2, 2022, now U.S. Patent No. 11,622,826, the entire contents of each of which are incorporated herein by reference.

[0002] The present disclosure is directed to collaborative robotic systems, such as those having coupling mechanisms for removably attaching to surgical instruments. [Background technology]

[0003] Managing visibility and access during laparoscopic procedures is a challenge. Surgical assistant paradigms are inherently imperfect because the assistant is required to anticipate and understand the surgeon's perspective without standing where the surgeon stands, and similarly anticipate and adjust the degree to which the surgeon desires tissue of interest to be exposed throughout the procedure. For example, during a laparoscopic procedure, one assistant may be required to hold a retractor device and expose tissue for the surgeon, while another assistant may be required to hold a laparoscopic device and provide the surgeon with a view of the surgical space within the patient during the procedure, and one may be required to hold an individual tool in an impractical position, such as between the surgeon's arms, while the surgeon is actively operating additional surgical instruments.

[0004] Various attempts have been made at solving this problem. For example, rail-mounted orthopedic retractors, which are purely mechanical devices mounted on a patient bed / table, can be used to hold laparoscopic devices in place during laparoscopic procedures, and other rail-mounted orthopedic retractors can be used to hold retractor devices in place during laparoscopic procedures. However, rail-mounted orthopedic retractors require extensive manual interaction to unlock, reposition, and lock the tools in place.

[0005] Complex robotic-assisted systems, such as the Da Vinci Surgical System (marketed by Intuitive Surgical, Sunnyvale, California), are being used by surgeons to enhance laparoscopic surgical procedures by allowing them to remotely perform the procedure from a surgeon console remote from the patient console where the surgeon holds the surgical instruments. Such complex robotic-assisted systems are very expensive and have a very large footprint, occupying a lot of space in the operating room. Furthermore, such robotic-assisted systems typically require unique, system-specific surgical instruments to be compatible with the system, thus preventing surgeons from using the standard, off-the-shelf surgical instruments they are accustomed to. Therefore, surgeons are required to learn how to perform an entirely different laparoscopic procedure.

[0006] In light of the aforementioned shortcomings of previously known systems and methods, there is a need for a system that provides surgeons with the ability to seamlessly position and manipulate various surgical instruments as needed, thus avoiding the workflow limitations inherent in both human and mechanical solutions. Summary of the Invention [Means for solving the problem]

[0007] The present disclosure overcomes the shortcomings of previously known systems and methods by providing a co-manipulated surgical system for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween. The co-manipulated surgical system may include a robotic arm having a proximal end, a distal end configured to be removably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints between the proximal and distal ends, the distal end of the robotic arm including a coupler interface. The co-manipulated surgical system may further include a coupler body configured to be removably coupled to the coupler interface. The coupler body may include a lumen sized and shaped to receive the elongate shaft of a surgical instrument therein and may be configured to transition between an open state, in which the elongate shaft is slidably movable within the lumen, and a closed state, in which longitudinal movement of the elongate shaft relative to the coupler body is prevented while rotational movement of the elongate shaft relative to the coupler body is permitted in response to movement at the handle of the surgical instrument. When the coupler body is coupled to the coupler interface, the coupler body may be configured to rotate relative to the distal end of the robotic arm, via the coupler interface, to self-align the lumen with the elongate shaft as the elongate shaft is inserted into the lumen. Furthermore, when the coupler body is coupled to the coupler interface in the closed state, the robotic arm may be enabled to be freely movable in response to movement at the handle of the surgical instrument to perform a laparoscopic surgical procedure. The coupler body may be disposable after a single laparoscopic surgical procedure.

[0008] The cooperatively operated surgical system may further include a switch configured to transition between an unlocked position and a locked position. The switch may include an engagement portion configured to engage the elongate shaft when the elongate shaft is disposed within the lumen and the switch is in the locked position, thereby securing the elongate shaft within the lumen. Thus, when the coupler body is coupled to the coupler interface and the elongate shaft is disposed within the lumen, the robotic arm may be configured to be freely movable in response to movements at the handle of the surgical instrument. Furthermore, when the elongate shaft is disposed within the lumen and the switch is in the locked position, the engagement portion may apply a frictional force to the elongate shaft, the frictional force configured to allow rotational movement of the elongate shaft relative to the coupler body while preventing longitudinal movement of the elongate shaft relative to the coupler body. The switch may include a handle portion configured to be actuated to transition the switch between the unlocked position and the locked position.

[0009] Further, the coupler body may include a beveled surface having a first valley configured to engage the switch in the unlocked position, a second valley configured to engage the switch in the locked position, and a ridge between the first valley and the second valley. The ridge may be configured to allow the switch to transition between the unlocked position and the locked position when a force applied to the switch exceeds a predetermined force threshold. The coupler body may further include a holder slidably disposed within the coupler body, the holder including a contact surface configured to define at least a portion of the lumen. Additionally, the holder may be biased toward the lumen when the elongate shaft is disposed within the lumen such that the contact surface engages with the elongate shaft. In some embodiments, the coupler interface may include a repulsive magnet, and the holder may include a magnet such that the repulsive magnet applies a magnetic force to the magnet, thereby biasing the holder toward the lumen. Further, the holder may include a harness configured to be coupled to the magnet, the harness sized and shaped to be slidably disposed within the channel of the coupler body. The cooperatively operated surgical system may further include a clamp pivotally coupled to the coupler body via the rod. The clamp may be configured to transition between an unlocked state, in which the lumen is permitted to receive the elongate shaft, and a locked state, in which the clamp secures the elongate shaft within the lumen. Further, the holder may include one or more cradles coupled to the contact surface. The one or more cradles may each have a channel sized and shaped to slidably receive a rod therethrough, such that the holder may be slidably disposed within the coupler body along the rod.

[0010] Additionally, the coupler interface may include a protrusion, and the coupler body may include a groove configured to receive the protrusion of the coupler interface. Furthermore, the protrusion may include one or more indentations, and the coupler body may include one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends into the groove of the coupler body and an unlocked configuration in which the one or more locking arms do not extend into the groove of the coupler body. Thus, the protrusion of the coupler interface may be received by the groove of the coupler body when the one or more locking arms are in the unlocked configuration, and at least a portion of the one or more locking arms may extend into the one or more indentations of the protrusion when the protrusion is disposed in the groove and the locking arms are in the locked configuration, thereby securing the coupler body to the coupler interface. The one or more locking arms may be biased toward the locked configuration. Additionally, each of the one or more locking arms may have a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration.

[0011] The protrusions of the coupler interface may have a first geometric shape, and the grooves of the coupler body may have a second geometric shape corresponding to the first geometric shape, such that when the protrusions are received by the grooves, rotational movement between the coupler body and the coupler interface is prohibited. Further, the coupler interface may include one or more additional protrusions having the first geometric shape, and the coupler body may include one or more additional grooves having a second geometric shape, such that when the one or more additional protrusions are received by the one or more additional grooves, rotational movement between the coupler body and the coupler interface is prohibited. The coupler body and the coupler interface may be configured to receive a sterile drape therebetween, such that the sterile drape prevents contact between surgical instruments and a robotic arm during laparoscopic surgery.

[0012] Additionally, the coupler body may include one or more tapered surfaces configured to guide the elongate shaft into the lumen and promote self-alignment of the elongate shaft with the lumen by rotating the coupler body relative to the distal end of the robotic arm via the coupler interface as the elongate shaft is inserted into the lumen along the one or more tapered surfaces. The coupler body may further include a clamp configured to transition between an unlocked state, in which the lumen is permitted to receive the elongate shaft, and a locked state, in which the clamp secures the elongate shaft within the lumen. The clamp may be biased toward the locked state. Furthermore, at least a portion of the clamp may include a tapered surface configured to guide the elongate shaft into the lumen and promote transition of the clamp from the locked state to the unlocked state in response to a force applied to the tapered surface by the elongate shaft as the elongate shaft is inserted into the lumen.

[0013] According to another aspect of the present disclosure, a method of using a robotic arm configured to be removably coupled to a surgical instrument having a handle, a working end, and an elongated shaft therebetween, for example, to assist in laparoscopic surgery, is provided. The method may include removably coupling a coupler body to a coupler interface at a distal end of the robotic arm, inserting the elongated shaft of the surgical instrument into a lumen of the coupler body, transitioning the coupler body from an open state, in which the elongated shaft is slidably movable within the lumen, to a closed state, in which longitudinal movement of the elongated shaft relative to the coupler body is prevented while rotational movement of the elongated shaft relative to the coupler body is permitted in response to movement in the handle of the surgical instrument, and freely moving the robotic arm by moving the handle of the surgical instrument once the coupler body is coupled to the coupler interface in the closed state, for example, to perform laparoscopic surgery. The coupler body may rotate relative to the distal end of the robotic arm via the coupler interface as the elongate shaft is inserted into the lumen, self-aligning the lumen with the elongate shaft.

[0014] For example, removably coupling the coupler body to the coupler interface may include actuating one or more locking arms of the coupler body to transition the one or more locking arms from a locked configuration in which at least a portion of the one or more locking arms extend into the grooves of the coupler body to an unlocked configuration in which the one or more locking arms do not extend into the grooves, inserting a protrusion of the coupler interface into the grooves of the coupler body, and releasing the one or more locking arms to transition the one or more locking arms from the unlocked configuration to the locked configuration such that at least a portion of the one or more locking arms extend into one or more recesses of the protrusion, thereby securing the coupler body to the coupler interface. Further, inserting an elongate shaft of a surgical instrument into a lumen of the coupler body may include guiding the elongate shaft into the lumen along one or more tapered surfaces of the coupler body. For example, guiding the elongate shaft into the lumen along one or more tapered surfaces of the coupler body may involve rotating the coupler body relative to the distal end of the robotic arm via the coupler interface, causing the lumen to self-align with the elongate shaft as the elongate shaft is inserted into the lumen along the one or more tapered surfaces.

[0015] Additionally, inserting the elongate shaft of the surgical instrument into the lumen of the coupler body may include actuating a clamp of the coupler body, transitioning the clamp from a locked state to an unlocked state in which the lumen is permitted to receive the elongate shaft, inserting the elongate shaft of the surgical instrument into the lumen, and releasing the clamp, transitioning the clamp from the unlocked state to the locked state, such that the clamp secures the elongate shaft within the lumen. Further, transitioning the coupler body from the open state to the closed state may include transitioning a switch of the coupler body from an unlocked position, in which the elongate shaft is slidably movable within the lumen, to a locked position in which an engagement portion of the switch engages the elongate shaft disposed within the lumen, thereby preventing longitudinal movement of the elongate shaft relative to the coupler body while allowing rotational movement of the elongate shaft relative to the coupler body. Thus, when the elongate shaft is disposed within the lumen and the switch is in the locked position, the engagement portion may apply a frictional force to the elongate shaft configured to allow rotational movement of the elongate shaft relative to the coupler body while preventing longitudinal movement of the elongate shaft relative to the coupler body. The method may further include positioning a sterile drape between the coupler body and the coupler interface prior to removably coupling the coupler body to the coupler interface.

[0016] According to another aspect of the present disclosure, a coupler device is provided for removably coupling a surgical instrument having a handle and an elongated shaft to a distal end of a robotic arm of a cooperative surgical system and for assisting in laparoscopic surgery performed using the surgical instrument, the distal end of the robotic arm including a coupler interface configured to be removably coupled to the coupler device. The coupler device may include a lumen sized and shaped to receive the elongated shaft of the surgical instrument therein, and a coupler body configured to transition between an open state, in which the elongated shaft is slidably movable within the lumen, and a closed state, in which longitudinal movement of the elongated shaft relative to the coupler body is prevented, while rotational movement of the elongated shaft relative to the coupler body is permitted in response to movement of the handle of the surgical instrument. Thus, when the coupler body is coupled to the coupler interface, the coupler body may be configured to rotate relative to the distal end of the robotic arm via the coupler interface to self-align the lumen with the elongate shaft as the elongate shaft is inserted into the lumen.

[0017] The coupler device may further include a switch configured to transition between an unlocked position and a locked position. The switch may include an engagement portion configured to apply a frictional force to the elongate shaft when the elongate shaft is disposed within the lumen and the switch is in the locked position, thereby allowing rotational movement of the elongate shaft within the lumen while prohibiting translational movement of the elongate shaft relative to the coupler body. Additionally, the coupler device may include a sloped surface having a first valley configured to engage the switch in the unlocked position, a second valley configured to engage the switch in the locked position, and a peak disposed between the first and second valleys. The peak may be configured to allow the switch to transition between the unlocked and locked positions when a force applied to the switch exceeds a predetermined force threshold. Furthermore, the switch may have a handle configured to be actuated to transition the switch between the unlocked and locked positions. When the coupler body is coupled to the coupler interface and the elongate shaft is positioned within the lumen, the robotic arm may be configured to be freely movable in response to movements in the handle of the surgical instrument.

[0018] Additionally, the coupler body may include one or more tapered surfaces configured to guide the elongate shaft into the lumen and promote self-alignment of the elongate shaft with the lumen by rotating the coupler body relative to the distal end of the robotic arm via the coupler interface as the elongate shaft is inserted into the lumen along the one or more tapered surfaces. The coupler device may further include a clamp configured to transition between an unlocked state, in which the lumen is permitted to receive the elongate shaft, and a locked state, in which the clamp secures the elongate shaft within the lumen. The clamp may be biased toward the locked state. Furthermore, at least a portion of the clamp may include a tapered surface configured to guide the elongate shaft into the lumen and promote transition of the clamp from the locked state to the unlocked state in response to a force applied to the tapered surface by the elongate shaft as the elongate shaft is inserted into the lumen. Additionally, the clamp may have a handle portion configured to be actuated to transition the clamp from the locked state to the unlocked state.

[0019] Additionally, the coupler device may include a holder slidably disposed within the coupler body, the holder including a friction pad configured to define at least a portion of the lumen. The holder may be biased toward the lumen when the elongate shaft is disposed within the lumen such that the friction pad engages the elongate shaft. In some embodiments, the coupler interface may include a repulsive magnet, and the holder may include a magnet such that the repulsive magnet applies a magnetic force to the magnet, thereby biasing the holder toward the lumen. The holder may include a harness configured to be coupled to the magnet, the harness being sized and shaped to be slidably disposed within the channel of the coupler body. Additionally, the coupler device may include a clamp pivotally coupled to the coupler body via a rod, the clamp configured to transition between an unlocked state, in which the lumen is permitted to receive the elongate shaft, and a locked state, in which the clamp secures the elongate shaft within the lumen. Additionally, the holder may include one or more cradles coupled to the friction pad, each of the one or more cradles including a channel sized and shaped to slidably receive the rod therethrough such that the holder is configured to be slidably disposed within the coupler body along the rod.

[0020] Further, the coupler body may include a groove configured to receive the protrusion of the coupler interface. The coupler device may further include one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends into the groove of the coupler body and an unlocked configuration in which the one or more locking arms do not extend into the groove such that the protrusion of the coupler interface is permitted to be received by the groove when the one or more locking arms are in the unlocked configuration. Additionally, at least a portion of the one or more locking arms may extend into one or more recesses in the protrusion of the coupler interface when the protrusion is disposed in the groove and the locking arm is in the locked configuration, thereby securing the coupler body to the coupler interface. The one or more locking arms may be biased toward the locked configuration. The one or more locking arms may each include a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration. Additionally, the protrusion of the coupler interface may have a first geometric shape and the groove of the coupler body may have a second geometric shape corresponding to the first geometric shape such that when the protrusion is received by the groove, rotational movement between the coupler body and the coupler interface is prohibited. The coupler body may be configured to receive a sterile drape between the coupler body and the coupler interface when the coupler body is coupled to the coupler interface.

[0021] According to another aspect of the present disclosure, another cooperative surgical system for assisting in laparoscopic surgery is provided, the system being performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween. The system may include a robotic arm having a proximal end, a distal end configured to be removably coupled to the surgical instrument, a plurality of links, and a plurality of joints between the proximal and distal ends, the distal end of the robotic arm including a coupler interface; and a coupler body configured to be removably coupled to the coupler interface. The coupler body may include a lumen sized and shaped to receive the elongated shaft of the surgical instrument therein and may be configured to transition between an open state, in which the elongated shaft is slidably movable within the lumen, and a closed state, in which longitudinal movement of the elongated shaft relative to the coupler body is prevented while rotational movement of the elongated shaft relative to the coupler body is permitted in response to movement of the handle of the surgical instrument. Furthermore, when the coupler body is coupled to the coupler interface in a closed state, the robotic arm may be enabled to move freely in response to movements at the handle of a surgical instrument to perform a laparoscopic surgical procedure. In some embodiments, the coupler body may be disposable after a single laparoscopic surgical procedure. Alternatively, the coupler body may be sterilizable so that it may be reused for multiple surgical procedures.

[0022] The system may further include a switch configured to transition between an unlocked position and a locked position. The switch may include an engagement portion configured to engage the elongate shaft when the elongate shaft is disposed within the lumen and the switch is in the locked position, thereby securing the elongate shaft within the lumen. Thus, when the coupler body is coupled to the coupler interface, the elongate shaft is disposed within the lumen, and the switch is in the locked position, the robotic arm may be configured to be freely movable in response to movements at the handle of the surgical instrument. Furthermore, when the elongate shaft is disposed within the lumen and the switch is in the locked position, the engagement portion may be configured to apply a frictional force to the elongate shaft, the frictional force being configured to allow rotational movement of the elongate shaft relative to the coupler body while preventing longitudinal movement of the elongate shaft relative to the coupler body. The switch may include a handle portion configured to be actuated to transition the switch between the unlocked position and the locked position.

[0023] The coupler body may further include a holder slidably disposed within the coupler body. The holder may include a contact surface configured to define at least a portion of the lumen and may be configured to be biased toward the lumen when the elongate shaft is disposed within the lumen, such that the contact surface is configured to engage with the elongate shaft. Additionally, the coupler interface may include a repulsive magnet, and the holder may include a magnet, such that the repulsive magnet may be configured to apply a magnetic force to the magnet, thereby biasing the holder toward the lumen. Further, the holder may include a harness configured to be coupled to the magnet, the harness being sized and shaped to be slidably disposed within the channel of the coupler body. Additionally, the contact surface may be configured to apply a frictional force to the elongate shaft when the elongate shaft is disposed within the lumen and the switch is in the locked position, such that the frictional force may be configured to facilitate allowing rotational movement of the elongate shaft relative to the coupler body while preventing translational movement of the elongate shaft relative to the coupler body.

[0024] The coupler body may further include a clamp configured to transition between an unlocked state, in which the lumen is permitted to receive the elongate shaft, and a locked state, in which the clamp secures the elongate shaft within the lumen. The clamp may be pivotally coupled to the coupler body via the rod and configured to be biased toward the locked state. Thus, the clamp may further include a handle portion configured to be actuated to transition the clamp from the locked state to the unlocked state, e.g., to release a surgical instrument from the coupler body. Furthermore, at least a portion of the clamp may include a tapered surface configured to guide the elongate shaft into the lumen and facilitate the transition of the clamp from the locked state to the unlocked state in response to a force applied to the tapered surface by the elongate shaft as the elongate shaft is inserted into the lumen. Additionally, the holder may include one or more cradles coupled to the contact surface, each including a channel sized and shaped to slidably receive a rod therethrough, such that the holder may be configured to be slidably disposed within the coupler body along the rod.

[0025] The coupler interface may further include a protrusion, and the coupler body may include a groove configured to receive the protrusion of the coupler interface. For example, the protrusion may include one or more indentations, and the coupler body may include one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends into the groove of the coupler body and an unlocked configuration in which the one or more locking arms do not extend into the groove of the coupler body. The protrusion of the coupler interface may be configured to be received by the groove of the coupler body when the one or more locking arms are in the unlocked configuration, and at least a portion of the one or more locking arms may extend into the one or more indentations of the protrusion when the protrusion is disposed in the groove and the locking arms are in the locked configuration, thereby securing the coupler body to the coupler interface. The one or more locking arms may be biased toward the locked configuration. Additionally, the one or more locking arms may each include a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration.

[0026] Further, the protrusions of the coupler interface may have a first geometric shape, and the grooves of the coupler body may have a second geometric shape corresponding to the first geometric shape, such that when the protrusions are received by the grooves, rotational movement between the coupler body and the coupler interface is prohibited. In some embodiments, the coupler interface may include one or more additional protrusions having the first geometric shape, and the coupler body may include one or more additional grooves having a second geometric shape, such that when the one or more additional protrusions are received by the one or more additional grooves, rotational movement between the coupler body and the coupler interface is prohibited. Additionally, the coupler body and the coupler interface may be configured to receive a sterile drape therebetween, such that the sterile drape prevents contact between surgical instruments and a robotic arm during laparoscopic surgery. The coupler body may further include one or more tapered surfaces configured to guide the elongate shaft into the lumen by rotating the coupler body and coupler interface to align the elongate shaft with the lumen as the elongate shaft is inserted into the lumen along the one or more tapered surfaces, thereby facilitating self-alignment of the distal end of the robotic arm with the surgical instrument.

[0027] According to another aspect of the present disclosure, a method is provided for using a robotic arm configured to be removably coupled to a surgical instrument having a handle, a working end, and an elongated shaft therebetween, for example, to assist in laparoscopic surgery. The method may include removably coupling a coupler body to a coupler interface at a distal end of the robotic arm, inserting the elongated shaft of the surgical instrument into a lumen of the coupler body, transitioning the coupler body from an open state, in which the elongated shaft is slidably movable within the lumen, to a closed state, in which longitudinal movement of the elongated shaft relative to the coupler body is prevented while rotational movement of the elongated shaft relative to the coupler body is permitted in response to movement in the handle of the surgical instrument, and freely moving the robotic arm by moving the handle of the surgical instrument once the coupler body is coupled to the coupler interface in the closed state, for example, to perform laparoscopic surgery.

[0028] For example, removably coupling the coupler body to the coupler interface may include actuating one or more locking arms of the coupler body, transitioning the one or more locking arms from a locked configuration in which at least a portion of the one or more locking arms extend into a groove of the coupler body, to an unlocked configuration in which the one or more locking arms do not extend into the groove; inserting a protrusion of the coupler interface into the groove of the coupler body; and releasing the one or more locking arms, transitioning the one or more locking arms from the unlocked configuration to the locked configuration, such that at least a portion of the one or more locking arms extend into one or more recesses of the protrusion, thereby securing the coupler body to the coupler interface. Additionally, inserting the elongate shaft of the surgical instrument into the lumen of the coupler body may include guiding the elongate shaft into the lumen along one or more tapered surfaces of the coupler body, for example, by rotating the coupler body and coupler interface to promote self-alignment of the elongate shaft with the lumen as the elongate shaft is inserted into the lumen along the one or more tapered surfaces.

[0029] Additionally, inserting the elongate shaft of the surgical instrument into the lumen of the coupler body may include actuating a clamp of the coupler body, transitioning the clamp from a locked state to an unlocked state in which the lumen is permitted to receive the elongate shaft, inserting the elongate shaft of the surgical instrument into the lumen, and releasing the clamp, transitioning the clamp from the unlocked state to the locked state, so that the clamp secures the elongate shaft within the lumen. Transitioning the coupler body from the open state to the closed state may include transitioning a switch of the coupler body from an unlocked position, in which the elongate shaft is slidably movable within the lumen, to a locked position in which an engagement portion of the switch engages the elongate shaft disposed within the lumen, thereby preventing longitudinal movement of the elongate shaft relative to the coupler body while allowing rotational movement of the elongate shaft relative to the coupler body. The method may further include positioning a sterile drape between the coupler body and the coupler interface prior to removably coupling the coupler body to the coupler interface.

[0030] According to another aspect of the present disclosure, there is provided a method for using a robotic arm configured to be removably coupled to a surgical instrument having a handle, a working end, and an elongated shaft therebetween, for example, to assist in laparoscopic surgery, the method being performed at a bedside adjacent to a bed holding a patient. The method may include, for example, positioning the robotic arm at the bedside to perform laparoscopic surgery, coupling a surgical instrument to the robotic arm, the robotic arm including a proximal end, a distal end, and multiple connections and joints between the proximal and distal ends, and freely moving the robotic arm while it remains positioned at the bedside, for example, by moving the handle of the surgical instrument coupled thereto to perform the laparoscopic surgery using the surgical instrument. The robotic arm may remain at the bedside while using the surgical instrument coupled to the robotic arm. Thus, a practitioner performing a laparoscopic surgery may remain at the bedside while performing the laparoscopic surgery using surgical instruments coupled to the robotic arm. Coupling the surgical instrument to the robotic arm may include coupling the surgical instrument to the robotic arm using a purely mechanical coupling, coupling the robotic arm only to the elongated shaft of the surgical instrument, coupling the surgical instrument to the robotic arm while keeping the handle of the surgical instrument fully exposed for contact with the surgeon's hand, coupling the surgical instrument to the robotic arm while the robotic arm is positioned at the bedside, and / or removably coupling a coupler body to a coupler interface disposed at the distal end of the robotic arm and removably coupling the surgical instrument to the coupler body.

[0031] The method may further include transitioning the coupler body from an open state, in which the elongate shaft is slidably movable within the lumen of the coupler body, to a closed state, in which the robotic arm is permitted to be freely movable in response to movement at the handle of the surgical instrument, for example, to perform laparoscopic surgery. Thus, when the coupler body is coupled to the coupler interface in the closed state, longitudinal movement of the elongate shaft relative to the coupler body may be prevented, while rotational movement of the elongate shaft relative to the coupler body may be permitted in response to movement at the handle of the surgical instrument. Furthermore, when the coupler body is coupled to the coupler interface in the closed state, the coupler body may apply a frictional force to the elongate shaft sufficient to prevent longitudinal movement of the elongate shaft relative to the coupler body while allowing rotational movement of the elongate shaft relative to the coupler body. Additionally, removably coupling the surgical instrument to the coupler body may include removably coupling the coupler body to a fixed point along the elongate shaft to provide a consistent reference point for force calculations on the surgical instrument. The method may further include positioning a sterile drape between the coupler body and the coupler interface prior to removably coupling the coupler body to the coupler interface. Additionally, the method may include discarding the coupler body after the single laparoscopic surgical procedure. The robotic arm may not be remotely controlled via user input received at the remote surgeon console.

[0032] Furthermore, freely moving the robotic arm by moving the handle of a surgical instrument coupled thereto may include applying a force to the robotic arm via the surgical instrument that exceeds a predetermined threshold, thereby automatically switching the robotic arm to a collaborative manipulation mode in which an impedance is applied to the robotic arm and accounts for the weight of the surgical instrument and the robotic arm. The method may further include adjusting the predetermined threshold of the force applied to the robotic arm via a graphical user interface operably coupled to the robotic arm to automatically switch the robotic arm to the collaborative manipulation mode. Additionally, the multiple joints of the robotic arm may include one or more motorized joints operably coupled to one or more motors disposed in a base coupled to a proximal end of the robotic arm. Thus, the method may include measuring a current in the one or more motors, the current being indicative of the force applied to the robotic arm via the surgical instrument. Additionally, an impedance applied to the robotic arm to account for the weight of the surgical instrument and the robotic arm may be applied via one or more motorized joints of the robotic arm.

[0033] The method may further include maintaining the movement of the robotic arm within a predetermined amount for at least a predetermined dwell period, thereby automatically switching the robotic arm to a passive mode in which the robotic arm maintains a static position. Furthermore, the method may include adjusting at least one of the predetermined amount of movement of the robotic arm or the predetermined dwell period via a graphical user interface operably coupled to the robotic arm to automatically switch the robotic arm to the passive mode. Additionally, the method may include moving the robotic arm by moving the handle of the surgical instrument outside a predefined haptic barrier, thereby automatically switching the robotic arm to a haptic mode in which an impedance is applied to the robotic arm, making movement of the robotic arm in response to movement in the handle of the surgical instrument more viscoelastic in the haptic mode. The method may further include adjusting the position of the predefined haptic barrier via a graphical user interface operably coupled to the robotic arm.

[0034] The method may further include selecting an identification of a surgical instrument coupled to the robotic arm via a graphical user interface operably coupled to the robotic arm. Furthermore, the proximal end of the robotic arm may be coupled to a base, the base coupled to the platform via a stage assembly configured to move the base relative to the platform in at least two degrees of freedom. Thus, the method may include adjusting at least one of a vertical height or a horizontal position of the robotic arm relative to the platform via the stage assembly. For example, adjusting at least one of a vertical height or a horizontal position of the robotic arm via the stage assembly may include providing user input via at least one of a graphical user interface operably coupled to the stage assembly or application of a force by a user at a distal region of the robotic arm in at least one of the at least two degrees of freedom.

[0035] The platform may include a plurality of wheels such that positioning the robotic arm at the bedside to perform laparoscopic surgery may include moving the platform relative to the bed via the plurality of wheels. The method may further include engaging and disengaging braking mechanisms of the plurality of wheels to enable movement of the platform via the plurality of wheels. Furthermore, the method may include displaying a virtual map comprising a graphical representation of the platform relative to the bed within an area surrounding the platform to facilitate positioning the robotic arm at the bedside to perform laparoscopic surgery. Additionally, the method may include adjusting at least one of a height or orientation of an optical sensor having a field of view of the surgical scene comprising at least one of the robotic arm, a surgical instrument coupled to the robotic arm, or the bed, thereby optimizing the field of view of the optical sensor. The method may further include selecting a laparoscope, a retractor tool, a grasper tool, or a surgical cutting tool such that coupling a surgical instrument to the robotic arm may include coupling the laparoscope, a retractor tool, a grasper tool, or a surgical cutting tool to the robotic arm.

[0036] According to another aspect of the present disclosure, another cooperatively manipulated surgical system is provided. The cooperatively manipulated surgical system may include a robotic arm having a proximal end, a distal end configured to be removably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints; a coupler interface disposed at the distal end of the robotic arm; and a coupler body configured to be removably coupled to the coupler interface and the elongated shaft of the surgical instrument. The coupler interface may include a ferrous rod and one or more sensors, e.g., one or more Hall effect sensors, configured to measure a magnetic field of the ferrous rod. The coupler body may include a magnet slidably disposed within the coupler body. For example, the magnet may be configured to induce a magnetic field in the ferrous rod based on the position of the magnet relative to the ferrous rod within the coupler body. Additionally, the collaborative surgical system may include a controller operably coupled to the robotic arm and one or more sensors such that the controller can be programmed to determine whether the coupler body is coupled to the coupler interface based on the magnetic field of the ferrous rod measured by the one or more sensors.

[0037] Further, the controller may be programmed to determine whether a surgical instrument is coupled to the coupler body when the coupler body is coupled to the coupler interface based on the magnetic field of the ferrous rod measured by the one or more sensors. Additionally, the controller may be programmed to identify the size of the surgical instrument coupled to the coupler body based on the magnetic field of the ferrous rod measured by the one or more sensors. The magnet may be biased away from the coupler interface when the coupler body is coupled to the coupler interface.

[0038] The magnet may be positioned at a predefined position within the coupler body such that when the coupler body is coupled to the coupler interface and a surgical instrument is not coupled to the coupler body, the controller may be configured to determine, based on a magnetic field induced in the ferrous rod by the magnet at the predefined position measured by the one or more sensors, that the coupler body is coupled to the coupler interface without a surgical instrument being coupled to the coupler body. Further, when the coupler body is coupled to the coupler interface and a surgical instrument is coupled to the coupler body, the magnet may be positioned at a predefined position within the coupler body such that the controller may be configured to determine, based on a magnetic field induced in the ferrous rod by the magnet at the predefined position measured by the one or more sensors, that the coupler body is coupled to the coupler interface and a surgical instrument is coupled to the coupler body.

[0039] The coupler body may include a holder slidably disposed within the coupler body. The holder may be configured to be coupled to the magnet such that when the elongated rod is coupled to the coupler body, the elongated shaft applies a force to the friction pad, thereby moving the magnet, via the holder, to a predefined position within the coupler body, and the holder may include a friction pad configured to engage with the elongated shaft when the elongated shaft is coupled to the coupler body and the coupler body is coupled to the coupler interface. The coupler interface may include a repulsive magnet configured to apply a magnetic force to the magnet and urge the magnet away from the coupler interface when the coupler body is coupled to the coupler interface. Alternatively, the magnet may be coupled to a compression spring configured to urge the magnet away from the coupler interface.

[0040] The controller may be configured to enable the robotic arm to be freely movable in response to movement at the handle of the surgical instrument when the surgical instrument is coupled to the coupler interface via the coupler body to perform a laparoscopic surgical procedure using the surgical instrument. The cooperatively manipulated surgical system may further include a sterile drape configured to cover the coupler interface and the robotic arm during the surgical procedure, such that the coupler body may be configured to be coupled to the coupler interface with the sterile drape sandwiched therebetween. The coupler body may be configured to be disposed of after a single surgical procedure, while the robotic arm and coupler interface may be configured to be reusable for additional surgical procedures.

[0041] According to another aspect of the present disclosure, a method is provided for using a robotic arm, for example, to assist in laparoscopic surgery, the robotic arm including a proximal end, a distal end having a coupler interface configured to be removably coupled to a surgical instrument via a coupler body, a plurality of links, and a plurality of joints. The method may include measuring, via one or more sensors, a magnetic field of a ferrous rod extending within the distal end of the robotic arm, and determining, via a controller operably coupled to the one or more sensors, whether the coupler body is coupled to the coupler interface at the distal end of the robotic arm based on the magnetic field of the ferrous rod measured by the one or more sensors, wherein the coupler body includes a magnet slidably disposed therein such that the magnetic field of the ferrous rod measured by the one or more sensors varies based on a position of the magnet relative to the ferrous rod, and the magnet is biased away from the coupler interface when the coupler body is coupled to the coupler interface.

[0042] According to another aspect of the present disclosure, a device is provided for removably coupling a surgical instrument having an elongated shaft to a distal end of a robotic arm of a cooperative surgical system to assist in laparoscopic surgery performed using the surgical instrument, the distal end of the robotic arm including a coupler interface having a protrusion. The device may include a coupler body configured to be removably coupled to the coupler interface and the elongated shaft of the surgical instrument. The coupler body may include a groove configured to receive the protrusion of the coupler interface, an opening sized and shaped to receive the elongated shaft therein, and a switch configured to transition between an unlocked position and a locked position. The switch may include an engagement portion configured to engage the elongated shaft when the elongated shaft is disposed within the opening and the switch is in the locked position, thereby securing the elongated shaft within the opening. Thus, when the coupler body is coupled to the coupler interface, the elongate shaft is positioned within the opening, and the switch is in the locked position, the robotic arm may be configured to be freely movable in response to movements in the handle of the surgical instrument.

[0043] Furthermore, when the elongate shaft is disposed within the opening and the switch is in the locked position, the engagement portion may apply a frictional force to the elongate shaft, the frictional force being configured to allow rotational movement of the elongate shaft within the opening while prohibiting translational movement of the elongate shaft relative to the coupler body. Additionally, the coupler body may include one or more tapered surfaces configured to guide the elongate shaft into the opening. The one or more tapered surfaces may be configured to rotate the coupler body and the coupler interface and align the elongate shaft with the opening as the elongate shaft is inserted into the opening along the one or more tapered surfaces, thereby promoting self-alignment of the distal end of the robotic arm with the surgical instrument. The device may further include a clamp configured to transition between an unlocked state, in which the opening is configured to receive the elongate shaft, and a locked state, in which the clamp secures the elongate shaft within the opening. The clamp may be biased toward the locked state. Additionally, at least a portion of the clamp may include a tapered surface configured to guide the elongate shaft into the opening and facilitate a transition of the clamp from a locked state to an unlocked state in response to a force applied to the tapered surface by the elongate shaft as the elongate shaft is inserted into the opening.

[0044] The coupler body may further include a holder slidably disposed within the coupler body. The holder may include a friction pad configured to define at least a portion of the opening, and the holder may be biased toward the opening when the elongated shaft is disposed within the opening such that the friction pad engages the elongated shaft. In some embodiments, the coupler interface may include a repulsive magnet, and the holder may include a magnet such that the repulsive magnet applies a magnetic force to the magnet, thereby biasing the holder toward the opening. Further, the holder may include a harness configured to be coupled to the magnet, the harness being sized and shaped to be slidably disposed within the channel of the coupler body. Additionally, the friction pad may be configured to apply a friction force to the elongated shaft when the elongated shaft is disposed within the opening and the switch is in the locked position, the friction force being configured to allow rotational movement of the elongated shaft within the opening while prohibiting translational movement of the elongated shaft relative to the coupler body.

[0045] The device may further include a clamp pivotally coupled to the coupler body via the rod, the clamp configured to transition between an unlocked state, in which the opening is configured to receive the elongated shaft, and a locked state, in which the clamp secures the elongated shaft within the opening. Furthermore, the holder may include one or more cradles coupled to the friction pad, each including a channel sized and shaped to slidably receive the rod therethrough, such that the holder may be slidably positioned within the coupler body along the rod. In some embodiments, the holder may be coupled to a compression spring configured to apply a spring force to the holder, thereby biasing the holder toward the opening. The clamp may include a handle portion configured to be actuated to transition the clamp from the locked state to the unlocked state. Additionally, the switch may include a handle configured to be actuated to transition the switch between the unlocked and locked positions.

[0046] The protrusion may include one or more recesses, and the coupler body may include one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extend into the grooves of the coupler body and an unlocked configuration in which the one or more locking arms do not extend into the grooves of the coupler body. Thus, the protrusion of the coupler interface may be received by the grooves of the coupler body when the one or more locking arms are in the unlocked configuration, and at least a portion of the one or more locking arms may extend into the one or more recesses of the protrusion when the protrusion is disposed in the grooves and the locking arms are in the locked configuration, thereby securing the coupler body to the coupler interface. The one or more locking arms may be biased toward the locked configuration. Additionally, the one or more locking arms may each include a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration. The protrusion of the coupler interface may have a first geometric shape and the groove of the coupler body may have a second geometric shape corresponding to the first geometric shape such that when the protrusion is received by the groove, rotational movement between the coupler body and the coupler interface is prohibited. Additionally, the coupler body may be configured such that a sterile drape can be placed between the coupler body and the coupler interface when the coupler body is coupled to the coupler interface.

[0047] According to another aspect of the present disclosure, another collaboratively manipulated surgical system is provided. The collaboratively manipulated surgical system may include a robotic arm having a proximal end operably coupled to a base, a distal end configured to be removably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints, a platform coupled to the base, the platform configured to move the base with at least one degree of freedom, a plurality of motors operably coupled to at least some of the plurality of joints, and one or more sensors configured to collect sensor data comprising at least one of 3D depth data or pixel image data. The collaborative surgical system may further include a controller operatively coupled to the robotic arm and the one or more sensors and configured to enable the robotic arm to be freely movable in response to movements at the handles of the surgical instruments to perform laparoscopic surgery using the surgical instruments, the controller being programmed to: identify at least one of a position or orientation of one or more objects in the operating room, e.g., a surgical bed, based on sensor data from the one or more sensors; estimate a relative distance between the one or more objects and the base or at least one of the robotic arms as the base or at least one of the robotic arms moves in the operating room; and apply a torque or impedance via the multiple motors to at least some of the multiple joints of the robotic arm to reposition the robotic arm or stop movement of the robotic arm and avoid a collision between the one or more objects and the base or at least one of the robotic arms when the estimated relative distance approaches a predetermined threshold.

[0048] Further, the controller may be configured to detect movement at the distal end of the robotic arm in a first direction in response to a first force applied to the distal end of the robotic arm by a user, cause the platform to move the base in the first direction in response to detecting movement at the distal end of the robotic arm in the first direction, and cause the platform to stop moving the base in the first direction if the first force applied to the distal end of the robotic arm by the user is below a predetermined threshold. The controller may be configured to cause the platform to move the base in the first direction if the first force applied to the distal end of the robotic arm exceeds a predetermined force threshold. The controller may be configured to identify planes of one or more objects in the operating room based on sensor data from the one or more sensors, and estimate a relative distance between the one or more objects and at least one of the base or the robotic arm based on the planes of the one or more objects. Additionally, the controller may be configured to determine a type of laparoscopic surgery to be performed, identify at least one of a position or orientation of the trocar port based on sensor data from the one or more sensors, and apply torques via the multiple motors to at least some of the multiple joints of the robotic arm to automatically position the robotic arm in a predetermined configuration relative to the trocar port based on the type of laparoscopic surgery to be performed.

[0049] According to another aspect of the present disclosure, another collaborative surgical system is provided. The collaborative surgical system may include a robotic arm having a proximal end, a distal end configured to be removably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints; one or more sensors configured to collect sensor data comprising at least one of 3D depth data or pixel image data; and a controller operably coupled to the robotic arm and the one or more sensors and configured to enable the robotic arm to be freely movable in response to movements at a handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument. The controller may be programmed to determine at least one of a position or orientation of a trocar port relative to the robotic arm based on the sensor data from the one or more sensors, detect movement of the trocar port based on the sensor data from the one or more sensors when a working end of the surgical instrument is inserted through the trocar port, and reposition the robotic arm to maintain a position of the working end of the surgical instrument relative to the trocar port during movement of the trocar port. For example, the controller may be configured to detect movement of the trocar port in response to movement of the surgical bed. Further, the controller may be configured to detect movement of the trocar port in response to movement of the patient's body in response to breathing by the patient. In some embodiments, the controller may be configured to cause the distal end of the robotic arm to retract the working end of the surgical instrument within the trocar port prior to repositioning the robotic arm and maintaining the position of the working end of the surgical instrument relative to the trocar port during movement of the trocar port.

[0050] According to another aspect of the present disclosure, a collaborative surgical system is provided that can be configured to calibrate a new robotic arm. The collaborative surgical system may include a robotic arm having a proximal end configured to be removably coupled to a cart, a distal end configured to be removably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints, an optical scanner configured to measure depth data, and a controller operably coupled to the robotic arm and the optical scanner and configured to enable the robotic arm to be freely movable in response to movements at a handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument. The controller may be programmed to: move the robotic arm in an intended predefined pattern of movement relative to the cart according to a preprogrammed routine; compare depth data from the optical scanner indicative of the actual movement of the robotic arm in response to the preprogrammed routine with the intended predefined pattern of movement; generate an error measure indicative of a deviation between the actual movement of the robotic arm and the intended predefined pattern of movement; and execute an optimization algorithm configured to reduce the error measure such that the deviation between the actual movement of the robotic arm and the intended predefined pattern of movement decreases. The controller may be configured to allow the robotic arm to be freely movable in response to movement at the handle of the surgical instrument to perform a laparoscopic surgical procedure using the surgical instrument when the error measure is below a predetermined threshold.

[0051] According to another aspect of the present disclosure, a collaborative surgical system is provided that can be configured to track a surgical instrument and overlay a virtual menu on a video feed. The collaborative surgical system may include a robotic arm having a proximal end, a distal end configured to be removably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints, and a controller operably coupled to the robotic arm and a laparoscope configured to generate a video feed. The controller may be configured to enable the robotic arm to be freely movable in response to movement at a handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument. Further, the controller may be programmed to overlay a virtual menu on the video feed displayed on the display, track movement of the working end of the surgical instrument in response to movement of the handle of the surgical instrument in the video feed to detect one or more predetermined gesture patterns of the working end movement, and activate a function of the collaborative surgical system associated with the virtual menu based on the detection of the one or more predetermined gesture patterns of the working end movement relative to the virtual menu. The virtual menu may include one or more menu options overlaid on at least one corner of the video feed.

[0052] The functionality of the collaborative surgical system associated with the virtual menu may include, for example, adjusting a holding force threshold that needs to be exceeded and causing the robotic arm to switch from a passive mode, in which the controller causes the robotic arm to maintain a static position, to a collaborative mode, in which the controller allows the robotic arm to be freely movable in response to movements at the handle of a surgical instrument to perform laparoscopic surgery using the surgical instrument. Additionally, the functionality of the collaborative surgical system associated with the virtual menu may include activation of an assisted speculum mode, in which the controller causes the laparoscope to automatically adjust at least one of the field of view or position to assist in the laparoscopic surgery. The controller may be configured to overlay the virtual menu on a video feed displayed on the display in response to user input received via a graphical user interface operably coupled to the controller, in response to a voice command by the user, and / or in response to actuation of an actuator disposed on the robotic arm. Additionally, the controller may be configured to track movement of the working end of the surgical instrument in response to user input received via a graphical user interface operably coupled to the controller, in response to voice commands by a user, and / or in response to actuation of an actuator disposed on the robotic arm.

[0053] According to another aspect of the present disclosure, a collaborative surgical system is provided that may be configured to provide an indication via haptic feedback. The collaborative surgical system may include a robotic arm having a proximal end, a distal end configured to be removably coupled to a surgical instrument, a plurality of linkages, and a plurality of joints, and a controller operably coupled to the robotic arm. The controller may be programmed to automatically switch the robotic arm between a collaborative mode, in which the controller allows the robotic arm to be freely movable in response to movements in the handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument, and a passive mode, in which the controller causes the robotic arm to maintain a static position, and to induce vibrations in the distal end of the robotic arm, the vibrations indicating that the robotic arm has switched from the collaborative mode to the passive mode. For example, the vibrations may be configured to induce negligible movement in the working end of the surgical instrument while being perceptible by a user holding the handle of the surgical instrument. Additionally, the controller may be configured to induce a second vibration in the distal end of the robotic arm when a surgical instrument is coupled to the distal end of the robotic arm, the second vibration being an indication that a surgical instrument has been coupled to the distal end of the robotic arm.

[0054] The controller may be configured to cause the robot arm to switch to a passive mode in response to determining that movement of the robot arm due to movement at the surgical instrument handle is less than a predetermined amount for at least a predetermined dwell period. Furthermore, the controller may be configured to cause the robot arm to switch to a collaborative manipulation mode in response to determining that a force applied to the robot arm due to a force applied to the surgical instrument handle exceeds a predetermined threshold. In the collaborative manipulation mode, the controller may be configured to apply a first impedance to the robot arm to account for the weight of the surgical instrument and the robot arm. Additionally, the controller may be configured to generate an audible alert indicating that the robot arm has switched from the collaborative manipulation mode to the passive mode. The robot arm may further include a base operably coupled to a proximal end of the robot arm, and the system may further include a plurality of motors disposed within the base, the plurality of motors operably coupled to at least some of the plurality of joints. Thus, the controller may be programmed to activate at least one motor of the plurality of motors to induce vibrations in the distal end of the robotic arm.

[0055] According to another aspect of the present disclosure, a collaborative surgical system is provided that may be configured for automated speculum detection. The collaborative surgical system may include a robotic arm having a proximal end, a distal end configured to be removably coupled to a laparoscope, a plurality of linkages, and a plurality of joints; an optical scanner configured to measure depth data; and a controller operably coupled to the robotic arm and the optical scanner. The controller may be programmed to compare movement of the laparoscope based on the depth data from the optical scanner and movement of the laparoscope's field of view during the laparoscope movement based on a video feed collected from the working end of the laparoscope, and identify the type of laparoscope based on the movement of the laparoscope's field of view during the laparoscope movement. The controller may be configured to execute a preprogrammed routine in a calibration mode and cause movement of the laparoscope in a predefined pattern of movement according to the preprogrammed routine. For example, the predefined pattern of movement may include a circular motion. Alternatively, movement of the laparoscope may be in response to movement at the handle of the laparoscope by the user.

[0056] The type of laparoscope may be the angle of the working end of the laparoscope. For example, the controller may be configured to identify the type of laparoscope as a flat-tipped laparoscope when the movement of the laparoscope includes a circular movement and the movement of the laparoscope's field of view during the circular movement of the laparoscope includes a corresponding circular movement. Further, the controller may be configured to identify the type of laparoscope as a flat-tipped laparoscope when the movement of the laparoscope includes a circular movement and the movement of the laparoscope's field of view during the circular movement of the laparoscope does not include a change in depth of field of view. In addition, the controller may be configured to identify the type of laparoscope as an angled-tipped laparoscope when the movement of the laparoscope includes a circular movement and the movement of the laparoscope's field of view during the circular movement of the laparoscope includes a change in depth of field of view. The controller may be configured to enable a robotic arm to be freely movable in response to movement at a handle of the laparoscope to perform laparoscopic surgery using the laparoscope. [Brief explanation of the drawings]

[0057] [Figure 1A] 1A and 1B illustrate a conventional laparoscopic procedure performed by a surgeon and one or more assistants. [Figure 1B] 1A and 1B illustrate a conventional laparoscopic procedure performed by a surgeon and one or more assistants.

[0058] [Figure 2] FIG. 2 illustrates an exemplary cooperative surgical system constructed in accordance with the principles of the present disclosure.

[0059] [Figure 3] FIG. 3 illustrates an exemplary robotic arm of the system of FIG. 2 constructed in accordance with the principles of the present disclosure.

[0060] [Figure 4A] 4A and 4B illustrate an exemplary wrist portion of a robotic arm constructed in accordance with the principles of the present disclosure. [Figure 4B] 4A and 4B illustrate an exemplary wrist portion of a robotic arm constructed in accordance with the principles of the present disclosure.

[0061] [Figure 4C] FIG. 4C is an enlarged view of an exemplary surgical instrument coupling mechanism of the wrist portion of FIGS. 4A and 4B.

[0062] [Figure 4D] 4D is a close-up view of an exemplary robotic arm coupler interface of the surgical instrument coupling mechanism of FIG. 4C constructed in accordance with the principles of the present disclosure.

[0063] [Figure 5A] 5A and 5B illustrate an exemplary surgical instrument coupler body of the surgical instrument coupling mechanism of FIG. 4C constructed in accordance with the principles of the present disclosure. [Figure 5B]5A and 5B illustrate an exemplary surgical instrument coupler body of the surgical instrument coupling mechanism of FIG. 4C constructed in accordance with the principles of the present disclosure.

[0064] [Figure 6A] FIG. 6A illustrates an alternative exemplary surgical instrument coupler body constructed in accordance with the principles of the present disclosure.

[0065] [Figure 6B] 6B-6D illustrate the attachment of the coupler body of FIG. 6A to a surgical retractor device in accordance with the principles of the present disclosure. [Figure 6C] 6B-6D illustrate the attachment of the coupler body of FIG. 6A to a surgical retractor device in accordance with the principles of the present disclosure. [Figure 6D] 6B-6D illustrate the attachment of the coupler body of FIG. 6A to a surgical retractor device in accordance with the principles of the present disclosure.

[0066] [Figure 7A] FIG. 7A illustrates another alternative exemplary surgical instrument coupler body constructed in accordance with the principles of the present disclosure.

[0067] [Figure 7B] 7B-7D illustrate the attachment of the coupler body of FIG. 7A to a laparoscopic surgical device in accordance with the principles of the present disclosure. [Figure 7C] 7B-7D illustrate the attachment of the coupler body of FIG. 7A to a laparoscopic surgical device in accordance with the principles of the present disclosure. [Figure 7D] 7B-7D illustrate the attachment of the coupler body of FIG. 7A to a laparoscopic surgical device in accordance with the principles of the present disclosure.

[0068] [Figure 8] 8A and 8B illustrate the robotic arm in a sterile drape ready configuration.

[0069] [Figure 9]9A and 9B illustrate the robotic arm covered in a sterile drape.

[0070] [Figure 10A] 10A-10D illustrate rotation of the shoulder joint of a robotic arm in accordance with the principles of the present disclosure. [Figure 10B] 10A-10D illustrate rotation of the shoulder joint of a robotic arm in accordance with the principles of the present disclosure. [Figure 10C] 10A-10D illustrate rotation of the shoulder joint of a robotic arm in accordance with the principles of the present disclosure. [Figure 10D] 10A-10D illustrate rotation of the shoulder joint of a robotic arm in accordance with the principles of the present disclosure.

[0071] [Figure 11A] FIG. 11A illustrates an exemplary cooperative surgical system having an optical scanner in accordance with the principles of the present disclosure, and FIG. 11B illustrates the optical scanner of FIG. 11A. [Figure 11B] FIG. 11A illustrates an exemplary cooperative surgical system having an optical scanner in accordance with the principles of the present disclosure, and FIG. 11B illustrates the optical scanner of FIG. 11A.

[0072] [Figure 11C] FIG. 11C illustrates an exemplary cooperative surgical system having multiple optical scanners in accordance with the principles of the present disclosure.

[0073] [Figure 12] FIG. 12 illustrates a user operating the cooperative surgical system of FIG. 11A in accordance with the principles of the present disclosure.

[0074] [Figure 13] FIG. 13A illustrates the field of view of an optical scanner during a laparoscopic surgical procedure, and FIG. 13B illustrates a depth map of the field of view of the optical scanner of FIG. 13A.

[0075] [Figure 14] FIG. 14 illustrates some exemplary components that may be included within a collaboratively manipulated robotic platform in accordance with the principles of the present disclosure.

[0076] [Figure 15] FIG. 15 is a flow chart illustrating the operation of a cooperatively manipulated surgical system in accordance with the principles of the present disclosure.

[0077] [Figure 16] FIG. 16 is a flow chart illustrating surgical instrument calibration of a cooperative surgical system in accordance with the principles of the present disclosure.

[0078] [Figure 17] FIG. 17 is a flow chart illustrating the operation of a robotic arm in accordance with the principles of the present disclosure.

[0079] [Figure 18] 18A and 18B are free body diagrams illustrating the forces applied to a surgical instrument coupled to a robotic arm during a laparoscopic surgical procedure.

[0080] [Figure 19] FIG. 19 is a table of example values ​​associated with several arrangements of passive modes of a robotic arm in accordance with the principles of the present disclosure.

[0081] [Figure 20] FIG. 20 illustrates an exemplary overview of some features and capabilities of a cooperative surgical system according to the principles of the present disclosure.

[0082] [Figure 21] FIG. 21 is a schematic overview of some of the electrical components and connectivity of a cooperatively manipulated surgical system in accordance with the principles of the present disclosure.

[0083] [Figure 22]FIG. 22 is a flow chart illustrating an exemplary process for obtaining and processing data from an optical scanner and an exemplary use of the data in accordance with the principles of the present disclosure.

[0084] [Figure 23] FIG. 23 is a schematic overview of data flow for a collaborative surgical system in accordance with the principles of the present disclosure.

[0085] [Figure 24] FIG. 24 is another schematic overview of data flow for a cooperative surgical system in accordance with the principles of the present disclosure.

[0086] [Figure 25] FIG. 25 is a schematic overview of the data flow and output control of a cooperative surgical system in accordance with the principles of the present disclosure.

[0087] [Figure 26] FIG. 26 is a schematic overview of data flow within a network of collaborative surgical systems in accordance with the principles of the present disclosure.

[0088] [Figure 27A] 27A-27D illustrate vertical and horizontal movement of a robotic arm in accordance with the principles of the present disclosure. [Figure 27B] 27A-27D illustrate vertical and horizontal movement of a robotic arm in accordance with the principles of the present disclosure. [Figure 27C] 27A-27D illustrate vertical and horizontal movement of a robotic arm in accordance with the principles of the present disclosure. [Figure 27D] 27A-27D illustrate vertical and horizontal movement of a robotic arm in accordance with the principles of the present disclosure.

[0089] [Figure 28A] 28A-28D illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 28B]28A-28D illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 28C] 28A-28D illustrate an exemplary graphical user interface of a collaborative surgical system. [Figure 28D] 28A-28D illustrate an exemplary graphical user interface of a collaborative surgical system.

[0090] [Figure 29] FIG. 29 is a schematic illustration of an alternative cooperative surgical system constructed in accordance with the principles of the present disclosure.

[0091] [Figure 30] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 31-1] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 31-2] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 32] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 33] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 34] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 35-1] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 35-2] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 36]30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 37] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 38] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 39] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 40] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 41] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 42] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure. [Figure 43] 30A-43 illustrate various alternative surgical instrument coupling mechanisms constructed in accordance with the principles of the present disclosure.

[0092] [Figure 44A] 44A and 44B illustrate another exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure. [Figure 44B] 44A and 44B illustrate another exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure.

[0093] [Figure 44C] FIG. 44C is a cross-sectional view of the surgical instrument coupling mechanism of FIG. 44A.

[0094] [Figure 45A] 45A and 45B illustrate the coupler interface of the surgical instrument coupling mechanism of FIGS. 44A-44C. [Figure 45B]45A and 45B illustrate the coupler interface of the surgical instrument coupling mechanism of FIGS. 44A-44C.

[0095] [Figure 45C] FIG. 45C is a cross-sectional view of the coupler interface of FIG. 45B.

[0096] [Figure 46A] 46A-46C illustrate the coupler body of the surgical instrument coupling mechanism of FIGS. 44A-44C. [Figure 46B] 46A-46C illustrate the coupler body of the surgical instrument coupling mechanism of FIGS. 44A-44C. [Figure 46C] 46A-46C illustrate the coupler body of the surgical instrument coupling mechanism of FIGS. 44A-44C.

[0097] [Figure 46D] FIG. 46D illustrates the magnet holder of the coupler body of FIGS. 46A-46C.

[0098] [Figure 46E] FIG. 46E is a cross-sectional view of the coupler body of FIG. 46A.

[0099] [Figure 47A] FIG. 47A is a cross-sectional view of the coupler body of FIG. 46A when the coupler body is removed from the coupler interface.

[0100] [Figure 47B] FIG. 47B is a cross-sectional view of the surgical instrument coupling mechanism of FIG. 44A when the coupler body is coupled to the coupler interface.

[0101] [Figure 48A] 48A and 48B illustrate various views of the surgical instrument coupling mechanism of FIG. 44A when coupling a coupler body to a coupler interface in accordance with the principles of the present disclosure. [Figure 48B]48A and 48B illustrate various views of the surgical instrument coupling mechanism of FIG. 44A when coupling a coupler body to a coupler interface in accordance with the principles of the present disclosure.

[0102] [Figure 49] FIG. 49 is a cross-sectional view of the surgical instrument coupling mechanism of FIG. 44A when a surgical instrument is coupled to the coupler body.

[0103] [Figure 50A] 50A and 50B illustrate various views of the surgical instrument coupling mechanism of FIG. 44A when coupling a surgical instrument to the coupler body in accordance with the principles of the present disclosure. [Figure 50B] 50A and 50B illustrate various views of the surgical instrument coupling mechanism of FIG. 44A when coupling a surgical instrument to the coupler body in accordance with the principles of the present disclosure.

[0104] [Figure 51] FIG. 51 is a cross-sectional view of an alternative exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure when a surgical instrument is coupled to the coupler body.

[0105] [Figure 52A] 52A-52E illustrate an alternative exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure. [Figure 52B] 52A-52E illustrate an alternative exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure. [Figure 52C] 52A-52E illustrate an alternative exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure. [Figure 52D] 52A-52E illustrate an alternative exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure. [Figure 52E] 52A-52E illustrate an alternative exemplary surgical instrument coupling mechanism constructed in accordance with the principles of the present disclosure.

[0106] [Figure 53A] 53A and 53B illustrate an alternative exemplary coupler body constructed in accordance with the principles of the present disclosure. [Figure 53B] 53A and 53B illustrate an alternative exemplary coupler body constructed in accordance with the principles of the present disclosure.

[0107] [Figure 54] FIG. 54 illustrates another alternative exemplary coupler body constructed in accordance with the principles of the present disclosure.

[0108] [Figure 55] FIG. 55 illustrates an exemplary virtual overlay of a graphical user interface of a collaborative surgical system.

[0109] [Figure 56] FIG. 56 illustrates an alternative exemplary robotic arm motorized joint constructed in accordance with the principles of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0110] Detailed Description Disclosed herein are cooperatively operated surgical robotic systems and methods of use for assisting an operator, e.g., a surgeon, in performing a surgical procedure, e.g., a laparoscopic procedure. Currently, laparoscopic procedures typically require a surgeon and one or more assistants. For example, as shown in FIG. 1A , during a laparoscopic procedure, assistant A1 may be required to hold retractor device 12 and expose tissue for surgeon S, while another assistant A2 may be required to hold laparoscopic device 10 during the procedure and provide surgeon S with a view of the surgical space within the patient via a display (not shown). As shown in FIG. 1A , assistant A2 may be required to hold laparoscopic device 10 in an impractical position, e.g., between the arms of surgeon S, while the surgeon actively operates additional surgical instruments, e.g., surgical instruments 14 and 16. As further shown in FIG. 1A, surgeon S may need to let go of surgical instruments 16 to guide / reposition laparoscopic device 10 held by assistant A2 to achieve the field of view desired by the surgeon.

[0111] As shown in FIG. 1B , rail-mounted orthopedic retractors 18 may be used to hold one or more surgical instruments in place during a laparoscopic procedure in an attempt to free the surgeon's and / or assistant's hands for other tasks and for stability. As shown in FIG. 1B , a first rail-mounted orthopedic retractor 18a may include a retractor end 20a for engaging and holding in place a laparoscopic device 10 in response to activation of a locking portion 22a. For example, the locking portion 22a may be disengaged so that the retractor 18a can be manually positioned at a desired location relative to the patient and reengaged to lock the retractor 18a, and thus the laparoscopic device 10 coupled thereto, in the desired position. As shown in FIG. 1B , a second rail-mounted orthopedic retractor 18b, having a retractor end 20b, may also be used to engage and hold in place another surgical instrument in response to activation of a locking portion 22b during the procedure. Thus, retractors 18a and 18b require extensive manual interaction with locks 22a and 22b, and with retractors 18a and 18b themselves, to reposition and lock the individual tools in place.

[0112] The collaborative surgical robotic systems described herein provide superior control and stability so that surgeons and / or assistants can seamlessly position a variety of off-the-shelf surgical instruments as needed, thus avoiding workflow limitations inherent in both human and mechanical solutions. For example, the robotic arms of the collaborative surgical robotic system can provide surgical assistance by holding a first surgical instrument, e.g., a laparoscope, via a first robotic arm and a second surgical instrument, e.g., a retractor, via a second robotic arm steadily throughout the procedure, providing optimal views of the surgical site and reducing variability in the forces applied by the surgical instruments to the body wall at the trocar point. As will be understood by one skilled in the art, the robotic arms of the collaborative robotic surgical systems described herein may preferably hold any surgical instrument having an elongated instrument shaft used for surgical procedures, such as laparoscopic procedures, including, for example, endoscopes / laparoscopes, retractors, graspers, surgical scissors, needle holders, clamps, suturing instruments, cautery tools, staplers, clip appliers, etc.

[0113] A collaborative surgical robotic system also allows the surgeon to easily manipulate both tools when necessary, providing superior control, stability, and overall safety throughout the procedure. Any implementation of the system described herein allows the surgeon to collaboratively manipulate instruments directly at the patient bedside, while remaining sterile. For example, the system may include two robotic arms that the surgeon can use to hold both a laparoscope and a retractor. During the surgical procedure, the system may seamlessly reposition either instrument to provide optimal visualization and exposure of the surgical field. Both instruments may be directly coupled to the system's robotic arms, and the system may constantly monitor and record the positions of the two instruments and / or the two robotic arms throughout the procedure. Additionally, the system may record information such as the location of surgical entry ports, the position and movement of the surgeon's hands, the position and orientation of surgical instruments, the position and orientation of surgical instruments attached to the robotic arm, including whether they are attached to a robotic arm, the patient position, and the orientation and height of the patient table, sensor readings related to the forces applied at the proximal and distal ends of surgical instruments attached to the robotic arm, the forces required to hold each instrument in place, endoscopic video streams, algorithm parameters, and operating room 3D streams captured using optical scanning devices.

[0114] Such data may, in some implementations, be used to develop a database of historical data that may be used to develop algorithms used to control one or more aspects of the system's operation. Additionally, such data may be used to control one or more aspects of the system's operation / one or more algorithms of the system during a procedure. For example, data may be used to assess the level of fatigue of a user of the system.

[0115] As an operator manipulates the robotic arm of a collaborative robotic surgical system by applying movements to a surgical instrument coupled to the robotic arm, the system may automatically transition the robotic arm between various operating modes in response to determining a predefined condition. For example, the system may transition the robotic arm to a passive mode in response to determining that movement of the robotic arm due to movement in the handle of a surgical instrument is less than a predetermined amount for at least a predetermined dwell period, such that in the passive mode the robotic arm maintains a static position, e.g., to prevent damage to equipment and / or injury to the patient. Additionally, the system may transition the robot arm to the collaborative manipulation mode in response to determining that a force applied at the robot arm exceeds a predetermined threshold due to a force applied at the surgical instrument handle such that a first impedance is applied to the robot arm in the collaborative manipulation mode and accounts for the weight of the surgical instrument and the robot arm, while in the collaborative manipulation mode the robot arm is allowed to move freely in response to movement at the surgical instrument handle to perform laparoscopic surgery using the surgical instrument. Furthermore, the system may transition the robot arm to the haptic mode in response to determining that at least a portion of the robot arm is outside a predefined haptic barrier such that a second impedance greater than the first impedance is applied to the robot arm, thereby making movement of the robot arm more viscoelastic in the haptic mode than in the collaborative manipulation mode in response to movement at the surgical instrument handle. The system may further transition the robotic arm into a robot-assisted mode in response to detecting various conditions that warrant automated movement of the robotic arm, for example, to guide a surgical instrument attached thereto along a planned trajectory or to avoid collision with another object or person within the surgical space.

[0116] Referring now to Figure 2, a collaborative surgical robotic system 200 is provided. As shown in Figure 2, system 200 may include a platform 100, e.g., a surgical cart, sized and shaped to support one or more robotic arms 300, e.g., robotic arm 300a and robotic arm 300b (each having a surgical instrument coupler interface 400 for removably coupling to a surgical instrument), and a computing system operably coupled to platform 100 and robotic arm 300. As shown in Figure 2, system 200 may further include a graphical user interface display 110 for displaying operational information and receiving user input.

[0117] Additionally, each robotic arm 300 may further include an indicator 334 to visually indicate, in real time, the operational mode associated with the respective robotic arm. For example, the indicator 334 may be positioned on at least the elbow joint of the robotic arm. Additionally or alternatively, the indicator 334 may be located anywhere on the system 200, such as on the platform 100, on the display 110, on one or more links and / or joints, etc. Furthermore, the indicator 334 may include a light, e.g., an LED light, that may illuminate in a variety of distinct colors and in distinct patterns, e.g., solid or flashing. For example, each operational mode of the system 200 may be associated with a uniquely colored light, such as red, yellow, blue, green, purple, white, orange, etc. Thus, the indicator 334 may indicate a transition from one operational mode to another. Additionally or alternatively, the transition from one operating mode to another may be indicated to the user via haptic feedback, such as a vibration delivered to the distal end of the robotic arm 300 and thus to the surgical instrument coupled thereto. For example, the distal end of the robotic arm 300 may vibrate as the robotic arm 300 transitions from the collaborative manipulation mode to the static mode, assuring the user that the robotic arm 300 is in the static mode and will remain in place upon release by the user. Additionally or alternatively, an audible alert may be emitted to indicate to the user when the robotic arm 300 transitions from one operating mode to another.

[0118] 2, platform 100 may include one or more stages coupled to base portions of one or more robotic arms, e.g., base portion 302a of robotic arm 300a and base portion 302b of robotic arm 300b, for example, to provide movement to the individual robotic arms at least in horizontal and vertical directions relative to platform 100. Each stage may include a vertical extension, e.g., vertical extension 106a or vertical extension 106b, for moving robotic arm 300a or robotic arm 300b, respectively, independently vertically relative to platform 100, and a horizontal extension, e.g., horizontal extension 108a or horizontal extension 108b, for moving robotic arm 300a or robotic arm 300b, respectively, independently horizontally relative to platform 100, thereby allowing operator flexibility in positioning robotic arm 300 relative to a patient.

[0119] Additionally, platform 100 may include multiple wheels 104, e.g., caster wheels, to provide mobility for platform 100, and thus robotic arm 300, within the operating room. Each wheel 104 may include a braking mechanism, which may be activated to prevent movement of platform 100 via wheel 104. Thus, platform 100 may independently move each of robotic arms 300a and 300b in any direction, including a first or vertical direction toward and away from the floor, a second or horizontal direction toward and away from the patient, and / or a third or horizontal direction aligned with the patient's height. In some embodiments, platform 100 may simultaneously move robotic arms 300a and 300b in the same direction and may further cause rotational movement of robotic arms 300a and 300b. Once ready for operation, platform 100 may be moved to a desired location on the side of a patient's bed via wheels 104 and locked into place, and the vertical and horizontal positions of robotic arms 300a and 300b may be adjusted to an optimal position relative to the patient for the procedure via vertical extensions 106a, 106b and horizontal extensions 108a, 108b in response to user input received via graphical user interface display 110. As described in further detail below, platform 100 may automatically move robotic arms 300a and 300b in response to detecting potential collisions with other objects and / or people in the operating room and / or user inputs applied via the robotic arms, for example, during a laparoscopic procedure and / or during setup of the robotic arms.

[0120] Surgical robotic system 200 is configured for cooperative operation so that system 200 can assist a user or operator, e.g., a surgeon and / or a surgical assistant, by allowing the user to freely move robotic arm 300 a and / or robotic arm 300 b due to manipulation of one or more surgical instruments coupled to the robotic arms in response to force inputs provided by the user to the surgical instruments. Thus, system 200 can be configured so that robotic arm 300 moves in direct response to the operator's movements of the surgical instruments coupled thereto, while not being remotely controlled, to compensate for the mass of the surgical instruments and individual robotic arms and provide localized impedance along the robotic arms, thereby increasing the accuracy of the operator's movements or actions as they manipulate the surgical instruments.

[0121] System 200 may be particularly useful in laparoscopic and / or other surgical procedures utilizing elongated instruments that may be inserted into a patient's body, for example, via a cannula, to enable surgical intervention. As will be understood by those skilled in the art, system 200 may be used for any desired or suitable surgical procedure. Furthermore, system 200 may be used in conjunction with or in cooperation with video surveillance provided by one or more cameras and / or one or more endoscopes, so that an operator of system 200 may view and monitor the use of instruments coupled to robotic arms 300 a, 300 b via respective coupler interfaces, as described in further detail below. For example, robotic arm 300 a may be removably coupled to an endoscope and manipulate the endoscope, while robotic arm 300 b may be removably coupled to a surgical instrument and manipulate the surgical instrument.

[0122] Referring now to FIG. 3 , a surgical support arm, e.g., robotic arm 300, is provided. As described above, system 200 may include multiple robotic arms, e.g., robotic arm 300a and robotic arm 300b. However, because each robotic arm may be similarly constructed, only a single robotic arm will be described with reference to FIG. 3 , collectively as robotic arm 300, for simplicity. Aspects of the robotic arm described herein may utilize structure from U.S. Pat. No. 10,118,289 to Louveau, the entire contents of which are incorporated herein by reference. Robotic arm 300 may include multiple arm sections / links and multiple articulation joints extending from a base portion. For example, robotic arm 300 may include a base portion, a shoulder portion, an elbow portion, and a wrist portion, thereby mimicking the kinematics of a human arm. 3, the robotic arm 300 may include a base, which includes a base portion 302 rotatably coupled to a shoulder portion 304 at a base joint 303. For example, the shoulder portion 304 may sit on top of the base portion 302 and may be rotated relative to the base portion 302 about axis Q1 at the base joint 303. In some embodiments, the robotic arm 300 may be swapped, replaced, or coupled to a base in any desired arrangement.

[0123] The robotic arm 300 may further include a shoulder link 305, which includes a proximal shoulder link 306 rotatably coupled to a distal shoulder link 308. A proximal end of the proximal shoulder link 306 may be rotatably coupled to the base shoulder portion 304 at shoulder joint 318 such that the proximal shoulder link 306 may be rotated relative to the shoulder portion 304 about axis Q2 at shoulder joint 318. As shown in FIG. 3 , axis Q2 may be perpendicular to axis Q1. A distal end of the proximal shoulder link 306 may be rotatably coupled to the proximal end of the distal shoulder link 308 at joint 320 such that the distal shoulder link 308 may be rotated relative to the proximal shoulder link 306 about axis Q3 at joint 320. As shown in FIG. 3 , axis Q3 may be parallel to the longitudinal axis of the shoulder link 305. Additionally, the robotic arm 300 may include an actuator 330, e.g., a lever, button, collar, or switch, operably coupled to the distal shoulder link 308 and / or the proximal shoulder link 306 such that the distal shoulder link 308 may only be rotated relative to the proximal shoulder link 306 in response to actuation of the actuator 330. Thus, axis Q3 may be a “set” axis such that the distal shoulder link 308 may be rotated and fixed relative to the proximal shoulder link 306 during a set phase prior to an operation phase in which the robotic arm 300 is used in a surgical procedure, as described in further detail with respect to FIGS. 10A-10D . As shown in FIG. 3 , the actuator 330 may be located on the elbow link 310.

[0124] In some embodiments, the distal shoulder link 308 may be manually rotated relative to the proximal shoulder link 306 in predefined increments in response to actuation of the actuator 330. Alternatively, in response to actuation of the actuator 330, the distal shoulder link 308 may be automatically rotated relative to the proximal shoulder link 306 until the actuator 330 is released, as described in further detail below with respect to FIG. 56 and in U.S. patent application Ser. No. 18 / 331,060 to Noonan, the entire contents of which are incorporated herein by reference. For example, the actuator 330 may be a button, collar, or switch operably coupled to a motor operably coupled to the distal shoulder link 308 and / or the proximal shoulder link 306 such that in response to actuation of the actuator 330, the associated motor rotates the distal shoulder link 308 relative to the proximal shoulder link 306. The motor may be located within the base of the robotic arm 300, or alternatively, the motor may be located adjacent the joint 320, for example, on the shoulder link 305. Thus, the actuator 330 may be a button, collar, or switch that allows dual actuation, for example, a first actuation to rotate the distal shoulder link 308 relative to the shoulder link 306 in a first direction, and a second actuation to rotate the distal shoulder link 308 in a second direction opposite the first direction. In some embodiments, the button or switch may be located on a graphical user interface, such as the display 110. Additionally, in some embodiments, the distal shoulder link 308 may be rotated relative to the proximal shoulder link 306 automatically, for example, during setup, by a processor of the collaborative robotic platform to avoid collisions, as described in more detail below.

[0125] The robotic arm 300 may further include an elbow link 310. A proximal end of the elbow link 310 may be rotatably coupled to a distal end of the distal shoulder link 308 at elbow joint 322 such that the elbow link 310 may be rotated relative to the distal shoulder link 308 about axis Q4 at elbow joint 322. The robotic arm 300 may further include a wrist portion 311, which may include a proximal wrist link 312 rotatably coupled to the distal end of the elbow link 310 at wrist joint 324, a central wrist link 314 rotatably coupled to the proximal wrist link 312 at joint 326, and a distal wrist link 316 rotatably coupled to the central wrist link 314 at joint 328, as further shown in FIGS. Thus, wrist portion 311 may be rotated relative to elbow linkage 310 about axis Q5 at wrist joint 324, central wrist portion 314 may be rotated relative to proximal wrist linkage 312 about axis Q6 at joint 326, and distal wrist linkage 316 may be rotated relative to central wrist linkage 314 about axis Q7 at joint 328.

[0126] The robotic arm 300 may further include a wrist portion 311, which may include a proximal wrist linkage 312 rotatably coupled to a distal end of the elbow linkage 310 at wrist joint 324, as further shown in FIGS. 4A and 4B, a central wrist linkage 314 rotatably coupled to the proximal wrist linkage 312 at joint 326, and a distal wrist linkage 316 coupled to / extending from the central wrist linkage 314, which may be rotatably coupled to a surgical instrument coupler interface (not shown) at joint 328, such as coupler interface 400 as further shown in FIGS. 4A and 4B, coupler interface 4500 as further shown in FIGS. 44A and 44B, and / or coupler interface 5200 as further shown in FIGS. 52A-52E. Thus, the wrist portion 311 may be rotated relative to the elbow joint 310 about axis Q5 at wrist joint 324, the central wrist portion 314 may be rotated relative to the proximal wrist joint 312 about axis Q6 at joint 326, and the surgical instrument coupler interface may be rotated relative to the distal wrist joint 316, and therefore the central wrist joint 314, about axis Q7 at joint 328.

[0127] 3 and 4B, the robotic arm 300 may include an actuator 332, e.g., a lever, button, or switch, operably coupled to the elbow joint 310 and / or the proximal wrist joint 312 at joint 324, such that the proximal wrist joint 312 can only rotate relative to the elbow joint 310 in response to actuation of the actuator 332. Thus, axis Q5 may be a “set” axis such that the proximal wrist joint 312 can be rotated and fixed relative to the elbow joint 310 during a set phase in response to actuation of the actuator 332, e.g., prior to an operation phase in which the robotic arm 300 is used in a surgical procedure. When the actuator 332 is in an unactuated state, the set joint 324 prevents relative movement between the proximal wrist joint 312 and the elbow joint 310, such that the proximal wrist joint 312 is fixed relative to the elbow joint 310. In some preferred embodiments, in response to actuation of the actuator 332, the proximal wrist link 312 may be manually rotated relative to the elbow link 310 in predefined increments, thereby eliminating the need for additional motors and / or electronics in the distal region of the robotic arm 300. Alternatively, in response to actuation of the actuator 332, the proximal wrist link 312 may be automatically rotated relative to the elbow link 310 until the actuator 332 is released, for example, via a motor operably coupled to the proximal wrist link 312 and / or elbow link 310 at joint 324.

[0128] 3, the robotic arm 300 may include multiple motors, such as motors M1, M2, and M3, which may all be located within the base of the robotic arm 300, and motor M4, which may preferably be located adjacent to joint 320. Alternatively, motor M4 may also be located within the base of the robotic arm 300. Motors M1, M2, and M3 may each be operatively coupled to a respective motorized joint of the robotic arm 300, such as base joint 303, shoulder joint 318, and elbow joint 322, thereby applying a localized impedance at the respective joint. For example, motors M1, M2, and M3 may each create an impedance / torque at either base joint 303, shoulder joint 318, or elbow joint 322, thereby effectively applying an impedance at the distal end of the robotic arm, such as at the point of attachment with a surgical instrument, to improve the sensation experienced by the operator during manipulation of the surgical instrument and the operator's performance during the surgical procedure. For example, impedance may be applied to the distal end of the robotic arm 300, and thus the surgical instrument coupled thereto, to provide a sense of viscoelasticity, stiffness, and / or inertia to the operator manipulating the surgical instrument. Additionally, the applied impedance may simulate tissue density or stiffness, communicate surgical boundaries to the operator, and may be used to direct the surgical instrument along a desired path or otherwise. In some embodiments, motors may actuate individual joints, thereby causing movement of the robotic arm 300 about the individual joints. Thus, axes Q1, Q2, and Q4 may each be “motor” axes, such that motors M1, M2, and M3 may apply impedance / torque to the base joint 303, shoulder joint 318, and elbow joint 322, respectively, to prevent or actuate rotation about the individual axes. As described in further detail below, motors M1, M2, and M3 may be controlled by a processor of the collaborative robotic platform.By using three motor axes, some implementations of the robotic arm 300 may apply force / torque in three directions at the distal end of the robotic arm 300, thereby moving a surgical instrument coupled to the distal end of the robotic arm 300 in three degrees of freedom.

[0129] Motor M4 is operably coupled to set joint 320, as described above, and may apply a torque to joint 320 to actuate rotation of distal shoulder link 308 relative to proximal shoulder link 306 about axis Q3. Unlike other motorized joints described herein, e.g., base joint 303, shoulder joint 318, and elbow joint 322, motorized joint 320 is preferably not “backdrivable,” in that a user cannot actuate motorized joint 320, e.g., through movement of a surgical instrument coupled to a robotic arm, when the system is in a collaborative manipulation mode. Instead, as described above, actuation of motorized joint 320 may occur automatically via one or more actuators, e.g., actuator 330, which can be actuated to cause rotation of distal shoulder link 308 relative to proximal shoulder link 306.

[0130] Axis Q6 and axis Q7 may each be a "passive" axis such that central wrist joint 314 can be rotated relative to proximal wrist joint 312 at passive joint 326 without any applied impedance from system 200, and surgical instrument coupling interface can be rotated relative to distal wrist joint 316 at passive joint 328 without any applied impedance from system 200. The distal end of distal wrist link 316 may be rotatably coupled to a surgical instrument coupler interface for removably coupling with a surgical instrument via coupler body 500 as further shown in Figures 4A and 4B, coupler body 4600 as further shown in Figures 44A and 44B, coupler body 5100 as further shown in Figure 51, coupler body 5300 as further shown in Figures 53A and 53B, and / or coupler body 5400 as further shown in Figure 54, which may be removably coupled to a surgical instrument and coupler interface as described in further detail below. Alternatively, wrist portion 11 may include a passive ball joint at the attachment point with the surgical instrument, as described in U.S. Pat. No. 10,582,977, the entire disclosure of which is incorporated herein by reference.

[0131] 3, the robotic arm 300 may further include a plurality of encoders, e.g., encoders E1-E7, disposed on at least some of the joints of the robotic arm 300. For example, encoder E1 for measuring the angle formed between the base portion 302 and the shoulder portion 304 may be disposed on or adjacent to base joint 303 in the base, encoder E2 for measuring the angle formed between the shoulder portion 304 and the proximal shoulder linkage 306 may be disposed on or adjacent to shoulder joint 318 in the base, encoder E3 for measuring the angular rotation between the proximal shoulder linkage 306 and the distal shoulder linkage 308 may be disposed on or adjacent to joint 320, and encoder E4 for measuring the angle formed between the distal shoulder linkage 308 and elbow linkage 310 may be disposed on or adjacent to joint 320, and the transmission of rotational motion at elbow joint 322 may be transmitted from the base to the elbow joint. 322, encoder E5 may be located on or adjacent to wrist joint 324 for measuring angular rotation between elbow joint 310 and proximal wrist joint 312, encoder E6 may be located on or adjacent to joint 326 for measuring the angle formed between proximal wrist joint 312 and central wrist joint 314, and encoder E7 may be located on or adjacent to joint 328 for measuring the angle formed between distal wrist joint 316 and surgical instrument coupler interface. Alternatively, encoder E4 may be located on or adjacent to elbow joint 322. The encoders may be absolute encoders or other position / angle sensors configured to generate data for accurately determining the position and / or angle of a corresponding linkage at an individual joint and / or the precise position of a surgical instrument coupled to the distal end of the robotic arm 300. Thus, the precise position of each linkage, joint, and distal end of the robot 300 may be determined based on measurements obtained from multiple encoders.Preferably, redundant encoders are placed at each location along the robot arm 300 where an encoder is installed, as described in more detail below, to provide more accurate position data and to detect fault conditions.

[0132] Prior to attachment to a surgical instrument, the robotic arm 300 may be manually manipulated by a user, for example, to position the robotic arm 300 at a desired position for coupling with a surgical instrument. For example, a user may manually manipulate the robotic arm 300 via the wrist portion 311, the actuator 330, and / or the actuator 332. In response to actuation of the actuator 330, the user may manually or automatically rotate the distal shoulder linkage 308, and in response to actuation of the actuator 332, the user may manually manipulate the proximal wrist portion 312. Additionally, the robotic arm 300 may also be manually moved by application of forces directly onto other links and / or joints of the robotic arm 300.

[0133] In some embodiments, in response to a force applied by a user to the robotic arm 300, e.g., at the wrist portion 311, wrist joint 324, elbow joint 310, etc., in a given direction, e.g., in a predetermined amount or pattern, a processor of the collaborative robotic platform may move the stage coupled to the base portion 302 of the robotic arm 300 in the same direction, e.g., via the vertical and horizontal extensions of the stage, e.g., when the user releases the robotic arm 300, until the force applied by the user to the robotic arm 300 is detected by the system to fall below a predetermined threshold. In some embodiments, the system may stop movement of the robotic arm 300 in the same direction as the force applied by the user when the user applies a reaction force to the robotic arm 300, e.g., in a direction opposite to the direction of movement of the robotic arm 300, facilitating setting of the robotic arm 300 relative to the patient.

[0134] For example, a user may apply a force on the wrist portion 311 that exceeds a predetermined force threshold in a first direction, which causes the stage of the platform 100 to move the robotic arm 300 in that same direction until the user stops the movement of the wrist portion 311, causing the system to stop the movement of the stage of the platform 100, for example, by releasing the robotic arm 300 or by applying a counter force to the robotic arm 300. Thus, movement of the distal end of the robotic arm, e.g., the wrist portion 311, wrist joint 324, elbow joint 310, etc., can serve as input for generated movement of the robotic arm in a particular direction via the stage coupled thereto. Such automated movement of the stage of the platform 100 in response to a force applied by a user to the distal end of the robotic arm 300 may be limited to when the system is in a predefined operational mode, which may be entered during setup and / or during a surgical procedure, for example, in response to actuation in the GUI 110 and / or via voice control.

[0135] Depending on the attachment to the surgical instrument, the robotic arm 300 may still be manually manipulated by a user by applying a force, for example, one or more linear forces and / or one or more torques, directly to the robotic arm 300. However, during a laparoscopic procedure, the operator preferably manipulates the robotic arm 300 solely through the handle of a surgical instrument, which applies a force / torque to the distal end of the robotic arm 300 and thus to the connections and joints of the robotic arm 300. As the operator applies a force to a surgical instrument attached to the robotic arm 300, thereby causing movement of the surgical instrument, the robotic arm 300 will move in response to the movement of the surgical instrument, providing the operator with the ability to freely move the surgical instrument relative to the patient. As described in further detail below, the robotic arm 300 may apply impedance as the operator moves the surgical instrument to account for the weight of the surgical instrument and the robotic arm 300 itself, e.g., gravity compensation, thereby making it easier for the operator to move the instrument despite gravity and / or inertial forces being imposed on the robotic arm and / or surgical instrument. As will be understood by one skilled in the art, the robotic arm 300 may include fewer or more articulating joints and a corresponding number of motors and encoders / sensors than those shown in FIG. 3 .

[0136] 4C , a close-up view of the coupling mechanism of coupler interface 400 and coupler body 500 is provided. Coupler interface 400 may be coupled to the distal end of distal wrist linkage 316 using any suitable fastener or connector, such as magnets, screws, pins, clamps, welds, adhesive, rivets, and / or any other suitable fastener or any combination of the foregoing. As shown in FIG. 4C , coupler interface 400 may be rotatably coupled to the distal end of distal wrist portion 316 using fastener 410, which may be threaded or have other features that allow coupler interface 400 to be selectively attached to distal wrist portion 316. Fastener 410 may be coupled to an insert element 408 having an opening for receiving fastener 410, positioned at or within the distal end of distal wrist portion 316. In some embodiments, the fastener 410 may be a pin or may have other features such as a ball, latch, or the like that allow the fastener 410 to selectively couple with the distal wrist portion 316.

[0137] A coupler body 500, which may have an opening 514 sized and shaped to slidably and releasably receive an elongated shaft of a surgical instrument therethrough, may be removably coupled to the coupler interface 400. For example, the coupler body 500 may be removably coupled to the coupler body 500 via a magnetic connection, thereby facilitating efficient attachment and detachment between the coupler body 500 and the coupler interface 400, for example, by overcoming the magnetic coupling forces between the coupler body 500 and the coupler interface 400. Thus, as shown in FIG. 4C , the coupler body 500 may have one or more magnets 506 that, in the assembled state, contact the surface of the coupler interface 400 and extend away from the surface of the coupler body 500. Alternatively, in embodiments without a coupler interface, the magnets 506 may directly contact the distal end of the distal wrist portion 316. Thus, the distal end of the coupler interface 400 or the distal wrist portion 316 may have an iron-based base component configured to receive and magnetically couple with the magnet 506 of the coupler body 500 such that the coupler body 500 may be removably coupled with the coupler interface 500 and / or the distal end of the distal wrist portion 316.

[0138] 4D illustrates a surgical instrument coupler interface 400. As shown in FIG. 4D, coupler interface 400 may have a recessed portion 404 that is sized and shaped to receive a complementary geometric shape of coupler body 500, defined by protuberance 402. Thus, when the complementary geometric shape of coupler body 500 is received within recessed portion 404 in the assembled state, rotational movement of coupler body 500 relative to coupler interface 400 may be limited or otherwise prevented. Additionally, the coupler interface 400 may have one or more recesses or depressions 406 sized and shaped to receive one or more magnets 506 therein. The coupler interface 400 may have a ferrous base component or magnet within the recess 406 to magnetically couple with the magnet 506. For example, the magnet within the recess 406 may have a south magnetic pole and the magnet 506 may have a north magnetic pole, or vice versa. Furthermore, the polarity of the magnets can ensure proper coupling orientation. The recess 406 may be sized and shaped to limit or otherwise prevent movement between the coupler body 500 and the coupler interface 400 in any direction, radial or normal to the axial (e.g., longitudinal) centerline of the magnet 506, when the coupler body 500 is assembled with the coupler interface 400. As will be understood by one skilled in the art, the coupler interface 400 may have fewer or more than two recesses 406, so that the coupler body 500 will have a corresponding amount of magnets.

[0139] 5A and 5B, a coupler body 500 is provided. As shown in FIG. 5A, the coupler body 500 may have one or more magnets 506 disposed on a portion 502 having a complementary geometry to the recessed portion 404 of the coupler interface 400, as described above, to facilitate alignment between the coupler body 500 and the coupler interface 400. In addition, the coupler body 500 may have one or more grooves 504 sized and shaped to engage with complementary protuberances 402 of the coupler interface 400. The grooves 504 and the protuberances 402 may interact to assist in alignment of the coupler body 500 and the coupler interface 400 by limiting or otherwise preventing movement between the coupler body 500 and the coupler interface 400 in at least two directions, D1 and D2, as shown in FIG. 4C. Thus, in the assembled state, the coupler body 500 can be prevented from moving in any axial direction relative to the coupler interface 400 .

[0140] 5A and 5B, coupler body 500 may have a first portion 508 and a second portion 510. First portion 508 may be coupled to or integrally formed with second portion 510 via hinge 512, which may be formed from the same material as first and second portions 508, 510 and / or may be an integral hinge formed integrally with first and second portions 508, 510, such that second portion 510 may be moved or rotated relative to first portion 508 to expand (increase in size) or contract (decrease in size) an opening 514 defined by first portion 508 and second portion 510. First portion 508 and second portion 510 may form a clamp that may constrict around an elongated shaft of a surgical instrument positioned within opening 514 as a screw 516, e.g., a thumbscrew, is tightened to couple instrument 112 and coupler body 141. Thus, coupler body 500 may transition between a first, unlocked / open state or position and a second, locked / closed state or position.

[0141] The diameter of opening 514 may be selected based on the surgical instrument to be coupled to coupler body 500. For example, a coupler body may be selected from a plurality of coupler bodies, each having an opening sized and shaped to receive the elongated shaft of a specific surgical instrument having a predefined elongated shaft diameter, such as surgical instruments used for orthopedic and trauma surgery (OTS), laparoscopic or other surgical instruments, including needle holders, clamps, scissors, etc. Coupler body 500 may be coupled with a surgical instrument at any desired axial location on the surgical instrument.

[0142] 4C , coupler body 500 may include a recess 520 extending through second portion 510 and a recess 522 extending through at least a portion of first portion 508. Recess 520 is aligned with recess 522 for receiving locking portion 518 of screw 516. For example, locking portion 518 may have a male-threaded surface, and recesses 520, 522 may have female-threaded surfaces and engage with locking portion 518. Screw 516 may be loosened by hand to open or expand opening 514 so that a surgical instrument may be removed, repositioned, rotated, and / or slid, etc. Once the coupler body 500 is coupled to a surgical instrument, for example via threads 516, the coupler body 500 and the surgical instrument coupled to the coupler body 500 may be removably coupled to the coupler interface 400 via the magnets 506.

[0143] The opening 514 may be defined by a first semicircular cutout in the first portion 508 of the coupler body 500 and a second semicircular cutout in the second portion 510, thereby engaging the circular outer surface of the elongated shaft of a surgical instrument. The opening 514 may include, for example, a rubber pad, sheet, bump, O-ring, protrusion, or other component or feature configured to contact and grip the outer surface of the elongated shaft of the surgical instrument. For example, the rubber material may be silicone rubber or any other suitable type of rubber. Thus, once the coupler body 500 is coupled to the surgical instrument, for example, by locking the threads 516, the surgical instrument may be blocked or otherwise prevented from moving at least axially, e.g., along the longitudinal axis of the surgical instrument, or in some embodiments, from moving axially and rotationally relative to the coupler body 500, in the locked state. Preferably, a surgical instrument coupled to coupler body 500 can be freely rotated by an operator relative to coupler body 500, while axial movement of the surgical instrument relative to coupler body 500 is blocked or otherwise prevented in the secured condition. For example, the frictional force between the outer surface of the elongate shaft of the surgical instrument and the inner surface of coupler body 500 defining opening 514 may be selected such that, in the secured condition, rotation of the surgical instrument relative to coupler body 500 requires less force than axial movement of the surgical instrument relative to coupler body 500. Thus, coupler 500 may be configured to account for diameter and surface variations of surgical instruments (including variations in the coefficient of friction of the surfaces).

[0144] In some embodiments, a surgical instrument may be moved axially relative to the coupler body 500 in response to the application of at least a threshold force on the surgical instrument against the coupler body 500, or in response to an actuation of a release or change in state of the coupler body 500. For example, such actuation may be achieved by, for example, depressing a button, loosening a locking screw such as the locking screw 516 or other connector, moving a dial, or otherwise changing the coupler body 500 and / or the coupler interface 400 from a second, locked state to a first, unlocked state. Thus, a surgical instrument may be axially repositioned relative to the coupler body 500 by loosening the screw 516 or other manual fastener or fastening mechanism such as a clamp in the coupler body 500, repositioning the surgical instrument to a desired axial position, and retightening the screw 516 or other manual fastener or fastening mechanism. The coupler body 500 may be disposable or, alternatively, sterilizable so that it can be sterilized between surgical procedures.

[0145] As explained above, the diameter of the coupler body opening may be selected based on the surgical instrument to be coupled to the coupler body. Most commonly used laparoscopic surgical instruments have predefined and known elongated shaft diameters, and therefore, multiple coupler bodies may be provided, each having an opening sized and shaped to receive and engage a specific surgical instrument. For example, FIG. 6A illustrates a coupler body 600 having an opening 614 sized and shaped to receive a 5 mm diameter surgical instrument, e.g., retractor device 12. Coupler body 600 may be constructed similarly to coupler body 500. For example, coupler body 600 may include a first portion 608 coupled to a second portion 610 via a hinge portion 612, and recesses 620, 622 for securely receiving a locking portion 618 of a screw 616. As shown in FIG. 6B, coupler body 600 may receive elongate shaft 12a of retractor 12, for example, from the working end of retractor 12, through opening 614, such that coupler body 600 may be slid over elongate shaft 12a until coupler body 600 engages proximal portion 12b of retractor 12, as shown in FIG. 6C. Preferably, coupler body 600 is coupled to retractor 12 because, when coupler body 600 contacts proximal portion 12b, this point along retractor 12 is fixed, thereby providing a consistent reference point for calculating force measurements, as described in further detail below. Thus, once coupler body 600 is in a desired location along the elongate shaft of retractor 12, for example, adjacent proximal portion 12b, a screw 616 may be coupled to coupler body 600 to secure coupler body 600 to retractor 12. As described above, the coupler body 600 is secured to the retractor 12 such that rotational movement of the retractor 12 relative to the coupler body 600 is permitted while axial movement of the retractor 12 relative to the coupler body 600 is constrained, e.g., the force required to move the retractor 12 relative to the coupler body 600 is much higher than the force required to rotate the retractor 12 relative to the coupler body 600.

[0146] 7A illustrates a coupler body 700 having an opening 714 sized and shaped to receive a 10 mm diameter surgical instrument, e.g., a laparoscopic device 10. The coupler body 700 may be constructed similarly to the coupler body 600. For example, the coupler body 700 may include a first portion 708 coupled to a second portion 710 via a hinge portion 712 and recesses 720, 722 for securely receiving a locking portion 718 of a screw 716. As shown in FIG. 7B, the coupler body 700 may receive the elongated shaft 10 a of the laparoscopic device 10, for example, from the working end of the laparoscope 10, through the opening 714, such that the coupler body 700 can be slid over the elongated shaft 10 a until the coupler body 700 engages the proximal portion 10 b of the laparoscope 10, as shown in FIG. 7C. Preferably, coupler body 700 is coupled to laparoscope 10 so that when coupler body 700 contacts proximal portion 10b, this point along laparoscope 10 is fixed, thereby providing a consistent reference point for calculating force measurements, as described in further detail below. Thus, once coupler body 700 is at a desired location along the elongated shaft of laparoscope 10, for example, adjacent proximal portion 10b, screws 716 may be coupled to coupler body 700 to secure coupler body 700 to laparoscope 10. As explained above, coupler body 700 is secured to laparoscope 10 such that rotational movement of laparoscope 10 relative to coupler body 700 is permitted, while axial movement of laparoscope 10 relative to coupler body 700 is constrained, e.g., the force required to move laparoscope 10 relative to coupler body 700 is much higher than the force required to rotate laparoscope 10 relative to coupler body 700.

[0147] Once an appropriately sized coupler body has been coupled to a selected surgical instrument, the coupler body may be removably coupled to the coupler interface 400 of the robotic arm 300. The coupler body 500 and coupler interface 400 may be configured for one-handed coupling so that an operator may use one hand to couple the coupler body 500, and therefore the surgical instrument coupled thereto, to the coupler interface 400 of the robotic arm 300. Preferably, a surgical drape may be sandwiched or crimped between the coupler body and the coupler interface 400 and draped over the robotic arm 300 to maintain the sterility of the surgical space and prevent contact with non-sterile components of the robotic arm 300. Thus, a sterile drape can pass continuously (e.g., without holes, slits, or any other type of opening) between the coupler body and the coupler interface such that the coupler body is on a first side of the sterile drape and the coupler interface, robotic arm 300, and / or other components of system 200 are on the other side of the sterile drape. In some embodiments, the coupler body may be integrated with the surgical drape. Additionally or alternatively, the surgical drape may include an adapter integrated therewith such that coupler body 500 can be coupled to coupler interface 400 via the adapter; for example, the adapter may be positioned between coupler body 500 and coupler interface 400.

[0148] 8A and 8B, the robotic arm 300 may be positioned in a surgical drape ready configuration. As shown in FIG. 8A, the robotic arm 300 may be extended such that the wrist portion 311, elbow joint 310, and shoulder joint 305 extend away from the base shoulder portion 304, allowing a surgical / sterile drape to be draped over each component of the robotic arm 300. 8B, when there are two robotic arms, e.g., robotic arm 300a and robotic arm 300b, robotic arms 300a and 300b may be angled away from each other, for example, by rotating shoulder portion 304a relative to base portion 302a of robotic arm 300a and rotating shoulder portion 304b relative to base portion 302b of robotic arm 300b so that wrist portion 311a, elbow link 310a, and shoulder link 305a extend away from wrist portion 311b, elbow link 310b, and shoulder link 305b. This configuration allows for efficient and accessible draping of the individual robotic arms with surgical / sterile drapes. Additionally, in the extended position, the robotic arm may be outside the virtual haptic boundary such that the robotic arm is in haptic mode and a high level of impedance is applied to the robotic arm, thereby making the robotic arm's movement more viscoelastic and making it easier for an operator to drape the robotic arm but provide movement thereto as needed. For example, FIG. 9A illustrates a single robotic arm 300 draped with a sterile drape 800, and FIG. 9B illustrates robotic arms 300 a, 300 b draped with sterile drapes 800 a, 800 b, respectively.

[0149] The sterile drape 800 may be completely closed at its end portions. In some embodiments, the sterile drape 800 may have openings (which may optionally have sterile seals or interfaces) in its distal portion through which portions of the robotic arm 300, coupler interface 400, coupler body 500, and / or surgical instruments may pass. A drape having sealed end portions without any openings and without sealing along its length may provide a better sterile barrier for the system 200. Thus, all of the robotic arm 300 may be located inside and / or completely enclosed within the sterile drape 800, except for an opening at the proximal end of the sterile drape 800, e.g., near the base of the robotic arm 300. In some embodiments, the coupler body 500 and coupler interface 400 may have electrical connectors to create an electronic connection between the robotic arm 300 and the surgical instruments. Thus, electrical signals may be transmitted through sterile drape 800. Alternatively, sterile drape 800 may include openings so that electrical wires or other components may pass through the openings to provide wired communication channels to electrical components, which may include, for example, memory chips for calibration, radio frequency probes for ablation, cameras, and other electronic components. Surgical instruments and coupler bodies may instead be passive or non-electronic, so that electrical wires do not need to pass through sterile drape 800.

[0150] 10A-10D , rotation of the distal shoulder link 308 relative to the proximal shoulder link 306 of the shoulder link 305 is provided. As explained above, axis Q3 may be a “setup” axis such that the distal shoulder link 308 may be rotated relative to the proximal shoulder link 306 in response to actuation of actuator 330 during a set up phase of the robotic arm 300, for example, prior to operation of the robotic arm 300 in a surgical procedure. As shown in FIG. 10A , the shoulder portion 304 may optionally first be rotated relative to the base portion 302 to a desired position, thereby causing rotation of all links distal to the proximal shoulder link 306 that are coupled to the shoulder portion 304 and rotating relative to the base portion 302 to provide sufficient space for rotation of the robotic arm 300 about joint 320. Additionally, as shown in FIG. 10A , the wrist portion 311 may be at least partially extended away from the base portion 302 upon rotation of the robotic arm 300 about joint 320 to avoid collision with any components of the robotic arm 300. As shown in FIG. 10B , an actuator 330 must be actuated to allow rotation of the distal shoulder link 308 relative to the proximal shoulder link 306 at joint 320. As explained above, the actuator 330 may be constructed as described in U.S. patent application Ser. No. 18 / 331,060. FIG. 10C illustrates the robotic arm 300 in a desired location for a specific laparoscopic procedure upon rotation of the distal shoulder link 308 relative to the proximal shoulder link 306. FIG. 10D illustrates the robotic arm 300a in a desired location upon rotation of the distal shoulder link 308a relative to the proximal shoulder link 306a, as compared to the robotic arm 300b. In some embodiments, the joint 320 may be operably coupled to a motor such that the distal shoulder link 308 may be automatically rotated relative to the proximal shoulder link 306, as described in further detail with respect to FIG. 56.

[0151] 11A and 11B, an exemplary collaborative robotic surgical system having an optical scanner is provided. As shown in FIG. 11A, the system may be constructed similarly to system 200 of FIG. 2 and include multiple robotic arms, e.g., robotic arm 300a and robotic arm 300b. As described above, while only two robotic arms are shown in FIG. 11A, fewer or more robotic arms may be used in conjunction with the optical scanner 1100. In addition, the system may include one or more optical scanners 1100, such as a LiDAR scanner or other suitable optical scanning device, such as an RGBD camera or sensor, an RGB camera with machine learning, a time-of-flight depth camera, structured light, multiple projection cameras, a stereoscopic camera, an ultrasonic sensor, a laser scanner, other types of coordinate measurement area scanners, or any combination of the foregoing. For example, the LiDAR camera / scanner may be capable of recording both color (RGB) and depth (D) of the surgical field and may include, for example, an Intel RealSense LiDAR Camera L515 or Intel RealSense Depth Camera D435i (commercially available from Intel, Santa Clara, California) or other LiDAR or depth camera having similar or suitable specifications, including, but not limited to, any of the following specifications: (i) range: 25 cm to 500 cm; depth accuracy: 5 mm or approximately 5 mm; depth field of view: 70 x 55 or approximately 70 x 55 (degrees); depth output resolution: 1,024 x 768 pixels or approximately 1,024 x 768 pixels; depth / RGB frame rate: 30 frames per second; RGB frame resolution: 1,920 x 1,080; and / or RGB field of view: 70 x 43 degrees or approximately 70 x 43 degrees. The LiDAR or optical scanner may further include both 1 / 4-20 UNC thread or 2×M3 thread mounting points. As will be understood by one skilled in the art, the optical scanner 1100 may be used in other collaborative robotic surgical systems described herein, such as system 200 or any variations thereof.

[0152] 11A , the platform supporting the robotic arms 300 a, 300 b may support the optical scanner 1100 and any other electronics, wiring, or other components of the system such that the optical scanner 1100 is mounted in a fixed location relative to other objects in the surgical space, and the position and orientation of the optical scanner 1100 may be known or determined relative to the global coordinate system of the system, and thus the robotic arms. This allows all data streams to be converted to a single coordinate system for development purposes. For example, the optical scanner 1100 may be supported on a rod or shaft, such as rod 1102, which may have an adjustable height or otherwise be adjustable in any direction, e.g., up / down, left / right, toward / away from the patient, allowing the optical scanner 1100 to obtain an optimal view or position relative to other components of the system, e.g., the robotic arms 300 a, 300 b, the surgical instruments attached thereto, the surgeon, and / or a surgical assistant. Additionally, telemetry data captured by the optical scanner 1100, for example, showing the movements of the surgeon's hands, other body parts, the patient bed, trocars, surgical instruments, and other components of the system, may be recorded to provide a rich and detailed data set describing the precise movements and forces applied by the surgeon throughout the procedure.

[0153] For example, the acquired data may be used to optimize procedures performed by the system, including, for example, automatic servo control (i.e., movement) of one or more portions of the robotic arm 300. By tracking a surgeon's tendency to keep a tool within a particular region of interest and / or the surgeon's tendency to avoid moving a tool into a particular region of interest, the system may optimize the automatic servo control algorithm to provide more stability within the particular region of interest. In addition, the acquired data may be used to optimize procedures performed by the system, including, for example, automatic recentering of the field of view of the system's optical scanning device. For example, if the system detects that the surgeon has moved outside of the field of view or predicts that the surgeon may move outside of the field of view, the system may cause the optical scanning device, e.g., the robotic arm supporting the laparoscope, to automatically adjust the laparoscope to track the desired location on the image as the surgeon performs the desired procedure. This behavior may be surgeon-specific and require understanding of the particular surgeon's preferences for the operating region of interest. Thus, the system may control the robotic arm according to the specific operating requirements and / or preferences of a particular surgeon. Additionally, if the system detects that the robot arm has been in an extended position for a period of time that exceeds a predetermined threshold, the system may cause a stage coupled to a base portion of the robot arm to move the robot arm in a manner that facilitates extension of the robot arm, thereby providing additional range for extension of the robot arm by a user.

[0154] 11C , another exemplary collaborative robotic surgical system having multiple optical sensors is provided. As shown in FIG. 11C , system 200 has multiple robotic arms, e.g., robotic arm 300a and robotic arm 300b, supported by platform 100 having multiple wheels for providing mobility to platform 100. As described above, each of the multiple wheels may include a braking mechanism that may be engaged and activated to prevent movement of platform 100. For example, the braking mechanism may be operably coupled to a controller of system 200. Furthermore, system 200 may include multiple optical sensors, e.g., optical scanners 1100a, 1100b, and 1100c, disposed on platform 100. For example, optical scanner 1100a may be disposed on the top of platform 100, as described above with respect to optical scanner 1100 of FIG. 11A , and optical scanners 1100b and 1100c may be disposed on the sides of platform 100. Additionally or alternatively, one or more optical scanners may be positioned beneath the platform 100. The optical scanners 1100a, 1100b, and 1100c are configured to capture depth data. For example, the optical scanners 1100a, 1100b, and 1100c may be, for example, depth cameras, stereoscopic RGB cameras, LiDAR devices, and / or electromagnetic, capacitive, or infrared proximity sensors, etc.

[0155] The depth data generated by the multiple optical sensors may be used by a controller of system 200 to generate, in real time, a virtual map, e.g., a “bird's-eye view,” of the area surrounding platform 100, e.g., within an operating room. For example, the virtual map may illustrate the operating room from a top-down perspective. Further, as shown in FIG. 11C , the virtual map may include a graphical representation of platform 100 (including robotic arms 300 a, 300 b) and one or more objects, e.g., patient table PT, and / or one or more persons, e.g., operator O, person P1, and person P2, within the area surrounding platform 100. Specifically, the virtual map may graphically illustrate the proximity between platform 100 and one or more objects / persons as, for example, platform 100 is moved through the operating room by operator O. The controller may cause the display 110 to display a virtual map such that the operator O may view the virtual map on the display 110 in real time as the operator O moves the platform 100 through the operating room. Thus, the operator O may see objects and / or people in the area surrounding the platform 100 that they would not otherwise be able to see with their own eyes, for example, due to the platform 100 and / or the robotic arms 300 a, 300 b obstructing the operator O's view, and avoid collisions between the platform 100 and / or the robotic arms 300 a, 300 b and objects / people in the operating room. Additionally, the controller may cause the display 110 to display an alert, e.g., a visual or audible alert, when the virtual map indicates that the platform 100 and / or the robotic arms 300 a, 300 b are approaching or within a predetermined distance of one or more objects / people in the operating room.

[0156] In some embodiments, the controller may cause the display 110 to display only the virtual map while the platform 100 is being moved within the operating room. For example, the platform 100 may include one or more actuators, e.g., buttons, levers, or handlebars, that may be operably coupled to braking mechanisms of the wheels of the platform 100 such that, upon actuation of the actuator, the braking mechanisms are disengaged, thus allowing mobility of the platform 100. Thus, when the actuator is not actuated, the braking mechanisms are engaged, such that mobility of the platform 100 is prevented. Thus, upon actuation of the actuator, the controller may automatically cause the display 110 to display the virtual map so that the operator O may view the area surrounding the platform 100 before, during, or after movement of the platform 100 while the braking mechanisms are disengaged. Once the actuator is released, such that the braking mechanisms are re-engaged, the display 110 may stop displaying the virtual map. In some embodiments, when the virtual map indicates that platform 100 and / or robotic arms 300a, 300b are approaching or within a predetermined distance of one or more objects / persons in the operating room, the controller may override actuation of the actuators by the operator and re-engage the braking mechanism, thereby preventing further movement of platform 100. Thus, the actuators may need to be released and re-actuated by the operator to disengage the braking mechanism and allow further movement of platform 100.

[0157] Additionally, the system may process color and / or depth data obtained from the optical scanners 1100a, 1100b, and / or 1100c to identify objects in the operating room, such as a patient bed or trocar, as well as planes associated with the identified objects. Using knowledge of the location of the platform 100 and the robotic arms 300a, 300b relative to the identified objects, the system may automatically move the robotic arms 300a, 300b (or stop the movement of the robotic arms 300a, 300b) during setup to avoid collision with the identified objects, for example, when the robotic arms 300a, 300b approach within a predetermined distance threshold. Additionally, the system may generate and emit audible alerts, for example, indicating the proximity of the platform 100's stage and / or the robotic arms 300a, 300b to the identified objects. For example, the audible alert may change amplitude and / or frequency as the distance between the stage and / or robotic arms 300a, 300b of the platform 100 and the identified object decreases, as perceived by the system based on depth data.

[0158] Additionally, using knowledge of the location of the platform 100 and the robotic arms 300 a, 300 b relative to the patient and trocars, combined with knowledge of where the robotic arms 300 a, 300 b will be positioned relative to the patient and trocars for a given surgical procedure, the system may automatically position the robotic arms 300 a, 300 b in a set configuration relative to the patient and trocars for a given surgical procedure. For example, the system may automatically position the distal ends of the robotic arms adjacent the trocars and further arrange the robotic arms, e.g., via motorized joints of the robotic arms, in a predetermined configuration preferred for the given surgical procedure.

[0159] Additionally, using knowledge of the location of platform 100 and robotic arms 300a, 300b relative to the trocar, if the system detects that the position of the patient bed, and therefore the trocar, is changing, for example, via a user adjustment, the system may automatically adjust the alignment of the robotic arms to accommodate the patient bed movement and maintain the relative position between the distal end of the robotic arms and the trocar. In some embodiments, in response to detecting patient bed movement, the system may automatically move the robotic arms and retract surgical instruments coupled thereto into the trocar prior to automatically adjusting the alignment of the robotic arms so that the distal ends of the surgical instruments are positioned within the trocar and away from anatomy within the patient, maintaining the relative position between the distal end of the robotic arms and the trocar.

[0160] FIG. 12 illustrates the system with optical scanner 1100 operating during a laparoscopic procedure. As shown in FIG. 12, an optional additional optical scanner, e.g., camera 1200, may be utilized to provide additional viewpoints for monitoring and analysis, e.g., redundant measurements of the movement of instruments held by the robotic arms, and / or to provide a video stream of the surgical scene, e.g., via streaming. As shown in FIG. 12, the system may include two robotic arms, e.g., robotic arms 300a and 300b, such that robotic arm 300a holds laparoscope 10 in a fixed position relative to the patient while the surgeon operates and manipulates retractor 12, which is coupled to the distal end of robotic arm 300b. Additionally, robotic arms 300a and 300b may be draped with sterile drapes 800a and 800b, respectively, during the surgical procedure. As described above, the surgeon can freely manipulate the retractor 12 while the retractor 12 is coupled to the robotic arm 300b, thereby causing movement of the robotic arm 300b due to the surgeon's movement of the retractor 12, and while the robotic arm 300b takes into account the weight of the retractor 12 and the robotic arm 300b. During a surgical procedure, the optical scanner 1100 may be used to monitor the identification, location, orientation, and / or movement of a surgical instrument, e.g., the laparoscope 10, coupled to the robotic arm 300a, the identification, location, orientation, and / or movement of a surgical instrument, e.g., the retractor 12, coupled to the robotic arm 300b, and whether any of the surgical instruments have been detached from the respective robotic arms, either intentionally or unintentionally. Additionally, the optical scanner 1100 may be used to monitor the identification, location, orientation, and / or movement / displacement of any of the trocars Tr to ensure proper alignment of the robotic arms and / or surgical instruments with the respective trocars. The system may be used in surgical procedures having one, two, three, four, or more trocars, depending on the surgical procedure intended to be performed by the system.

[0161] 13A and 13B illustrate example data produced by the optical scanner 1100. For example, FIG. 13A illustrates image data captured by the optical scanner 1100, and FIG. 13B illustrates a depth map of at least some objects in the surgical space generated from the data captured by the optical scanner 1100. Specifically, the optical scanner 1100 may create a depth map, e.g., a point cloud, in which the value of each pixel is associated with a distance from the optical scanner 1100. For example, the difference between pixels for a first object (such as a first surgical instrument) and a second object (e.g., a trocar) would allow the system to calculate the distance between the surgical instrument and the trocar. Furthermore, the difference between pixels for the first object (such as a first surgical instrument) at a first time point and the first object at a second time point would allow the system to calculate whether the first object moved, the trajectory of the movement, the speed of the movement, and / or other parameters associated with the changed position of the first object.

[0162] 13A and 13B, a surgeon S manipulates surgical tools and / or draped and undraped robotic arms (DA and UA) positioned relative to an insufflated abdomen (A). As described above, data streams from the robotic arms, camera feeds from the laparoscope, data obtained from the optical scanner 1100, and, optionally, data captured from one or more imaging devices located on structures adjacent to the robotic arms, walls, ceilings, or other structures within the operating room, may be recorded, stored, and used individually or in combination to understand and control the surgical system and its procedures. The aforementioned components, devices, and combinations thereof are collectively referred to herein as optical scanners or optical scanning devices.

[0163] For example, the system may measure and record any of the following within the system's coordinate space: the movement of handheld surgical instruments (attached to a robotic arm or separate therefrom) operated by the surgeon, the presence or absence of other surgical staff (e.g., instrument nurses, circulating nurses, anesthesiologists, etc.), the height and angular orientation of the surgical table, the patient position and volume on the surgical table, the presence or absence of drapes on the patient, the presence or absence of trocar ports and their position and orientation if present, gestures made by the surgical staff, the tasks being performed by the surgical staff, the interaction of the surgical staff with the system, surgical instrument identification, attachment or detachment "actions" of surgical instruments with the system, position and orientation tracking of specific features of the surgical instrument (e.g., camera head, couplers, fiducial markers, etc.) relative to the system, measurements of motion profiles or specific features within the scene that allow phases of the surgery to be identified, and the position, orientation, identification, and / or movement of any other instruments, features, and / or components of the system or used by the surgical team.

[0164] The system may combine the measurements and / or other data described above with any other telemetry data from the system and / or video data from the laparoscope to provide a comprehensive data set that can be used to improve the overall usability, functionality, and safety of the collaborative robotic-assisted surgical system described herein. For example, as the system is configured to begin a procedure, the optical scanner 1100 may detect the height and orientation of the surgical table. This information may enable the system to automatically configure the degrees of freedom of the platform 100 supporting the robotic arm 300 to the desired or correct position relative to the surgical table. Specifically, the optical scanner 1100 may be used to ensure that the height of the platform 100 is optimally positioned to ensure that the robotic arm 300 overlaps with the intended surgical workspace. Additionally, as described above, the system may automatically reconfigure the degrees of freedom of the platform 100 as well as the alignment of the robotic arm 300 in response to movement of the surgical table, and thus the trocar, to maintain the relative position between the distal end of the robotic arm and the trocar.

[0165] Additionally, based on data acquired by the optical scanner 1100, the system may alert surgical staff to potential collisions between the system and other pieces of capital equipment in the operating room, such as a surgical table, laparoscope tower, camera boom, etc., and members of the surgical staff, e.g., an inadvertent bump by a staff member (either during setup or during surgery). The system may use this information to recommend repositioning the platform 100 and / or other components of the system, the surgical table, and / or the patient, and / or prevent the robotic arm from switching into cooperative operation mode as a result of a force applied to the robotic arm due to a collision with a staff member, even if the force exceeds a predetermined force threshold for the robotic arm.

[0166] Additionally, data acquired from the optical scanner 1100 may be used to monitor the progress of setup for a surgical procedure and, combined with the known state of the system, may inform remote hospital staff (e.g., a surgeon) of the overall readiness to begin the procedure. Such progress steps may include confirmation of (i) the patient on the table, (ii) the patient draped, (iii) the sterile instruments available, (iv) the robotic arm draped, (v) the trocar port to be inserted, and (vi) the instruments (e.g., laparoscope and retractor) attached to the robotic arm of the system. For example, data acquired from the optical scanner 1100 may include detected gestures indicating the system state (e.g., the system is draped), readiness to begin the procedure, etc., and may further be used to prepare the system for the attachment or detachment of surgical instruments.

[0167] Additionally, the optical scanner 1100 may identify the specific surgeon performing the procedure, such that the system may use the surgeon's identity to load a system profile associated with the particular surgeon into the system. The system profile may include information related to the surgeon's operating parameters and / or preferences, the surgeon's patient list with patient-specific parameters, desired or required algorithm sensitivities for the surgeon, positioning freedom of the support platform, etc. Examples of algorithm sensitivities that may be surgeon-specific include adapting / adjusting the force required to transition from a passive mode to a collaborative manipulation mode (e.g., low to high force), adapting / adjusting the viscoelasticity felt by the surgeon when collaboratively manipulating the robotic arm (e.g., low to high viscoelasticity), etc. Furthermore, the surgeon's preferences may include the preferred orientation of the robotic arm 300 with respect to a specific surgical instrument, e.g., the positioning of the articulations and joints of the robotic arm 300 relative to the patient; for example, the preferred orientation may differ between a laparoscope and a retractor.

[0168] In some embodiments, surgeon preferences may be learned based on data from sensors that collect information about past procedures and / or the current procedure, including the surgeon's current posture, surgeon height, surgeon's handedness, and other similar factors. For example, the system may record when users interact with the system and what they are doing with it, so that a dataset may allow surgeon preferences to "learn" and be updated over time. This learning may occur either through traditional algorithmic methods (i.e., trends over time, averages, optical flow, etc.) or through machine learning approaches (classification, discrimination, neural networks, reinforcement learning, etc.). FIG. 24 illustrates a data flow 2400 for updating system configuration based on a user's learned behavior. As shown in FIG. 24, the system may be connected to an online database that may store surgeon profiles and each of several possible data sources, which may include a database of optical sensors, encoders, and / or other sensors, and / or manually entered user inputs. Data sources may be associated with a given surgeon, their preferred robotic arm configuration and operating parameters, and each procedure performed with the system, which may allow for recording and analysis of system configuration and how it varies from procedure to procedure and within a procedure. In the case of machine learning, the collaborative capabilities of the system may be leveraged so that user actions can be used to annotate data and create training datasets.

[0169] Regarding positioning freedom, the height of a surgical table is typically adjusted to accommodate the surgeon's height in some operating rooms. Thus, by detecting the surgeon and loading the surgeon's specific profile, the system may position the platform at a suitable height for the individual surgeon to match the preferred height of the surgical table. In addition, the horizontal translation of the robotic arm may depend on the patient's size. Thus, by accessing a patient list, the system may adjust the arm position based on the patient's body mass index ("BMI"). For example, for patients with a high BMI, the system may move the robotic arm away from the operating table, and for patients with a low BMI, the system may move the robotic arm closer to the operating table. Thus, the system allows the surgical team to fine-tune the position of the robotic arm relative to the patient as needed. The system may further be configured to access a hospital medical record database and access the procedure type and any other available medical data (e.g., CT scan images, X-ray images, MRI images, and / or other patient-specific information), which may be used to inform the positioning of the trocar port and the position and orientation of the platform 100 relative to the patient.

[0170] Based on the data captured by the optical scanner 1100, the system may generate virtual models of pieces of capital equipment and / or other objects in the operating room that are within the range of movement of the robotic arm in the same coordinate space as the robotic arm and its associated surgical instruments, so that the virtual models can be stored and monitored, e.g., to detect potential collisions. Additionally, the system may track the position and orientation of objects within the virtual models as they move relative to one another, so that the system may alert a user if the proximity (i.e., the spacing between) of either the virtual model or the object falls below a predefined threshold, e.g., within 50 mm, 75 mm, 30 mm, or less to 100 mm, or more. In some embodiments, the distance threshold may be based on the Euclidean distance between the closest points on two virtual models, the normal distance between two surfaces of the virtual models, or the like. Further, the system may stop or block (e.g., prevent) further movement of the robotic arm, e.g., freeze the robotic arm, if the proximity of either the virtual model or an object, e.g., the robotic arm, reaches or falls below a predefined threshold relative to the laparoscopic tower, or the surface of the surgical table, or other objects in the surgical space. Additionally, the system may freeze the robotic arm if it detects that the proximity between an object, e.g., capital equipment or a member of surgical staff other than the surgeon, moving toward the individual robotic arm reaches or falls below a predefined threshold, thereby preventing inadvertent movement of the robotic arm that might otherwise result from such a collision or inadvertent force, e.g., an inadvertent bump from a member of staff or another piece of capital equipment, etc.

[0171] Additionally, based on the data captured by the optical scanners 1100a, 1100b, 1100c, the system may generate a virtual map with graphical representations of objects and / or people present within a predefined area surrounding the platform and robotic arm in the operating room in the same coordinate space as the platform and robotic arm so that the virtual map can be stored and displayed to a user, for example, to detect potential collisions while the user moves the platform throughout the operating room. Additionally, the system may track the position and orientation of the graphical representations within the virtual map so that the system may alert the user if the proximity between any of the objects and / or people from the platform and / or robotic arm falls within a predetermined threshold, e.g., within 50 mm, 75 mm, 30 mm or less to 100 mm, or more.

[0172] Additionally, based on data captured by the optical scanner 1100, the system may track the motion of handheld surgical instruments that are not coupled to a robotic arm and are controlled directly and independently by the surgeon. For example, the optical scanner 1100 may track clearly defined features on the instrument, fiducial markers attached to the instrument or the surgeon's gloves (e.g., sterile gloves), the connector between the robotic arm and the instrument, the distal tip of the instrument, and / or any other defined location on the instrument. For example, the fiducial markers may include Manus virtual reality gloves (commercially available by Manus, The Netherlands) or other wearables and / or the OptiTrack system (commercially available by Natural Point, Corvallis, Oregon). The following are examples of uses and purposes of motion data: (i) close the control loop between the handheld instrument and the robotic arm holding the camera, thus allowing the surgeon to servo-control (i.e., move) the camera by "pointing" the handheld instrument; (ii) track information and identify phases of the surgical procedure, which can be used independently or in combination with other data streams; (iii) identify the surgeon's handedness; (iv) monitor measurements correlated with the surgeon's experience; (v) identify the tools the surgeon is using and when to change them for other tools; and / or (vi) track the number, location, and orientation of the patient's skin surface and trocar ports. This data and information may also be used and calculated by the system as part of a collaborative control framework. By measuring the true position and orientation of the trocar ports, the system is provided with an additional safety check to ensure that system-level calculations are correct, for example, ensuring that the actual movement of the robotic arm or instrument matches the commanded movement of the robotic arm or instrument in robot-assisted mode.

[0173] Based on the data captured by the optical scanner 1100, the system may further track instruments being used at individual ports, how often instruments are swapped between ports, if the system holds instruments in place while the patient or surgical table is moved (in which case the system may adjust for the movement by changing the robotic arm's operating mode to passive mode and repositioning the robotic arm 300 and / or platform 100), ports with manually held instruments versus instruments coupled to a robotic arm, and / or other conditions or parameters of the operating room or system to monitor and determine if additional trocar ports are added. Knowledge of the location and orientation of the skin surface and trocar ports relative to the robotic arm can facilitate the implementation of a "virtual boundary," as described in more detail below.

[0174] Furthermore, based on image data captured by the laparoscope as well as data acquired by optical scanner 1100, e.g., tracked movement of the distal end of a laparoscope coupled to robotic arm 300, the system may identify the type of laparoscope coupled to robotic arm 300. For example, laparoscopes commonly used during laparoscopic procedures include flat-tipped laparoscopes and angled-tipped laparoscopes, e.g., laparoscopes with a 30-degree angled tip. The system may determine the laparoscope type currently coupled to robotic arm 300 by comparing image data acquired by optical scanner 1100 of a predefined pattern of laparoscope movement, e.g., moving the distal end of the laparoscope in a circular pattern in a plane perpendicular to the longitudinal axis of the laparoscope, with image data acquired by the laparoscope as it is moved in a predefined pattern of movement. For example, with a flat-tipped laparoscope, when the distal end of the laparoscope is moved in a circular pattern in a plane perpendicular to the longitudinal axis of the laparoscope, the image data captured by the laparoscope will move along a circular planar path, and there will be no change in, for example, the depth of field of the laparoscope, whereas with an angled-tipped laparoscope, when the distal end of the laparoscope is moved in a circular pattern in a plane perpendicular to the longitudinal axis of the laparoscope, the image data captured by the laparoscope will observe a change in the depth of field of the laparoscope.

[0175] Additionally, the system may calibrate the new robotic arm based on data acquired by optical scanner 1100, for example, during a surgical procedure, when the current robotic arm is replaced, with or without a tracker at the distal end of the new robotic arm, to ensure the system accurately perceives the kinematics of the new robotic arm. Specifically, the system may calibrate optical scanner 1100 to platform 100 when the new robotic arm is coupled to platform 100, calibrate the new robotic arm to a base portion of the new robotic arm, and calibrate the new robotic arm to platform 100. For example, based on telemetry data acquired by the optical scanner 1100, the system may compare the actual real-time movement of the new robotic arm as captured by the optical scanner 1100 with the expected movement based on commands sent by the system to the new robotic arm, e.g., to execute a preprogrammed routine intended to move the new robotic arm to a specific location, and generate an error measure indicative of the deviation between the actual real-time movement of the new robotic arm and the expected movement of the robotic arm based on the preprogrammed routine. The system may further execute an optimization algorithm to reduce or eliminate the error measure between the actual real-time movement and the expected movement, e.g., until the error measure is below a predetermined threshold. This calibration process may occur after the new robotic arm is coupled to the platform 100, when the system is in a predefined calibration mode, or alternatively, in real time during the surgical procedure.

[0176] Based on data acquired by the optical scanner 1100, e.g., knowledge of the position and orientation of the surgical bed relative to the platform 100 and / or trocar port, the system may automatically position the robotic arm 300 in a preferred configuration relative to the patient, via the motorized joints of the robotic arm 300 and / or the stage of the platform 100, in response to actuation by a user, e.g., via the GUI 110, based on the surgical procedure to be performed, while avoiding collisions between the stage of the platform 300, the robotic arm 300, and objects in the operating room, such as a surgical bed, during setup. Thus, the system may store pre-set robotic arm configurations, e.g., based on surgeon preferences stored for various surgical procedures.

[0177] 14 , components that may be included within a collaboratively manipulated robotic platform 1400 are described. The platform 1400 may include one or more processors 1402, communication circuitry 1404, a power supply 1406, a user interface 1408, and / or memory 1410. One or more electrical components and / or circuits may perform some or all of the roles of the various components described herein. Although described separately, it should be understood that the electrical components need not be separate structural elements. For example, the platform 1400 and communication circuitry 1404 may be embodied within a single chip. Additionally, while the platform 1400 is described as having memory 1410, the memory chip may be provided separately.

[0178] Platform 1400 may contain memory and / or be coupled via one or more buses to read or write information from or to the memory. Memory 1410 may include a processor cache, including a multi-level hierarchical cache, in which different levels have different capacities and access speeds. Memory may also include random access memory (RAM), other volatile storage devices, or nonvolatile storage devices. Memory 1410 may be RAM, ROM, flash, other volatile or nonvolatile storage devices, or other known memory, or some combination thereof, and preferably includes storage in which data may be selectively stored. For example, storage devices may include, for example, hard drives, optical disks, flash memory, and Zip drives. Programmable instructions may be stored on memory 1410 and may execute algorithms, for example, to calculate desired forces to be applied along robotic arm 300 and / or a surgical instrument coupled thereto, and to apply impedances at individual joints of robotic arm 300 to produce the desired forces.

[0179] Platform 1400 may incorporate processor 1402, which may be comprised of one or more processors, which may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any suitable combination thereof designed to perform the functions described herein. Platform 1400 may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0180] Platform 1400 may execute an operating system (e.g., operating system 1446), such as, for example, Windows, Mac OS, QNX, Unix, or Solaris 5.10, in conjunction with firmware / software stored in memory. Platform 1400 also executes software applications, which are stored in memory. For example, the software may be a program in any suitable programming language known to those skilled in the art, including, for example, C++, PHP, or Java.

[0181] The communications circuitry 1404 may include circuitry that allows the platform 1400 to communicate with image capture devices, such as optical scanners and / or endoscopes. The communications circuitry 1404 may be configured for wired and / or wireless communications, such as via the Internet, a telephone network, a Bluetooth network, and / or a Wifi network, using techniques known in the art. The communications circuitry 1404 may be a communications chip known in the art, such as a Bluetooth chip and / or a Wifi chip. The communications circuitry 1404 allows the platform 1400 to transfer information, such as force measurements on the body wall at the trocar insertion point, locally and / or to a remote location, such as a server.

[0182] The power supply 1406 may provide alternating current or direct current. In direct current embodiments, the power supply may include a suitable battery, such as a replaceable or rechargeable battery, and the device may include circuitry for charging the rechargeable battery and a detachable power cord. The power supply 1406 may be a port to allow the platform 1400 to be plugged into a conventional wall socket, for example, via a cord with an AC / DC power converter and / or a USB port, to power the components within the platform 1400. The power supply 1406 may be operably coupled to an emergency switch such that, upon activation of the emergency switch, power being supplied to components within the platform 1400, including, for example, braking mechanisms disposed on at least some of the joints of the robotic arm 300, is stopped. For example, the braking mechanism may require power to be turned off such that the braking mechanism acts to prevent movement of the robotic arm 300 without power, even when power is not being supplied to the braking mechanism.

[0183] A user interface 1408 may be used to receive input from and / or provide output to a user. For example, the user interface 1408 may include a touchscreen, a display, switches, dials, lights, etc. Thus, the user interface 1408 may display information such as a selected surgical instrument identification and force measurements observed during operation of the robotic arm 300. Additionally, the user interface 1408 may receive user input including adjustments to a predetermined amount of movement or a predetermined dwell period at the handle of a surgical instrument to automatically switch the robotic arm to a passive mode, a predetermined threshold of force to be applied at the handle of a surgical instrument to automatically switch the robotic arm to a collaborative manipulation mode, the location of a predefined force sense barrier, the identification of a surgical instrument coupled to the distal end of the robotic arm, the vertical height of the robotic arm, the horizontal position of the robotic arm, etc., so that the platform 1400 may adjust the information / parameters accordingly. In some embodiments, the user interface 1408 is not present on the platform 1400 but instead is provided on a remote, external computing device that is communicatively connected to the platform 1400 via the communications network 1404.

[0184] Memory 1410, an example of a non-transitory computer-readable medium, may be used to store an operating system (OS) 1446, a surgical instrument identification module 1412, a surgical instrument calibration module 1414, an encoder interface module 1416, a robotic arm position determination module 1418, a trocar position detection module 1420, a force detection module 1422, an impedance calculation module 1424, a motor interface module 1426, an optical scanner interface module 1428, a gesture detection module 1430, a passive mode determination module 1432, a collaborative manipulation mode determination module 1434, a haptic mode determination module 1436, a robot-assisted mode determination module 1438, an obstacle detection module 1440, an indicator interface module 1442, and a fatigue detection module 1444. The modules are provided in the form of computer-executable instructions / algorithms that may be executed by processor 1402 to perform various operations according to the present disclosure.

[0185] For example, during a procedure, the system may continuously invoke the algorithms described herein based on data collected by the system. That data may be collected and / or recorded using any of the components and methods disclosed herein, including, for example, from sensors / encoders within the robot, from an optical scanning device communicating with other components of the robotic system, and / or from manual input by an operator of the system. Thus, the algorithms, data, and configurations of the system may enable a user to coordinate the robotic arm with minimal shock and impact from the weight of the robotic arm and / or surgical instruments coupled thereto, gravity, and other forces that conventional robotic arms cannot compensate for. Some of the parameters of the algorithms described herein may control aspects of the system's behavior, including, for example, the robustness of detected features, sensitivity to false positives, robot control gain, the number of features to track, the dead-zone radius, etc.

[0186] A surgical instrument identification module 1412 may be executed by the processor 1402 to identify the surgical instruments coupled to each of the robotic arms and load the appropriate calibration file into the controller system. For example, a calibration file for each surgical instrument may be stored in a database accessible by the surgical instrument identification module 1412 and may include information associated with the surgical instrument, such as instrument type, weight, center of mass, length, instrument shaft diameter, etc. Thus, once the appropriate calibration file is loaded and the associated surgical instrument is coupled to the robotic arm 300, the system will automatically account for the mass of the surgical instrument based on the data in the calibration file, for example, compensating for the force of gravity acting on the surgical instrument when it is attached to the robotic arm 300, so that the robotic arm 300 can hold the surgical instrument in place after it is coupled to the robotic arm and the operator releases it. For example, the surgical instrument identification module 1412 may identify the surgical instrument based on user input via the user interface 1408, e.g., an operator may select the surgical instrument from a database of surgical instruments stored in the memory 1410.

[0187] In some embodiments, the surgical instrument identification module 1412 may automatically identify surgical instruments coupled to the robotic arm via the coupler body and coupler interface using, for example, an RFID transmitter chip and reader or receiver (e.g., placing an RFID sticker or transmitter on the surgical instrument that can transmit information about the surgical instrument to a receiver in the system), a near-field communication (“NFC”) device such as a near-field magnetic induction communication device, a barcode and scanner or other optical device, a magnet-based communication system, a reed switch, a Bluetooth transmitter, an optical scanner and instrument weight and / or data gathered from a look-up table, and / or any other feature or mechanism described herein or suitable for identifying surgical instruments. As described above, the coupler body may be selected based on the size and shape of the lumen extending therethrough to accommodate and engage surgical instruments having a known elongated shaft diameter. Thus, the surgical instrument identification module 1412 can automatically identify a surgical instrument based on the coupler body that is coupled to the surgical instrument via the magnetic connection between the coupler body and the coupler interface.

[0188] 44A-50B , surgical instrument identification module 1412 may identify the surgical instrument, e.g., the type of surgical instrument, based on data obtained from a sensor, e.g., a Hall Effect sensor in distal wrist link 316, indicative of the strength of a magnetic field induced by a magnet that is displaced in response to coupling of the surgical instrument with the coupler body coupled to the coupler interface. Based on the strength of the detected magnetic field, the system may determine whether a coupler body is coupled to the coupler interface, whether a surgical instrument is coupled to the coupler body, and the diameter of the shaft of the surgical instrument, e.g., a 5 mm or 10 mm surgical instrument, as described in further detail below.

[0189] The surgical instrument identification module 1412 may further, as described above, move the distal end of the robotic arm 300, when coupled to the laparoscope, in a predefined pattern, for example, in a circular motion, in a plane perpendicular to the longitudinal axis of the laparoscope, and compare image data captured by the laparoscope as the distal end of the laparoscope is moved in the predefined circular pattern with image data acquired by the laparoscope's optical scanner 1100 as the laparoscope is moved in the predefined pattern to identify whether the laparoscope is a flat-tip laparoscope or an angled-tip laparoscope.

[0190] In some embodiments, the surgical instrument identification module 1412 may identify the surgical instrument, e.g., the type of surgical instrument, based on data acquired by the optical scanner 1100 via the optical scanner interface module 1428, described in further detail below. For example, the data may include such data associated with a specific instrument such that the surgical instrument identification module 1412 may compare the measurement data with information contained in a database, identify the instrument, and load the appropriate calibration file into the controller system. Similarly, the surgical instrument identification module 1412 may detect if the instrument has been removed and return the calibration parameters to a default configuration.

[0191] The surgical instrument calibration module 1414 may be executed by the processor 1402 to calibrate a surgical instrument, for example, a surgical instrument that does not currently have an associated calibration file in the database stored in the memory 1410. Thus, when the surgical instrument calibration module 1414 is coupled to the robotic arm 300 and the system is in calibration mode, it may calculate measurements and specifications of the surgical instrument based on force measurements of the robotic arm 300 applied by the surgical instrument via the force detection module 1422, as described in further detail below with respect to FIG. 16 . For example, the surgical instrument calibration module 1414 may generate a calibration file for the surgical instrument that includes information such as the instrument type, weight, center of mass, length, instrument shaft diameter, viscoelastic parameters, etc. of the surgical instrument. At least some of the surgical instrument information in the calibration file, e.g., instrument type, may be provided by user input via the user interface 1408, or, for example, the instrument type, center of mass of the instrument, instrument length, and instrument diameter may be detected by the optical scanner interface module 1428.

[0192] If the surgical instrument calibration module 1414 determines that the recalibration results are consistently different from configurations already loaded into the system, the surgical instrument calibration module 1414 may replace the existing information or add to its list of known tools and automatically load them without any user input. The surgical instrument calibration module 1414 may determine that the calibration coefficients are not appropriate to compensate for gravity, for example, if the surgical instrument, when coupled with the robotic arm, moves solely due to gravity acting on the robotic arm and / or surgical instrument, which may occur when the surgical instrument is positioned entirely outside the patient's body. Furthermore, the surgical instrument calibration module 1414 may automatically update or adjust the calibration coefficients (e.g., forces applied to the joints of the robotic arm) if it determines that the calibration coefficients are not appropriate to compensate for gravity. Thus, the surgical instrument calibration module 1414 may update the calibration coefficients for a particular surgical instrument and store the updated calibration coefficients for the particular surgical instrument in an associated calibration file for future use.

[0193] The encoder interface module 1416 may be executed by the processor 1402 to receive and process formation angle measurement data in real time from multiple encoders of the robot arm 300, e.g., encoders E1-E7. For example, the encoder interface module 1416 may calculate the change over time in the formation angle of a linkage of the robot arm 300 that is rotatably coupled to a given joint associated with an encoder. As described above, the system may include redundant encoders at each joint of the robot arm 300 to ensure safe operation of the robot arm 300. Furthermore, additional encoders may be disposed on the platform 100 to measure the formation angle / position of each robot arm relative to the platform 100, e.g., the vertical and horizontal positions of the robot arm relative to the platform 100. Thus, one encoder may be disposed on the platform 100 to measure the movement of the robot arm along the vertical axis of the platform 100, and another encoder may be disposed on the platform 100 to measure the movement of the robot arm along the horizontal axis of the platform 100.

[0194] The robotic arm position determination module 1418 may be executed by the processor 1402 in real time to determine the position of the robotic arm 300 and, if applicable, a surgical instrument attached thereto in 3D space based on the angle measurement data generated by the encoder interface module 1416. For example, the robotic arm position determination module 1418 may determine the positions of the various linkages and joints of the robotic arm 300 and the position along a surgical instrument coupled to the robotic arm 300. Based on the position data of the robotic arm 300 and / or the surgical instrument, the robotic arm position determination module 1418 may calculate, in real time, the velocity and / or acceleration of the movement of the robotic arm 300 and the surgical instrument attached thereto. For example, by determining the individual velocities of the various joints of the robot arm 300 via encoders associated with each of the various joints, the robot arm position determination module 1418 may determine a resultant velocity of the distal end of the robot arm 300, which may be used by the passive mode determination module 1432 to determine whether the movement of the distal end of the robot arm 300 is within a predetermined threshold for purposes of transitioning the system 200 to passive mode, as described in further detail below.

[0195] The trocar position detection module 1420 may be executed by the processor 1402 to determine the position and / or orientation of one or more trocar ports to be inserted into a patient. The position and / or orientation of the trocar ports may be derived based on data obtained from, for example, inertial measurement units and / or accelerometers, optical scanners, electromechanical tracking devices, linear encoders, sensors and data such as those described above. For example, the position of the trocar port on the patient may be determined using a laser pointing system that may be mounted on one or more of the system's components, such as the wrist portion 311 of the robotic arm, and controlled by the system to point the trocar to an optimal or determined position on the patient's body for insertion. Furthermore, upon insertion of a surgical instrument attached to the robotic arm 300 through a trocar, a virtual line may be continuously established along the longitudinal axis of the surgical instrument, the alignment / orientation of which may be automatically determined in real time as the surgical instrument moves about the trocar point, responsive to attachment of the surgical instrument to the coupler interface 400 via the coupler body via a magnetic connection, as described above. Furthermore, when the surgical instrument is inserted into the trocar port, it is pointed toward the trocar point, and therefore the distal wrist joint 316 is also pointed toward the trocar point, the angle of which may be measured by an encoder associated therewith. Thus, the trocar point may be calculated as the intersection of multiple virtual lines continuously established along the longitudinal axis of the surgical instrument. In this manner, the calculated trocar point will remain fixed relative to the patient as the surgical instrument is manipulated about the trocar port, e.g., rotated or moved in and out of the patient. Additionally, the orientation of the trocar port and its position relative to the robotic arm 300 may be determined based on image data received from one or more optical scanners, for example, a LiDAR camera and / or an RGBD camera.

[0196] Based on the known position of the distal end of the robotic arm 300 from the robotic arm positioning module 1418, plus the known position and / or orientation of the trocar port, the system may maintain the position of the distal end of the robotic arm 300 relative to the trocar point as the robotic arm 300 moves, for example, via its vertical or horizontal adjustment by the platform 100, or as the height of the patient table is adjusted, thereby moving the height of the patient's abdomen, thereby keeping a surgical instrument within the patient's body and coupled to the robotic arm 300 stationary during these external movements. To accomplish this, the known position of the distal end of the robotic arm 300 from the robotic arm positioning module 1418 is calculated within the global frame of the system by adding the position of the platform 100 to the kinematic calculations (e.g., the "forward kinematics" of the robotic arm 300 in the context of a serial-chain robotic manipulator).

[0197] Using a globally known position of the distal end of the robot arm 300, the system can hold that position steady during external movement by applying appropriate forces to the robot arm 300 that minimize the error between its current position and the desired position. Thus, for example, if a surgical instrument coupled to the distal end of the robotic arm 300 is inserted through a trocar port so that the tip of the instrument is inside the patient, and the user adjusts the height of the patient table, the system may apply a force / torque to the robotic arm 300 to reconfigure the robotic arm 300 and / or cause a movement of the stage of the platform 100 to maintain the relative position between the distal end of the robotic arm 300, and thus the surgical instrument, and the trocar port. In some embodiments, the system may retract the distal end of the robotic arm 300 slightly so that the tip of the surgical instrument is positioned within the trocar port and out of contact with anatomical structures within the patient's body prior to reconfiguring the robotic arm 300 and maintaining the relative position between the surgical instrument and the trocar port.

[0198] The force detection module 1422 may be executed by the processor 1402 to detect forces, e.g., body wall forces, applied on the robotic arm 300, e.g., at a joint or linkage of the robotic arm 300, or along a surgical instrument, and on a trocar. For example, the force detection module 1422 may receive, in real time, motor current measurements at each motor, e.g., M1, M2, M3, disposed in the base of the robotic arm 300, which is operably coupled to a joint, e.g., base joint 303, shoulder joint 318, elbow joint 322, wrist joint 332, respectively, of the robotic arm 300. The motor current measurements indicate the amount of force applied to the associated joint. Thus, the force applied to each joint of the robotic arm 300 and the surgical instrument attached thereto may be calculated based on the motor current measurements and position data generated by the robotic arm position determination module 1418 and / or the trocar position detection module 1420.

[0199] Due to the passive axis at the distal end of the robotic arm 300, the force applied by an instrument coupled to the robotic arm on the trocar can generally remain consistent throughout the robotic arm's workspace. The force on the trocar can be affected by the interaction of the distal tip of the instrument with tissue within the body. For example, when a tissue retractor advanced through the trocar engages (e.g., grasps) body tissue or another object inside the body, the force imparted on the end of the instrument from the body tissue or other object can cause a change in the force applied to the trocar. In some aspects, the force on the trocar can be a function of the amount of weight being lifted by the instrument being used.

[0200] The impedance calculation module 1424 may be executed by the processor 1402 to determine the amount of impedance / torque needed to be applied to individual joints of the robotic arm 300 to achieve a desired effect, such as holding the robotic arm 300 in a static position in a passive mode, allowing the robotic arm 300 to move freely while compensating for the gravity of the robotic arm and a surgical instrument attached thereto in a collaborative manipulation mode, applying increased impedance to the robotic arm 300 when the robotic arm 300 and / or a surgical instrument attached thereto is within a predefined virtual haptic barrier in a haptic mode, etc.

[0201] For example, the impedance calculation module 1424 may determine the amount of force required by the robotic arm 300 to achieve a desired effect based on the position data of the robotic arm 300 generated by the robotic arm position determination module 1418 and the trocar position data generated by the trocar position detection module 1420. For example, by determining the position of the distal end of the robotic arm 300 and the entry point of the surgical instrument into the patient, e.g., the trocar position, and with knowledge of one or more instrument parameters, e.g., the mass and center of mass of the surgical instrument stored by the surgical instrument calibration module 1414, the impedance calculation module 1424 may calculate the amount of force required to compensate for the gravity of the surgical instrument (compensation force), as described in further detail below with respect to FIG. 18A. Thus, the amount of compensation force required to compensate for the gravity of the surgical instrument may be converted into a torque to be applied at a joint of the robotic arm 300 by a motor operably coupled to the joint of the robotic arm 300, as indicated, for example, by motor current measurements.

[0202] Furthermore, by determining the position of the distal end of the robotic arm 300 and therefore its change over time due to, for example, external forces applied to the distal end of the robotic arm 300, for example, by tissue held by the working end of a surgical instrument, and using knowledge of one or more instrument parameters, for example, the mass, center of mass, and length of the surgical instrument stored by the surgical instrument calibration module 1414, the impedance calculation module 1424 may calculate the amount of force required to maintain the surgical instrument in a static position (the holding force), as described in further detail below with respect to FIG. 18B. Thus, the amount of holding force required to resist changes in the position of the distal end of the robotic arm 300, plus the amount of compensation force required to compensate for the gravitational force of the surgical instrument, may be converted into torque to be applied at the joints of the robotic arm 300 to maintain the robotic arm 300 in a static position, as indicated, for example, by motor current measurements, by motors operably coupled to the joints of the robotic arm 300. Additionally, the impedance calculation module 1424 and / or the force detection module 1422 may calculate the amount of force applied by the surgical instrument to the patient at the entry point, e.g., the trocar, and the amount of force applied to the working end of the surgical instrument, e.g., the grasper end of the surgical instrument, based on one or more parameters of the surgical instrument, such as the compensation force, the holding force, the mass, center of mass, and length of the surgical instrument, and the distance from the center of mass to the entry point.

[0203] Additionally or alternatively, by determining the force applied on the robot arm 300 via the force detection module 1422 and the position / velocity / acceleration of the distal end of the robot arm 300 in 3D space via the robot arm position determination module 1418, a desired force / impedance to be applied to the robot arm 300 to compensate for the applied force may be calculated, e.g., for gravity compensation or to hold the robot arm 300 in a static position in a passive mode. Thus, the desired force may be converted into a torque to be applied at the joints of the robot arm 300, e.g., by motors operably coupled to the joints of the robot arm 300. For example, a robot Jacobian matrix may be used for this purpose. The Jacobian matrix is ​​a matrix calculated at each given support of the robot arm and relates the velocity at the joint to the velocity at the distal end of the robot arm 300. [ka]

[0204] where V is the velocity vector at the distal end of the robot arm 300, J is its Jacobian matrix, and q dot is the joint velocity, expressed in vector form. Using energy principles and assuming negligible mass for the joints of the robot arm 300 and negligible friction / damping, the power of the system can be determined by multiplying its force and velocity. [ka]

[0205] where F is the generalized force vector at the distal end of the robot 300. Furthermore, the vector manipulation yields: [ka]

[0206] where t denotes the transpose of a matrix so that the forces at the distal end of the robot arm 300 can be converted into torques to be applied at the joints using the Jacobian matrix.

[0207] The motor interface module 1426 may be executed by the processor 1402 to receive motor current readings at each motor, e.g., M1, M2, M3, disposed within the base of the robotic arm 300, and to actuate the individual motors, e.g., by applying a predetermined impedance, to achieve a desired result as described herein and / or to move a joint operably coupled to the individual motor, such as in a robot-assisted mode. In some embodiments, when the joint 320 is operably coupled to a motor, e.g., M4, the motor interface module 1426 may actuate M4 to cause rotation of the distal shoulder link 308 relative to the proximal shoulder link 306, such that the distal shoulder link 308 may be automatically rotated relative to the proximal shoulder link 306, as described in further detail with respect to FIG. 56 .

[0208] The optical scanner interface module 1428 may be executed by the processor 1402 to receive depth data acquired by the optical scanner 1100 and process the depth data, e.g., to detect predefined conditions therein. Additionally, the optical scanner interface module 1428 may generate a depth map indicative of the received depth data, which may be displayed to an operator, e.g., via a monitor. For example, the optical scanner interface module 1428 may map the locations of trocar ports in 3D space so that the mapping of the trocar ports can be communicated to an operator, e.g., via a display or user interface 1408. Based on the depth and / or color data received from the optical scanner 1100, the optical scanner interface module 1428 may determine relative distances between, for example, the stage of the platform 100, the robotic arm 300, any surgical instruments attached thereto, and objects / people in the operating room, such as a surgical table, drapes, etc.

[0209] The optical scanner interface module 1428 may further receive image data from additional optical scanning devices as defined herein, including, for example, endoscopes operably coupled to the system. Additionally, the optical scanner interface module 1428 may receive depth data acquired by optical scanners 1100a, 1100b, 1100c coupled to platform 100, as described above with respect to FIG. 11C , process the depth data, and generate a virtual map of the area surrounding platform 100, which may be displayed to an operator via a monitor, e.g., display 110. For example, the optical scanner interface module 1428 may generate a graphical representation of system 200, including platform 100 and robotic arms 300a, 300b, and any objects and / or people in the area surrounding platform 100, for display within the virtual map.

[0210] The gesture detection module 1430 may be executed by the processor 1402 to detect predefined gesture patterns as user inputs and perform actions associated with the user inputs. The predefined gesture patterns may include, for example, movement of a surgical instrument (whether attached to the robotic arm 300 or not), movement of the robotic arm 300 or other components of the system, such as foot pedals, buttons, etc., and / or movement of an operator in a predefined pattern. For example, back-and-forth movement of a surgical instrument in a first direction (e.g., left / right, up / down, forward / backward, circular) may be associated with a first user input requesting a first action by the system, and / or back-and-forth movement in a second direction different from the first direction (e.g., left / right, up / down, forward / backward, circular) may be associated with a second user input requesting a second action by the system. Similarly, pressing a foot pedal or button operatively coupled to the system in a predefined manner may be associated with a third user input requesting a third action by the system, and repeatedly moving the operator's head back and forth or up and down may be associated with a fourth user input requesting a fourth action by the system. Various predefined gesture patterns associated with different components or operators of the system may be redundant, such that the associated user input may be identical for different gesture patterns. The predefined gesture patterns may be detected, for example, by an optical scanning device, such as a laparoscope or optical scanner 1100, via the optical scanner interface module 1428, or directly by forces applied to the robotic arm 300 via the force detection module 1422 or other components of the system.

[0211] Actions responsive to user input associated with predefined gesture patterns may include, for example, enabling tool tracking and servoing (i.e., moving) the laparoscope based on the movement of a handheld tool, applying a braking force on the robotic arm (e.g., preventing its further movement), applying a software lock on the robotic arm, dynamically changing the amount of time it takes the robotic arm to transition between states from a default setting, loading a virtual menu overlay onto the video feed whereby a surgical instrument within the laparoscope's field of view acts as a pointer to trigger further actions available from the virtual menu, and / or, if applicable, identifying a member of the surgical staff touching the robotic arm. This information may be used so that the system does not move the robotic arm if the surgeon is not touching it, for example, to avoid scenarios where an external force is acting on the robotic arm (e.g., an optical cable or other wire is pulled across the robotic arm) and ensure that the system perceives forces that are intentional from the surgeon. The same information may be used to detect the surgeon's gaze direction, e.g., whether the surgeon is looking at a video feed or elsewhere in the room, so that the system can freeze the robotic arm if the surgeon's gaze is not in the direction it should be. Additionally, the system may reposition the camera's field of view based on the direction the surgeon is facing, for example, based on data from the optical scanner 1100, or based on the object the surgeon is believed to be looking at.

[0212] As described above, in response to detecting a predefined gesture pattern by the user, for example, a predefined pattern of movement of the distal tip of a surgical instrument within the laparoscope's field of view, the gesture detection module 1430 may overlay a virtual menu on the video feed such that the surgical instrument within the laparoscope's field of view acts as a pointer, as shown in FIGURE 52. Additionally, the gesture detection module 1430 may detect additional predefined patterns of movement of the distal end of the surgical instrument, for example, two rapid movements in the same direction or a circular movement over a selection area of ​​the virtual menu, which may be interpreted as a selection actuation, for example, a click on the virtual menu. 52, a virtual menu overlay on the video feed may include menu options, e.g., "hot corners," in the corners of the video feed for the system to automatically move a robotic arm coupled to the laparoscope to follow the surgical instrument and / or zoom in or out, change the field of view of the laparoscope, turn assisted speculum mode on / off, adjust the holding force of a robotic arm coupled to a retractor, e.g., the amount of force that may be applied to the distal tip of a surgical instrument before the system transitions from a passive mode to a collaborative manipulation mode, turn audio on / off, turn haptic feedback on / off, etc. As will be understood by one skilled in the art, more or fewer menu options may be provided via the virtual menu. In some embodiments, initiation of the display of the virtual menu overlay on the video feed may be triggered by, for example, actuation of an external actuator such as a foot pedal, a predefined pattern of force applied to the robotic arm, such as double-tapping the wrist portion 311 and / or a surgical instrument coupled to the robotic arm as detected by an encoder at the distal end of the robotic arm, voice activation, a radio button, a hot button, etc.

[0213] In some embodiments, the operator may actively switch the system, for example via user interface 1408, into a command mode in which certain movements or gestures of the robotic arm, surgical instrument, operator, or other movements as described herein are monitored by gesture detection module 1430 to determine whether they are consistent with predefined gesture patterns associated with predefined user inputs.

[0214] A passive mode determination module 1432 may be executed by the processor 1402 to analyze the motion characteristics of the robotic arm 300 and determine whether to switch the operation of the robotic arm 300 to a passive mode in which the system, via the motor interface module 1426, applies impedance to the joints of the robotic arm 300 in an amount sufficient to maintain the robotic arm 300, and therefore, if applicable, the surgical instrument attached thereto, in a static position, thereby compensating for the mass of the robotic arm 300 and the surgical instrument and any other external forces acting on the robotic arm 300 and / or the surgical instrument. If the robotic arm 300 is moved slightly while in passive mode, but not with enough force to switch out of passive mode, the system may adjust the amount of impedance applied to the robotic arm 300 to maintain the static position, and continue this process until the robotic arm 300 is held in a static position. For example, the passive mode determination module 1432 may determine to switch the operational mode of the robotic arm 300 to passive mode if the movement of the robotic arm due to movement at the handle of a surgical instrument, as determined by the force detection module 1422, is less than a predetermined amount, e.g., 1-5 mm or less, for at least a predetermined dwell period associated with the robotic arm 300. The predetermined dwell period refers to the length of time that the robotic arm 300 and / or, if applicable, the surgical instrument attached thereto, are held in a static position. For example, the predetermined dwell time may range from, e.g., 0.1 to 3 seconds or more and may be adjusted by the operator. FIG. 19 illustrates a table or example values ​​of threshold dwell times for a range of sample instrument types.

[0215] In some embodiments, the passive mode determination module 1432 may determine to switch the operational mode of the robotic arm 300 to passive mode if movement of the distal end of the robotic arm due to movement at the handle of the surgical instrument, as determined by the force detection module 1422, has a velocity that is less than a predetermined dwell speed / velocity. For example, if the passive mode determination module 1432 determines that the distal end of the robotic arm 300 and / or, if applicable, the surgical instrument attached thereto is moving at a velocity slower than a predetermined dwell speed for the entire predetermined dwell period, the passive mode determination module 1432 may switch the operational mode of the robotic arm 300 to passive mode. Figure 19 illustrates a table or example values ​​of threshold dwell speeds for a range of sample instrument types. For example, for surgical instruments such as specula and tissue manipulation devices, the threshold dwell speed may be, for example, 3-5 mm / sec, and for surgical instruments such as suturing instruments, needle holders, high force instruments, staplers, and clip appliers, the threshold dwell speed may be, for example, 1-2 mm / sec. In some embodiments, the passive mode determination module 1432 may determine to switch the operating mode of the robotic arm 300 to the passive mode in response to the attachment of the surgical instrument to the robotic arm 300 and / or the corresponding detachment of the surgical instrument from the robotic arm 300 based on the identification of the surgical instrument.

[0216] A collaborative operation mode determination module 1434 may be executed by the processor 1402 to analyze the motion characteristics of the robotic arm 300 and determine whether to switch the motion mode of the robotic arm 300 to a collaborative operation mode in which the robotic arm 300 is allowed to move freely in response to movements at the handles of surgical instruments to perform laparoscopic surgery using surgical instruments, while the system applies impedance to the robotic arm 300 via the motor interface module 1426 in an amount sufficient to account for the mass of the surgical instruments and the robotic arm 300. Additionally, the impedance applied to the robotic arm 300 may provide a predetermined level of viscoelasticity perceptible by the operator. FIG. 19 illustrates a table or example values ​​of viscoelasticity levels for a range of sample instrument types. In some embodiments, the viscoelasticity level may be a function of the speed at which the surgical instrument is being moved and the distance from the trocar point to the tip of the instrument. For example, the collaborative manipulation mode determination module 1434 may determine to switch the operational mode of the robotic arm 300 to the collaborative manipulation mode if the force applied at the robotic arm 300 due to a force applied at the handle of a surgical instrument exceeds a predetermined threshold (e.g., a "breakaway force") associated with the robotic arm 300. The predefined force threshold may be, for example, at least 7 Newtons, approximately 7 Newtons, at least 7 Newtons, 4-15 Newtons, or 4-10 Newtons. The predefined force threshold may depend on the type of surgical instrument being used and / or whether there is an external force being applied to the surgical instrument.

[0217] FIG. 19 illustrates a table of predefined force thresholds or example values ​​for a range of sample instrument types. As shown in FIG. 19, the predefined force thresholds may reflect typical external tissue forces that may be imparted on a surgical instrument. In some embodiments, the predefined force thresholds may be increased depending on the direction of the breakaway force if the force is imparted on the surgical instrument by tissue or organ or otherwise. For example, if the breakaway force is in the same direction as the force imparted on the surgical instrument from the tissue or organ, the predefined force threshold may be increased by an amount equal to or corresponding to the force imparted on the surgical instrument from the tissue or organ. In some embodiments, the predefined force thresholds for individual robotic arms are adjusted based on the patient's body mass index ("BMI"). For example, patients with higher BMIs may have heavier livers that will likely impart greater forces on the instrument. Thus, the predefined force thresholds may be selected to be higher for patients with higher BMIs. Thus, an operator may activate a "high force mode," for example, via user interface 1408, in which the predefined force threshold is increased to accommodate working with heavier tissue or organs. For example, the predefined force threshold may be selectively increased by 20-100% or more.

[0218] Furthermore, the force applied by the user on the surgical instrument and any external tissue force applied to the surgical instrument may be direction-dependent. For example, if the force applied by the user on the surgical instrument is in the same direction as the external tissue force applied to the surgical instrument, the two forces may be additive, such that the amount of force applied by the user on the surgical instrument required to overcome a predefined force threshold may be reduced by the magnitude of the external tissue force, such that a force below the predefined force threshold would be required to exit the passive mode and enter the collaborative operation mode. On the other hand, if the force applied by the user on the surgical instrument is in the opposite direction to the external tissue force applied to the surgical instrument, the necessary amount of force applied by the user on the surgical instrument required to overcome the predefined force threshold may be increased by the magnitude of the external tissue force, such that a force above the predefined force threshold would be required to exit the passive mode and enter the collaborative operation mode.

[0219] Additionally, if the force applied by the user on the surgical instrument is in a direction perpendicular to the external tissue force applied to the surgical instrument, the required amount of force applied by the user on the surgical instrument required to overcome the predefined force threshold may not be affected by the magnitude of the external tissue force, such that the required force applied by the user on the surgical instrument required to exit the passive mode and enter the collaborative manipulation mode will be equal to the predefined force threshold. For other directions, the force vector of the applied force may be added to or offset from the force vector of the external tissue force, if applicable, to overcome the predefined force threshold for the particular surgical instrument coupled with the system or robotic arm, depending on the external tissue force and the direction of the force applied by the user. In some embodiments, the collaborative manipulation mode determination module 1434 may determine to switch the operating mode of the robotic arm 300 to the collaborative manipulation mode based on the identification of the surgical instrument.

[0220] The haptic mode determination module 1436 may be executed by the processor 1402 to analyze the motion characteristics of the robotic arm 300 and determine whether to switch the motion mode of the robotic arm 300 to a haptic mode in which the system applies impedance to the robotic arm 300 via the motor interface module 1426 at a higher amount than that applied in the collaborative manipulation mode, making movement of the robotic arm 300 more viscoelastic than in the collaborative manipulation mode in response to movement at the handle of a surgical instrument. For example, the haptic mode determination module 1436 may determine to switch the motion mode of the robotic arm 300 to the haptic mode if at least a portion of the robotic arm 300 and / or a surgical instrument attached thereto is within a predefined virtual haptic boundary. Specifically, a virtual haptic boundary may be established by the system such that the robotic arm or a surgical instrument coupled thereto should not violate the boundary. For example, a virtual boundary may be established at the surface of a patient to prevent the robotic arm or any portion of an instrument supported by the robotic arm from contacting the patient except through one or more trocars. Similarly, the virtual haptic boundary may include a haptic funnel to help guide the instrument into the patient as the operator inserts the instrument into the trocar port. Thus, for example, based on position data of the robotic arm 300 and / or the surgical instrument coupled thereto received by the robotic arm position determination module 1418 and / or the trocar position detection module 1420, the haptic mode determination module 1436 may determine whether the robotic arm 300 and / or the surgical instrument are within a predefined virtual haptic boundary and therefore transition the robotic arm 300 into a haptic mode in which the processor 1402 may instruct the associated motors to apply an effective amount of impedance to the joints of the robotic arm 300 that is perceptible by the operator and communicate the virtual haptic boundary to the operator. Thus, the viscoelasticity of the robotic arm 300 as observed by the operator will be much higher than in the cooperative manipulation mode.In some embodiments, the haptic mode determination module 1436 may determine to switch the operational mode of the robotic arm 300 to the haptic mode based on the identification of the surgical instrument.

[0221] The robot-assisted mode determination module 1438 may be executed by the processor 1402 to analyze the motion characteristics of the robot arm 300 and determine whether to switch the motion mode of the robot arm 300 to a robot-assisted mode in which the processor 1402 may command associated motors, via the motor interface module 1426, to cause movement of corresponding links and joints of the robot arm 300 to achieve a desired result. For example, the robot-assisted mode determination module 1438 may determine to switch the motion mode of the robot arm 300 to the robot-assisted mode if a predefined condition exists, e.g., based on data obtained from the optical scanner interface module 1428.

[0222] For example, the robotic-assisted mode determination module 1438 may determine, based on image data obtained from the laparoscope or optical scanner 1100 via the optical scanner interface module 1428, that a condition exists, e.g., the field of view of the laparoscope coupled to the robotic arm 300 or the optical scanner 1100 is not optimal for a given surgical procedure due to, e.g., obstruction by a surgeon or assistant or another component of the system, such that the robotic arm coupled to the laparoscope or optical scanner 1100 should be repositioned or zoomed in or out to optimize the view of the surgical site for the operator. Thus, in the robotic-assisted mode, the processor 1402 may move, reposition, and / or zoom the laparoscope or increase the resolution of the image, or otherwise instruct the robotic arm 300, either automatically / semi-automatically or in response to user input by the operator. For example, user input by the operator may be determined by gesture detection module 1430 as described above such that in a first direction, movement of the robotic arm or surgical instrument in a predefined gesture pattern causes the endoscope to increase resolution or magnification, in a second direction, causes the endoscope to decrease resolution or magnification, and movement in another predefined gesture pattern causes the robotic arm to hold the laparoscope and retract it away from the patient's body.

[0223] Additionally, the robot-assisted mode determination module 1438 may determine that a condition exists, e.g., one or more trocars are not in an optimal position, e.g., due to patient movement, such that the robotic arm 300 should be repositioned to maintain the trocars in an optimal position, e.g., the approximate center of the robotic arm 300's range of motion, thereby minimizing the risk of reaching the robotic arm's joint limits during the procedure. Thus, in robot-assisted mode, the processor 1402 may instruct the system to reposition the robotic arm 300, e.g., via vertical / horizontal adjustments by the platform 100 or via the joints and linkages of the robotic arm 300, to better align the surgical instrument workspace.

[0224] The robot-assisted mode determination module 1438 may determine, based on image data obtained from the laparoscope or optical scanner 1100 via the optical scanner interface module 1428, that a condition exists, e.g., the distance between an object and the robotic arm 300 is within a predetermined threshold, such that the robotic arm should be frozen to avoid collision with the object. Thus, in the robot-assisted mode, the processor 1402 may instruct the robotic arm 300 to apply a braking force to slow the robotic arm or to block or prevent movement within a predetermined distance from another object.

[0225] The robot-assisted mode determination module 1438 may further determine that a condition exists such that the robotic arm should be repositioned to provide the user with more available workspace in the vicinity of the surgical instrument coupled to the extended robotic arm, e.g., the robotic arm 300 is in the extended position for a period of time that exceeds a predetermined threshold during the surgical procedure. Thus, in the robot-assisted mode, the processor 1402 may instruct the system to reposition the robotic arm 300 and move it closer to the surgical instrument, e.g., via vertical / horizontal adjustment by the platform 100 and / or via the joints and linkages of the robotic arm 300.

[0226] Fault detection module 1440 may be executed by processor 1402 to analyze data indicative of operational characteristics of the system, such as position data generated by robotic arm position determination module 1418 and / or trocar position detection module 1420 and / or force measurements calculated by force detection module 1422, to detect whether a fault condition exists. For example, fault detection module 1440 may detect a fault condition in the system and determine whether the fault condition is a "minor fault," a "major fault," or a "catastrophic fault," and each category of fault condition may be resolved in a different predefined manner.

[0227] For example, the obstacle detection module 1440 may detect a minor fault condition, such as the robot arm 300 being moved at a speed that exceeds a predetermined speed threshold, which may be resolved by, for example, slowing the movement of the robot arm 300. In some embodiments, the system may automatically apply additional impedance to the robot arm 300 when the robot arm 300 is moving too fast and thereby unable to force an operator to slow the movement of the robot arm 300. Additionally, the obstacle detection module 1440 may detect a major fault condition, such as an inadvertent bump of the robot arm 300, as indicated by a large force applied to the robot arm 300 by someone other than the operator. In response to detecting a major fault condition, the obstacle detection module 1440 may activate a braking mechanism associated with each motorized joint of the robot arm 300 (or at least the joint associated with the major fault condition), thereby freezing the robot arm 300 and preventing further movement of the robot arm 300. Such a critical fault condition may be cleared by the operator activating a “Clear” option displayed on the user interface 1408. The fault detection module 1440 may detect a fatal fault condition, such as redundant encoders associated with a given joint of the robot arm 300 generating different formation angle measurements with a delta exceeding a predetermined threshold. In response to detecting a fatal fault condition, the fault detection module 1440 may activate braking mechanisms associated with each motorized joint of the robot arm 300, thereby freezing the robot arm 300 and preventing further movement of the robot arm 300. Such a fatal fault condition may be cleared by the operator restarting the system. If the fatal fault condition is still detected by the fault detection module 1440 upon restarting the system, the robot arm 300 will remain frozen until the fatal fault condition is cleared.

[0228] The indicator interface module 1442 may be executed by the processor 1402 to cause the indicator 334 to communicate the status of the system, e.g., the operational mode of the robotic arm 300, to an operator or other user based on, for example, decisions made by the passive mode determination module 1432, the collaborative manipulation mode determination module 1434, the haptic mode determination module 1436, and / or the robot-assisted mode determination module 1438. For example, the indicator interface module 1442 may cause the indicator 334 to illuminate in a specific color light associated with a specific state of the system. For example, the indicator interface module 1442 may illuminate the indicator 334 in a first color (e.g., yellow) to indicate that no surgical instrument is attached to the robotic arm and the robotic arm can be freely moved to allow the system to compensate for the mass of the robotic arm, in a second color (e.g., purple) to indicate that no surgical instrument is attached to the robotic arm and the robotic arm can be freely moved to allow the system to compensate for the mass of the robotic arm and the mass of a surgical instrument coupled to the robotic arm. Indicator 334 may be illuminated in a third color (e.g., blue) to indicate that the robotic arm is in passive mode as determined by passive mode determination module 1432, in a fourth color (e.g., pulsing orange) to indicate that at least a portion of the robotic arm and / or surgical instrument attached thereto is within a virtual force sense boundary, e.g., 1.4 m or more above ground, and in a fifth color (e.g., pulsing red) to indicate that an obstacle has been detected by the system by obstacle detection module 1440. As will be understood by those skilled in the art, different colors and patterns may be communicated by indicator 334 to indicate the status of the system as described above.

[0229] Additionally, the indicator 334 may be illuminated in other distinct colors and / or patterns to communicate additional maneuvers by the robotic arm 300, for example, when the robotic arm 300 retracts the surgical arm in robotic-assisted mode or performs another robotically-assisted maneuver in robotic-assisted mode. As described above, the indicator 334 may also include devices for emitting other alerts, such as audible or text alerts. Thus, the indicator interface module 1442 may cause the indicator 334 to communicate the status of the system to the operator using audio or text and / or light, or in lieu of light. For example, the indicator interface module 1442 may cause one or more speakers to emit an audible alert, e.g., that changes in amplitude and / or frequency, as the robotic arm 300 approaches a potential collision with one or more objects / persons in the operating room.

[0230] Additionally or alternatively, the indicator interface module 1442 may communicate the state of the system, e.g., a transition from collaborative manipulation mode to passive mode, via haptic feedback at the distal end of the robotic arm 300 and, therefore, on the surgical instrument coupled thereto. For example, when the surgical instrument is held in a position for a predetermined dwell time so that the system switches to passive mode, the user may feel a vibration at the surgical instrument indicating that the system has transitioned to passive mode and the user may release the surgical instrument. As another example, after the surgical instrument is coupled to the coupler body, the user may feel a vibration indicating that the surgical instrument is successfully coupled to the robotic arm. The vibration may be strong enough to be felt by the user, but weak enough so that any resulting movement at the distal tip of the surgical instrument is negligible.

[0231] 25 , the fatigue detection module 1444 may be executed by the processor 1402 to detect user fatigue, which may occur during operation of the robotic arm 300 in a surgical procedure. For example, based on data from, e.g., the robotic arm position determination module 1418, the force detection module 1422, and the impedance calculation module 1424, the fatigue detection module 1444 may determine a level of fatigue of an operator using a surgical instrument coupled to the robotic arm 300 and compare the level of fatigue to a predetermined fatigue threshold. For example, the fatigue detection module 1444 may assess an overall score for a given procedure and determine a level of fatigue based on, e.g., the operator's hand tremor, the distance / minimum path traveled by the instrument tip, the time to accomplish a procedure step, and / or the time to complete the procedure. Based on the data generated by the fatigue detection module 1444, the impedance calculation module 1422 may determine the amount of impedance needed to apply to the robotic arm 300, for example, to reduce operator tremors, such that the motor interface module 1426 may cause an associated motor to apply the required impedance to the robotic arm 300. Further, based on the data generated by the fatigue detection module 1444, the motor interface module 1426 may cause an associated motor to move a linkage of the robotic arm 300 to guide the operator's manipulation of a surgical instrument attached thereto.

[0232] The collaborative surgical robotic systems described herein may include additional modules within the memory 1410 of the platform 200 to perform additional tasks based on the acquired data. For example, the system may determine that a surgical instrument is attached to the robot arm 300 by detecting, via the force detection module 1422, a rapid or sudden change in force (a “snap movement”) applied to the robot due to, for example, the attractive force of the magnetic connection between the coupler body and the coupler interface 400. For example, the attractive force of the magnets on the coupler body and the coupler interface 400 may cause a sudden movement on at least the end portion of the robot arm and / or a sudden rotation of the last joint of the robot arm when the magnets are aligned. This sudden movement may thus be detected and trigger the surgical instrument identification module 1412 to determine that an instrument is attached to or detached from the robot arm. Similarly, the surgical instrument identification module 1412 may determine that a surgical instrument has been removed from the robotic arm 300, for example, when subsequent movement of the distal end of the robotic arm 300 involves little or no rotation at the distal-most joint of the robotic arm 300.

[0233] Additionally, the system may determine whether a surgical instrument has been removed from the robotic arm 300 based on the position of the distal end of the robotic arm 300 relative to the trocar point generated by the trocar position detection module 1420 and data indicating the direction of the instrument shaft and / or the orientation of the distal-most link of the robotic arm 300, e.g., the distal wrist link 316. For example, if the instrument is pointing directly at the trocar, there is a higher probability that a tool is attached to the robotic arm. Furthermore, axis Q7 of the robotic arm 300 may indicate the direction in which the instrument is pointing; if an instrument is passing through a trocar port, the distal wrist link 316 will be pointing toward the trocar port. Thus, if the distal wrist link 316 is not pointing toward the trocar port, the system may determine that the robotic arm is not supporting an instrument or that an instrument has not been advanced through the trocar port. For example, if an instrument is detached from the robotic arm 300 and the robotic arm 300 is moved, the calculated orientation of the instrument shaft (e.g., the direction the instrument would face if attached to the robotic arm 300) may no longer be pointing toward the trocar entry point, and likely will no longer be pointing toward the trocar entry point. Thus, if the system determines that a tool is no longer coupled to the robotic arm 300, it may alert the user, for example, via indicator 334.

[0234] Additionally, the system may identify when a user may be attempting to remove or uncouple a surgical instrument from the robotic arm 300 and adjust the removal force required to uncouple the surgical instrument, and thus the coupler body, from the coupler interface 400. For example, if one or more magnets are used to provide a biasing force and bias the surgical coupler body against the coupler interface, a force greater than the attractive force provided by the one or more magnets must be exerted on the surgical instrument and / or the coupler body coupled to the surgical instrument in a direction opposite to the force provided by the one or more magnets to overcome the attractive force and uncouple the coupler body and surgical instrument from the coupler interface. For example, the removal force may be 30-60 Newtons.

[0235] Additionally, the system may collect and analyze telemetry data regarding the force being applied to the robotic arm to assess or estimate whether a user is attempting to remove a tool from the robotic arm and, if applicable, reduce the coupling force between the coupler body and the coupler interface, making it easier for the user to disengage the surgical instrument from the robotic arm. For example, the coupling / removal force may be reduced by 50-80%. Based on historical data and user feedback, as well as data such as whether the user replaces the instrument without adjusting its location, which may indicate inadvertent removal of the instrument, the system may estimate the optimal time to reduce the coupling force between the coupler body and the coupler interface. Additionally, the coupling force may be increased during operation to prevent inadvertent removal of the surgical instrument from the robotic arm.

[0236] Additionally, the system may determine an optimal positioning of the robotic arm 300 and its joints, a surgical instrument coupled to the robotic arm, or other components of the robotic arm and / or the system based on data obtained from an optical scanning device used in conjunction with the system, and provide guidance to the operator of the system to achieve the optimal positioning. Data indicative of the optimal positioning may further be used by the processor 1402, for example, during or after a setup phase, e.g., in a robot-assisted mode, to command motors to move corresponding links and joints of the robotic arm 300 and automatically reposition the robotic arm 300 and / or the optical scanning device to an optimal position.

[0237] Additionally, the system may collect data from sensors, e.g., position data of the robotic arm 300 or a surgical instrument attached thereto via an encoder or optical scanning device, and / or operator position data via a body sensor or optical scanning device, during a procedure, e.g., during setup or operation of the robotic arm 300, so that the processor 1402 may detect deviations in the current user's movements or processes compared to the model or optimal movement pattern and communicate the deviations to the current user in real time. For example, the processor 1402 may cause a monitor to display deviations and optimal and / or actual movement patterns to the current user in real time. Additionally or alternatively, the indicator interface module 1440 may cause the indicator 334 to indicate deviations from the model or optimal movement pattern, e.g., by illuminating in a specific color and / or pattern. Additionally or alternatively, the motor interface module 1426 may apply impedance perceptible by the operator to the robotic arm 30 as haptic feedback, including vibrations, restrictions on movement, or sensations, to indicate deviations from the model or optimal movement pattern. Thus, the system may be used as a training tool for new users, as such data may be used to optimize the position of surgical devices in real time.

[0238] The system may further analyze the depth map generated by the optical scanning device and cluster different groups of (depth) pixels into unique objects, a process referred to as object segmentation. Examples of such algorithms for segmentation may include matching acquired depth map data to known templates of objects for segmentation, using a combination of depth and RGB color images to identify and isolate relevant pixels for objects, and / or machine learning algorithms trained on real or synthetic datasets for identifying and segmenting objects. Examples of such segmentation on a depth map may include locating or determining the position of a robotic arm, identifying a patient port (e.g., a trocar port) and determining the distance from an instrument to the trocar port, identifying a surgeon and distinguishing the surgeon from other operators in the room, and / or identifying the surgeon within the field of view of a sensor. Additionally, the system may use object segmentation algorithms to uniquely identify a surgeon and track the surgeon, for example, relative to a surgical table, a patient, one or more robotic arms, etc. Additionally, the system may use an object segmentation algorithm to determine whether the surgeon is touching or handling any of the robotic arms, and, if applicable, identify the robotic arms being touched or handled by the surgeon. The system may further use object segmentation to locate the surgical instruments and the distal end of the robotic arm in 3D space, such that the system may determine whether a surgical instrument is attached to the distal end of the robotic arm based on, for example, the proximity between the surgical instrument and the distal end of the robotic arm.

[0239] 15 , operation 1500 of the collaborative surgical robotic system described herein is provided. As shown in FIG. 15 , in step 1502, an operator may couple a selected surgical instrument to coupler interface 400 of robotic arm 300 via a coupler body, e.g., coupler body 500, 600, 700. As described above, the operator may select a coupler body sized and shaped to couple with the selected surgical instrument based, for example, on the elongated shaft diameter of the surgical instrument. Once the surgical instrument and coupler body are ready to be coupled to robotic arm 300, the operator may load a calibration file for the selected surgical instrument, e.g., a laparoscope or retractor, via user interface 1408, for example, so that information associated with the selected surgical instrument is loaded into the system. For example, the operator may select a calibration file from a database of calibration files for various surgical instruments. Calibration files may be stored from previous procedures or may be pre-loaded to include calibration files for commonly used laparoscopic instruments.

[0240] If a calibration file for the selected surgical instrument is not available in the database, the operator may use the system to self-calibrate the surgical instrument. For example, FIG. 16 illustrates a surgical instrument calibration process 1600 for calibrating the surgical instrument and determining the center of mass of the surgical instrument, which can be used, for example, in calculating accurate force measurements on the surgical instrument and robotic arm 300 during operation. In step 1601, the operator may activate the “Start” option on the user interface 1408. In step 1602, the operator may select “Load Tool Calibration” to begin the calibration process. In step 1603, the system does not apply any impedance to the robotic arm 300 for gravity compensation of the surgical instrument. The system may apply impedance to the robotic arm 300 to account for the weight of the robotic arm 300, for example, to prevent the robotic arm 300 from dropping off the ground. In step 1604, the surgical instrument is coupled to the coupler interface 400 of the robotic arm 300 via an appropriately sized coupler body, which can cause the wrist portion 411 of the robotic arm 300 to rotate about axis Q7 and engage the coupler body.

[0241] In step 1605, the system compensates for the force applied by the gravity of the surgical instrument and the force applied by the operator's hand by measuring the force applied to the distal end of the robotic arm 300, for example, due to the mass of the surgical instrument. As explained above, the force applied to the distal end of the robotic arm 300 may be measured by measuring the motor current across a motor located in the base of the robotic arm 300. If the system overcompensates for the gravity of the surgical instrument, in step 1606, the robotic arm 300 may "run away," e.g., drift upward. The runaway effect may be detected in step 1607, and in step 1608, the indicator 334 may flash to indicate the runaway to the operator. In step 1609, the system may identify the runaway as a minor fault and therefore apply additional impedance to the robotic arm 300, freezing the robotic arm 300 as it decelerates before removing the additional impedance. Once the minor fault has been addressed, the calibration process 1600 may return to step 1603 .

[0242] After step 1605, when the system compensates for the gravity force of the surgical instrument, if the surgical instrument is removed, either accidentally or manually by the operator in step 1611, the system detects the removal of the surgical instrument from the robotic arm 300 in step 1610. As a result, the system stops compensating for the gravity force of the surgical instrument, and calibration process 1600 may return to step 1603. After step 1605, once the system has compensated for the gravity force of the surgical instrument, calibration process 1600 is ready to enter calibration mode in step 1612. For example, the operator may initiate calibration mode via user interface 1408 in step 1613. In step 1614, the system may indicate to the operator that it is safe to release the surgical instrument, for example, via user interface 1408 and / or flashing indicator 334, so that the operator may release the surgical instrument in step 1616. In step 1615, the system calibrates the surgical instrument.

[0243] 15 , once the surgical instrument and coupler body are ready to be coupled to the robotic arm 300 and the appropriate calibration file has been loaded, the operator can simply place the coupler body near the coupler interface 400 so that the magnetic connection between the coupler body and the coupler interface 400 automatically aligns and couples the surgical instrument to the robotic arm 300. The system will now accurately compensate for the gravity force of the selected surgical instrument. In step 1504, the user may use the cooperatively manipulated surgical system by freely manipulating the surgical instrument coupled to the robotic arm 300 in the normal manner that an operator would do without the robotic arm 300 coupled thereto. 15, as the operator manipulates the surgical instrument, and thus the robotic arm 300 coupled thereto, the system may automatically switch between, for example, a collaborative manipulation mode 1506, a passive mode 1508, a haptic mode 1510, and a robot-assisted mode 1512 (collectively referred to as "operational modes") in response to the detection of predefined conditions, as described below with respect to FIG. 17. In some embodiments, the system may automatically switch only between the collaborative manipulation mode 1506, the passive mode 1508, and the haptic mode 1510. In some embodiments, the operator may select an operational mode for configuring the collaboratively manipulated surgical system in step 1504 prior to using the system.

[0244] For example, an operator may apply a particular force on the distal end of the robotic arm 300, e.g., by manipulating a surgical instrument coupled to the robotic arm 300, to indicate that the operator desires to change a particular robotic arm operating mode. Sensor and / or motor current readings may be used to detect the force applied to the distal end of the robotic arm 300 and determine whether the force matches a predefined force signature associated with the operating change, e.g., by comparing the force to one or more predefined force signatures stored within the system. If a match exists, the system may change the operating mode of the robotic arm to the particular operating mode that matches the force signature.

[0245] As described above, during operation of the collaborative surgical system, the system may continuously monitor the robotic arm and forces applied thereto and detect predefined conditions that require switching the operating mode of the system, as described in method 1700 of FIG. 17 . As shown in FIG. 17 , in step 1702, the system continuously collects data related to a first motion characteristic of the robotic arm and / or a surgical instrument coupled to the robotic arm. For example, as described above, the system may measure motor currents of motors operably coupled to joints of the robotic arm and angles formed at joints of the robotic arm based on measurements by encoders of the robotic arm, and calculate in real time the position of the robotic arm and the surgical instrument, and forces acting on any portion of the robotic arm and, if applicable, the surgical instrument. In step 1704, the system may analyze the data related to the first motion characteristic and determine whether a first condition exists. For example, based on the position and force data of the robotic arm and / or the surgical instrument, the system may determine whether the movement of the robotic arm due to movement of the surgical instrument coupled thereto is within a predetermined movement threshold of the robotic arm for a period of time longer than a predetermined dwell time of the robotic arm. In response to detecting this first condition, in step 1706, the system may modify a first operating parameter of the robotic arm. For example, the system may switch the operating mode of the robotic arm to a passive mode in which the robotic arm maintains the surgical instrument in a static position.

[0246] For example, a first robotic arm may be coupled to a laparoscope, and an operator may manipulate the laparoscope within the patient, e.g., via a monitor displaying images fed from the laparoscope, until the laparoscope provides a desired field of view. In the collaborative manipulation mode, to freely move the laparoscope coupled to the first robotic arm, the operator must apply sufficient force to the laparoscope that exceeds a predetermined force threshold. The predetermined force threshold should be low enough so that little force is required by the operator to freely move the laparoscope. Furthermore, the predetermined force threshold may be selected to resist inadvertent movement away from the passive mode. As the operator freely moves the laparoscope in the collaborative manipulation mode, as described above, the system will apply sufficient impedance to the first robotic arm so that the mass or weight of the first robotic arm is not detectable by the operator or is otherwise significantly damped, compensating for the effects of mass (i.e., inertia) and / or gravity of the first robotic arm and laparoscope during movement. In some embodiments, when the operator couples the laparoscope to the first robotic arm, if the laparoscope is not yet positioned within the patient's body, the system may determine that there are no external forces acting on the surgical instrument and may automatically switch the first robotic arm to a force-sensing mode to guide the operator in moving the laparoscope to the appropriate location through the trocar port, for example, via a virtual force-sensing funnel established around the trocar port.

[0247] When the laparoscope is in a desired position relative to the patient and the surgical site within the patient, the system will automatically switch from the collaborative operation mode to the passive mode in response to detecting that the movement of the first robotic arm due to movement of the surgical instrument is within a predetermined movement threshold for a period of time that exceeds a predetermined dwell time. For example, in response to reaching the desired position, the operator will hold the laparoscope in the desired position for, for example, at least 0.25 seconds. Thus, if the predetermined dwell time is 0.25 seconds, holding the laparoscope in the desired position for a period of time longer than the predetermined dwell period will automatically switch the system to the passive mode. Furthermore, because it may be impossible for the operator to hold the laparoscope perfectly still, at least some movement of the laparoscope is allowed for the duration of the predetermined dwell time in order to enter the passive mode. As described above, in passive mode, the first robotic arm will hold the laparoscope in a static position, for example, by the system applying sufficient impedance to the first robotic arm to compensate for all external forces acting on the laparoscope.

[0248] Similarly, a second robotic arm may be coupled to the retractor, and the operator may freely manipulate the retractor within the patient in a cooperative manipulation mode, for example, by applying sufficient force to the second robotic arm due to a force applied at the retractor exceeding a predetermined force threshold of the second robotic arm to grasp tissue within the patient and retract the tissue, providing a clear view of the surgical site through a laparoscope coupled to the first robotic arm. As the operator grasps / lifts / retracts tissue with the retractor, the system may only compensate for the gravity of the second robotic arm and / or the instrument, and not that of the grasped tissue, so that the operator may feel any other forces acting on the retractor, including, but not limited to, forces acting from the tissue on the instrument. In this optional configuration, the force sensation associated with the grasped tissue may thus be preserved.

[0249] Once the retractor has sufficiently gripped and retracted the tissue, the system may automatically transition to a passive mode in response to the operator holding the retractor in place for a period of time, e.g., exceeding a predetermined dwell period of the second robotic arm, with movement not exceeding a predetermined movement threshold of the second robotic arm. Thus, when the retractor retracts tissue within a patient in passive mode, the second robotic arm will account for the mass of the tissue in addition to the mass of the retractor and second robotic arm. Thus, the predetermined force threshold for switching the second robotic arm from passive mode must exceed the force applied to the second robotic arm due to the force applied by the tissue to the tip of the retractor, such that if the force applied by the tissue to the surgical instrument exceeds a predetermined first threshold of the second robotic arm, the system will automatically switch the second robotic arm from passive mode to, e.g., a collaborative manipulation mode. However, the predetermined force threshold should not be so high that it is too difficult for the operator to move the retractor. As described above, the operator may adjust the predetermined force threshold, for example, via user interface 1408.

[0250] In response to tissue retraction through a retractor coupled to the second robotic arm, the operator may need to readjust the field of view of the laparoscope coupled to the first robotic arm. Thus, the operator may apply a force to the laparoscope that exceeds a predetermined force threshold of the first robotic arm, causing the system to automatically switch the first robotic arm from passive mode to collaborative manipulation mode. Once a new desired position for the laparoscope is achieved, the first robotic arm may automatically return to passive mode if the predefined conditions described above are met. Alternatively, the operator may choose to uncouple the laparoscope, readjust the robotic arm and / or laparoscope, and reattach the laparoscope to the first robotic arm (or another robotic arm) to readjust the laparoscope or reposition the first robotic arm's joint to avoid potential collisions during the laparoscopic procedure, or to switch the laparoscope to a different robotic arm entirely. Upon reattachment of the laparoscope to the first robotic arm, the first robotic arm may automatically switch to passive mode if the predefined conditions described above are met.

[0251] Furthermore, as the operator freely moves the retractor in collaborative manipulation mode, for example, prior to inserting the retractor tip through a trocar into the patient, if the operator moves the retractor tip away from the trocar port so close to the patient's skin that a virtual haptic boundary is established by the system on the patient's skin outside the trocar port, the system may automatically switch to haptic mode. Thus, the system may apply a much higher impedance to the second robotic arm than that applied to the second robotic arm in collaborative manipulation mode to indicate to the operator that they are approaching or within the virtual haptic boundary. For example, the operator's movement of the retractor may feel much more viscoelastic in haptic mode. The system may remain in haptic mode until the operator moves the retractor out of the virtual haptic boundary. In some embodiments, in haptic mode, the second robotic arm may reduce the effects of gravity, eliminate instrument tip tremors, apply force feedback, and avoid critical structures as defined by the virtual haptic boundary. Thus, the system does not replace the operator, but rather augments the operator's capabilities through features such as gravity compensation, tremor cancellation, haptic barriers, and force feedback.

[0252] In some embodiments, the system may switch the second robotic arm into a robot-assisted mode. For example, as the operator attempts to retract the tissue, if more force is required to retract the tissue than the operator is able or willing to apply to the retractor, the operator may provide user input to the system indicating that the operator desires the second robotic arm to assist in retracting the tissue. For example, as described above, the operator may perform a predefined gesture pattern, which may be detected, for example, by the optical scanner 1100, to cause the system to switch the second robotic arm into a robot-assisted mode and move the motors of the second robotic arm to the second robotic arm, and thus the retractor, to provide the additional force required to retract the tissue.

[0253] Additionally, as described, instead of manually manipulating the laparoscope coupled to the first robotic arm, the operator may provide a separate user input to the system indicating that the operator desires the system to reposition the laparoscope. For example, if the operator actively manipulates surgical scissors, which may or may not be coupled to the robotic arm of the system, so that the tips of the surgical scissors are within the field of view of the laparoscope coupled to the first robotic arm, the operator may use the tips of the surgical scissors to perform a predefined gesture pattern, e.g., quickly move the surgical scissors back and forth in a particular direction. A predefined gesture pattern of the surgical scissors may be captured as image data by the laparoscope, and based on the data, the system may detect the predefined gesture pattern and associate it with a predefined user input that requests the system to switch the first robotic arm from a passive mode to a robot-assisted mode, causing the first robotic arm to reposition itself and thus the laparoscope to adjust its field of view in the direction of the pattern movement of the surgical scissors. As described above, additional gesture patterns may be performed within the field of view of the laparoscope via the surgical scissors to cause the first robotic arm to retract the laparoscope and / or zoom in or out or improve resolution on the laparoscope itself. In some embodiments, using object tracking of an additional tool within the laparoscope's field of view, such as surgical scissors that are actively operated by the operator, based on data images captured by the laparoscope, the system may automatically switch a first robotic arm coupled to the laparoscope into a robot-assisted mode, reposition itself, adjust the field of view, and ensure that the tip of the surgical scissors remains within an optimal position within the laparoscope's field of view during the procedure.

[0254] The operational mode of any one of the robotic arms may be varied independently of the operational modes of the other robotic arms of the system. Additionally, the operational parameters of each robotic arm may be tailored to the specific surgical instrument coupled thereto. For example, the predetermined force threshold for a robotic arm coupled to a retractor device may be higher than the predetermined force threshold for a robotic arm coupled to a laparoscope because the retractor will withstand higher forces during the procedure. The sensors, motors, etc. of the system may be active in all modes, but may also act very differently in each mode, including, for example, acting as if inactive. As will be understood by one skilled in the art, the system may include more than two robotic arms so that an operator may couple a third surgical instrument, e.g., a grasper device, to the third robotic arm and a fourth surgical instrument, e.g., a surgical scissors device, to the fourth robotic arm for operation during a laparoscopic procedure.

[0255] In some embodiments, the operational mode of the robotic arm may be changed in response to user input provided by an operator. For example, the operator may selectively change the operational mode of the robotic arm by activating a button, dial, or switch located on the robotic arm, a foot pedal or foot switch, a voice command, input on a touchscreen, or using gestures or force signatures, as described above. In some embodiments, the operational mode of the robotic arm may be changed solely based on the coupling of a surgical instrument to the coupler interface via the coupler body. As described above, the system may automatically identify the surgical instrument based on the coupling of the coupler body to the coupler interface. Thus, based on the identification of the surgical instrument coupled to the robotic arm, the system may automatically switch the operational mode of the robotic arm to a predetermined operational mode, e.g., passive mode, if the surgical instrument is an endoscope, or if the robotic arm is already in passive mode, the system will remain in passive mode in response to the coupling of the endoscope with the robotic arm.

[0256] Similarly, based on the identification of the surgical instrument responsive to its attachment to the robotic arm, the system may automatically switch the operational mode of the robotic arm to a collaborative operation mode, for example, if the surgical instrument identification indicates that the surgical instrument is a tool that will be actively operated by the operator during a laparoscopic procedure. Additionally, based on the identification of the surgical instrument responsive to its attachment to the robotic arm, the system may automatically switch the operational mode of the robotic arm to a robot-assisted mode, for example, if the surgical instrument identification indicates that the operator desires a tool that is fully robotically controlled, such as an irrigation device. Thus, in response to the attachment of an irrigation device to the robotic arm, the system will switch to a robot-assisted mode and have the robotic arm position the irrigation device at a desired location within the body.

[0257] Additionally, the system may be instructed by an operator, for example, via the user interface 1408, to operate the robotic arm in fewer than the four operational modes discussed above. For example, the operator may deactivate any one of the operational modes for a given procedure. In some embodiments, the system may operate the robotic arm in additional operational modes, such as a locked mode, which may be similar to the passive mode, except that the predetermined force threshold of the robotic arm for switching out of the passive / locked mode may be high enough to effectively freeze the robotic arm, protecting the robotic arm from inadvertently switching out of the passive / locked mode, e.g., to avoid movement due to an inadvertent bump of the robotic arm. In this locked mode, if the force from an inadvertent bump is high enough to cause even slight movement of the robotic arm, the system may cause the robotic arm to reposition itself to the position it was in prior to the inadvertent bump.

[0258] Additionally, when a surgical instrument is not coupled to the distal end of the robotic arm of the system, the system is still capable of automatically switching the operating mode of the robotic arm in response to movement of the robotic arm by an operator upon detection of the predefined conditions described above. Thus, the system allows the robotic arm to remain in a static position when in passive mode, and will apply impedance to the joints of the robotic arm and compensate for the mass of the robotic arm when in collaborative mode if the system detects that the force applied to the robotic arm by the operator exceeds the robotic arm's predetermined force threshold. Additionally, the system will switch the robotic arm to haptic mode if the operator attempts to move any part of the robotic arm within a predefined virtual haptic barrier. In step 1514, once the laparoscopic procedure is complete, the operator may remove the surgical instruments from the individual robotic arms.

[0259] 18A-18C, force measurements during operation of the robotic arm 300 are provided. As described above, upon attachment of a surgical instrument to the coupler interface 400 via a coupler body coupled to the surgical instrument, the orientation of the surgical instrument may be automatically determined based on the magnetic connection between the coupler interface and the coupler body. Further, as described above, a calibration file of a surgical instrument coupled to the robotic arm 300 loaded onto the system may include information about the surgical instrument, including, for example, the mass of the surgical instrument, the center of mass of the surgical instrument, and the length of the surgical instrument, so that the distance D3 between the center of mass and the instrument tip may be derived. Additionally, as described above, the position of the surgical instrument at the trocar, for example, through which the surgical instrument enters the patient's body, may be calculated in real time so that the distance D2 between the center of mass of the surgical instrument and the trocar may be derived in real time. Additionally, as described above, the coupler body is preferably coupled to the surgical instrument at a fixed, known position (which may be included in a calibration file) along the elongate shaft of the surgical instrument, e.g., adjacent a proximal portion of the surgical instrument, so that the distance D1 between the center of mass of the surgical instrument and the coupler body, e.g., the attachment point to the distal end of the robotic arm 300, may be derived. Alternatively, or in addition, as described above, an optical scanning device may be used to determine any one of D1, D2, or D3.

[0260] As shown in FIG. 18A, when a surgical instrument is positioned through a trocar Tr without any additional external forces acting on the surgical instrument other than at the trocar Tr, e.g., the surgical instrument is not lifting or retracting tissue within the patient, the force applied by the body wall to the surgical instrument at the trocar Tr (e.g., "body wall force" or "trocar force") may be calculated using the following equation: [ka] In the formula, F eff is the force at the distal end of the robotic arm 300 (e.g., the “end effector force” of the robotic arm 300), W is the weight vector of the surgical instrument (=−mgz), and F tr is the trocar force. Therefore, F eff is the desired force sent to the system, which is the sum of all forces generated in the algorithm pipeline, including, for example, gravity compensation, holding, etc.

[0261] As shown in FIG. 18B, when a surgical instrument is positioned through the trocar Tr and retaining / retracting tissue such that an external force is applied to the tip of the surgical instrument, two forces for disassembly, namely, F tr and F tt Therefore, two equations are required to solve for the two unknown vectors, which are related to the center of mass of the surgical instrument, e.g., L cg It can be a balance of forces around the axis and also a balance of moments. [ka]

[0262] where distances D1 and D3 are known as described above, and D2 may be derived based on the known position of the distal end of the robotic arm 300 and the calculated position of the trocar Tr. As shown in FIG. 18B, the center of mass L of the surgical instrument cg is behind the attachment point of the coupler body to the distal end of the robot arm 300.

[0263] As explained above, the system is configured to measure a force, e.g., a force F applied to the tip of the instrument. tt and / or the force F applied by the instrument on the trocar in use. trexceeds a particular threshold force, the operator may be alerted, and thus, if the calculated force exceeds a threshold force, the system may freeze and / or automatically apply a braking or stopping force to the robotic arm 300 to reduce the force being applied at the tip of the instrument or trocar, by slowing or preventing further movement of the instrument in a direction that would increase the force applied at the tip of the instrument or trocar, and / or by automatically moving the robotic arm in a direction that reduces the force being applied at the tip of the instrument and / or trocar point at the body wall.

[0264] 20 , a high-level example 2000 of different combinations of data inputs for the various sensors and devices of the systems disclosed herein, e.g., system 200, and features and capabilities that any implementation of the systems disclosed herein may have and produce based, at least in part, on multiple possible data inputs is provided. As shown in FIG. 20 , some implementations of the systems may be configured to collect data from at least three monitoring sources 2002, including telemetry from the system (which may include force data from a robotic arm, position data from a robotic arm, etc.), video from a laparoscope tower, and / or data from an optical scanner 1100. The data collected from the monitoring sources 2002 may undergo a data processing step 2004 using one or more processors within the system. The data processing step may include, for example, data fusion (e.g., fusion of data collected from the monitoring sources 2002) and data analysis, which may include algorithmic computation. Additionally, data from monitoring sources 2002 may undergo processing 2004 for the development of system usability features 2006, system safety features 2008, and system performance features 2010. The system may provide features in real time. For example, system usability features may include identifying the surgeon and adjusting the platform height based on the surgeon's profile, detecting the patient's skin surface and creating a virtual boundary around the skin surface to prevent inadvertent contact with the patient's skin surface, detecting the instrument type and automatically loading the appropriate calibration file for the particular instrument, etc. Additionally, system safety features may include, for example, displaying a virtual map of the area surrounding platform 100 as the operator moves platform 100 throughout the operating room, providing the operator with a view of the area surrounding platform 100 so that the operator may avoid collisions between platform 100 and any objects and / or people in the area surrounding platform 100.

[0265] Referring to FIG. 21 , a schematic overview of the electrical components of the electrical system and connectivity 2100 of the system is provided. This includes the flow of energy throughout the illustrated portion of the system, ports that may be used for connectivity, and other details related to the various electronic components. For example, the system may include a non-real-time computer 2102 that may be used to obtain data from an optical scanning device and perform other functions. The non-real-time computer 2102 may also control the system's graphical user interface for interaction by the surgeon. As explained above, the graphical user interface may include a touch screen. The non-real-time computer 2102 may be, for example, a 10th Generation Intel® Core™ processor. TM i7-10700 processor, 32GB RAM (optionally 2x16GB, DDR4, 2933Mhz), standard keyboard and 512GB PCIe M.2 SSD + 1TB SATA 7200 RPM hard drive, wireless and Bluetooth card, e.g., Killer TM Wi-Fi 6 AX1650i (2x2) 802.11ax radio and Bluetooth® 5.1, and / or NVIDIA® GeForce RTX TM The system may include a 2060 6GB GDDR6 graphics card. The system may further include a real-time computer 2104, which may be used to operate and control the robotic arm and associated robot controller and / or other functions, such as obtaining data and information from an optical scanning device. The real-time computer 2104 may include, for example, an Intel Core i7 (8th generation) processor, 32GB of RAM for memory, a 500GB SSD hard drive, and / or two or more RJ45 connectors for Ethernet connectivity.

[0266] Referring now to FIG. 22 , a flowchart of a process 2200 for obtaining and processing data from an optical scanning device is provided. As shown in FIG. 22 , in step 2202, depth data may be obtained from one or more optical scanning devices, e.g., optical scanner 1100. In step 2204, filtering / other signal processing algorithms, such as median filters, Gaussian noise removal, anti-aliasing algorithms, morphological operations, ambient light adjustment, etc., may be performed. In step 2206, 3D object segmentation may be performed using, for example, template matching, machine learning, brute-force matching, color+depth segmentation, 2D-3D detection, pixel value thresholding, etc. In step 2208, object coordinates may be transformed into a task space. For example, transforming object coordinates into a task space may include converting the object's position and orientation from the coordinate frame of the optical scanning device to the coordinate frame of the required task (e.g., a robot frame for robot control, a cart frame for system setup, etc.). Additionally or alternatively, transforming object coordinates into task space may include using known optical scanning device to support platform (e.g., cart) transforms, surgical robot transforms, and / or user interface screen transforms, and generating new transforms for specific tasks, such as tracking the surgeon's body (e.g., face, hands, etc.) relative to different elements of the system (e.g., support platform, robotic arm, screen, etc.), tracking the surgical table relative to the cart platform, tracking patient orientation for system setup, tracking trocar port location and orientation for setup, tracking operating room staff position for safety, etc. In step 2210, the desired task may be performed, such as moving the robotic arm to the vicinity of the patient / trocar port for easy setup, tracking operating room staff to ensure the system responds only to surgeon commands, recording the surgeon's hand movements during different phases of surgery, etc.

[0267] 22 illustrates a flowchart of a process 2212 for obtaining and processing data from an optical scanning device. In step 2214, depth data may be obtained from one or more optical scanning devices, such as the optical scanner 1100. In step 2216, specular noise filtering may be performed. In step 2218, patient / trocar port segmentation and identification may be performed. In step 2218, the tracked port coordinates may be transformed into robot coordinate space. In step 2222, the robot arm may be moved to a desired proximity of the patient / trocar port.

[0268] 23, an exemplary data flow 2300 of the system is provided. As shown in FIG. 23, a non-real-time computer 2302 may collect data from an optical scanning device, e.g., optical scanner 1100, and / or from a camera feed from a laparoscope. The non-real-time computer 2302 may also receive data from a real-time computer 2308, which has a robot controller, including telemetry information such as the position of the robot arm, forces applied to the various motors / sensors of the robot arm, operating mode information, etc. The non-real-time computer 2302 may also receive data from a patient database 2310, which has information specific to the patient in the procedure, including, for example, CT scan data, relevant health conditions, and other information that may be desired by the surgeon.

[0269] The non-real-time computer 2302 may further provide user feedback 2312 to the user via a user interface 2314. User feedback may include, for example, collision notifications, positioning information and / or recommendations regarding various components of the system, the operating mode being detected by the system, etc. The non-real-time computer 2302 may also provide commands 2318, e.g., high-level commands, to the real-time computer 2308. High-level commands may include, for example, mode changes, trajectories, force barriers, user configurations, etc. The real-time computer 2308 may include a robot controller 2320 programmed to provide robot commands 2322, e.g., motion or force commands, to one or more robotic arms 2324, e.g., robot arm 300. The robot controller 2320 may receive robotic feedback data 2326, e.g., motion, force, and / or touch point data, etc., from the one or more robotic arms 2324.

[0270] 25 , a method 2500 for estimating user fatigue during a surgical procedure using the robotic arm 300 is provided. As described above, algorithms related to gravity compensation, viscoelasticity, and / or mass effects may be used to account for user fatigue. Specifically, during a laparoscopic procedure, a surgeon may incur fatigue as the procedure progresses and may experience hand tremors or erroneous tool motion with respect to surgical tools such as scissors, needle holders, cautery tools, graspers, etc. As shown in FIG. 25 , in step 2502, the system may receive and monitor data indicative of operator performance from an optical scanner 1100, such as a LiDAR camera, robotic telemetry, and / or endoscope, for example, while the operator manipulates a surgical instrument coupled to the robotic arm 300 during the surgical procedure. Learning from large datasets of clinical procedures and / or collecting and analyzing data during procedures or portions of procedures may enable the system to estimate the surgeon's level of performance as the procedure progresses, step 2504, and may further enable the system to adapt algorithm parameters to help the surgeon more effectively move while coordinating the surgical instruments attached to the robotic arm. For example, in step 2506, the system may adjust one or more motion parameters of the robotic arm 300 to change its behavior. If the fatigue level exceeds a particular threshold, the system may alert the surgeon, step 2608. Additionally, a procedure ranking step may be used to enable the system to provide the surgeon with an overview of their performance for a given procedure and indicate their overall progress per procedure.

[0271] In some embodiments, the system may collect data during a procedure indicative of at least one of the operator's hand tremor, the distance / minimum path traveled by the instrument tip, the time to accomplish a procedure step, and / or the time to complete the procedure, compare such data to a threshold or predefined value for each coefficient, and determine whether the magnitude of any one of the coefficients reaches a level sufficient to cause the system to alert the operator and / or adjust one or more operating parameters to reduce user fatigue. For example, the system may eliminate or reduce instrument tip tremor by applying force on the instrument, increasing the instrument's impedance or viscoelasticity, avoiding critical structures, and / or applying force feedback. User fatigue may be identified, for example, when procedure time increases above a threshold for a particular procedure, when the number of surgical instrument movements increases above a threshold for a particular procedure or otherwise indicates erroneous or uncontrolled movements, if the operator moves the instrument into a kinematic barrier a predefined number of times, if the operator applies excessive force on a trocar one or a predetermined number of times, etc. As described above, such data may be collected using sensors and / or one or more optical scanning devices on the robotic arm. When a particular level of user fatigue is identified by the system, the system may increase the viscoelasticity or impedance of the instrument and / or the robotic arm associated with the instrument, and reduce the magnitude and / or number of movements of the surgical instrument and / or robotic arm.

[0272] Additionally, the system may use an estimate of the amount of tremor involved in movement to collect data on the speed and frequency with which the operator moves various instruments / laparoscopes and estimate the additional viscoelastic required to reduce tremor without interfering with the movement or adding unnecessary fatigue to the operator. In some embodiments, the controller of the robotic arm 300 may iteratively adjust the viscoelastic value for a particular instrument, collect data related to the instrument's movement, and assess whether additional adjustments are needed to the viscoelastic applied to the instrument. Furthermore, the system may employ an iterative approach to optimize a particular motion characteristic or parameter of the robotic arm 300 using an additive algorithm that includes collecting data related to a particular motion characteristic or parameter, varying the motion characteristic or parameter, collecting additional data related to the motion characteristic or parameter, and analyzing the data to determine whether additional changes to the motion characteristic or parameter should be made, which may be based, for example, on deviations between the actual data value of the motion characteristic or parameter and a preferred or optimal value.

[0273] Referring now to FIG. 26, a data flow 2600 for a distributed network of collaborative robotic surgical systems is provided. For example, a distributed network of collaborative robotic ("cobot") surgical systems may be used in multiple hospitals, each of which may be connected to an online database. This arrangement may provide significantly more data and user information that may be used by any of the cobot systems during operation. The system may aggregate data from the distributed network of systems and identify an optimal configuration based on factors such as procedure type, surgeon experience, patient demographics, etc. Through analysis or clinician input, the cobot system may identify routine procedures versus procedures that may be more complex. This information may be used to provide advice or guidance to less experienced surgeons.

[0274] Furthermore, centralized procedure data may enable the initiation of large amounts of data analysis for a wide range of clinical procedures originating from different users. Analysis of the data may result in optimized settings for specific procedures, including, for example, optimized system positioning, optimal port placement, optimal algorithm settings for each robotic arm, and / or detection of procedural abnormalities (e.g., excessive force, time, bleeding, etc.). These optimal settings or parameters may depend on patient and tool characteristics. As described above, a surgeon may load and use optimal settings from another surgeon or group of surgeons. In this way, optimal settings may be achieved, for example, depending on the surgeon's level of expertise. In order to track various users within a distributed network of cobot systems, it may be beneficial to identify each user. Thus, a user may log in to the cobot system and access their profile online as needed. In this way, a user may have access to their profile at any location and be able to perform clinical procedures using their settings at different hospital locations.

[0275] An exemplary user profile may contain the user's specific settings and information, including, for example, user name, level of expertise, different procedures performed, and / or area of ​​clinical practice. In addition, clinical procedures may require the user to store specific settings, such as the clinical procedure (e.g., cholecystectomy, hernia, etc.), table orientation and height, preferred port placement, settings per auxiliary arm per algorithm, patient characteristics (e.g., BMI, age, gender), and / or surgical tool characteristics and specifications (e.g., weight, length, center of gravity, etc.). A user may be able to activate their own profile and, optionally, another user's profile, such as a colleague's profile, a profile most representative of surgeons in the user's area of ​​practice, a profile most representative of surgeons with a specific level of expertise, and / or a profile recommended according to patient characteristics.

[0276] User identification may be performed via password, RFID key, facial recognition, etc. Learning from multiple procedures may result in a greater level of optimization of the cobot system settings for a given procedure. This may include, for example, cart position, individual robotic arm positions, surgical table height and orientation, port placement, and / or set joint positions. These settings may be based on patient height, weight, and gender and may even be interdependent. For example, optimal port placement may depend on patient table orientation.

[0277] Additionally, a clinical procedure may be described as a sequence of clinical procedure steps. Learning these different steps may enable the cobot system to infer the actual steps in real time for a given procedure. For example, learning the clinical steps from a procedure may enable or enable the system to adjust algorithm settings, provide practical custom reminders, notify staff of an estimated procedure completion time, alert staff if required equipment is unavailable in the room, and / or alert staff of an emergency situation.

[0278] During clinical procedures, surgeons will often recognize simple and routine surgical tasks, such as grasping, retracting, and cutting. Learning these different tasks may enable the cobot system to infer, in real time, the surgeon's preferences and habits regarding real-time procedural sequences. Several algorithms of the cobot system may be adjusted (i.e., tuned and optimized) during the procedure based on this sequence recognition to help the user better perform this simple surgical task. An example of such a task is automated retraction of the liver during a gallbladder procedure. By aggregating information across many cases, optimized force vectors may be developed.

[0279] Additionally, several complications may arise during clinical procedures that may result in unexpected steps or surgical actions. Learning how to distinguish these unexpected events will help the cobot system enable several specific safety features. In case of an emergency, the robotic arm may be stopped or motion restricted depending on the level of emergency detected by the system.

[0280] 27A-27D, a configuration for a collaborative surgical system is provided. Platform 2700 may be constructed similarly to platform 100 such that platform 2700 may support one or more robotic arms, e.g., robotic arm 300a' and robotic arm 300b', and move the robotic arms relative to platform 2700. As shown in FIG. 27A, platform 2700 may be moved by a user, e.g., via wheels 104', to a desired position relative to patient table PT while robotic arms 300a', 300b' are in their individual, stowed configurations.

[0281] As the platform 2700 is moved toward the patient, the scene may be observed directly by a depth mapping sensor, e.g., the optical scanner 1100′, which may be mounted on the platform 2700. From the depth map observed and generated by the optical scanner 1100′, important features may be identified, such as the height and / or location of the patient table PT, the surface of the patient's abdomen, the surgeon's position and other characteristics, including the surgeon's height, and the trocar ports, the bases of the robotic arms 300a′, 300b′, e.g., base portions 302a′, 302b′ and shoulder portions 304a′, 304b′, the robotic arms 300a′, 300b′, and / or one or more surgical instruments coupled thereto. Identification of such important features may be performed using standard computer vision techniques, such as template matching, feature tracking, edge detection, etc. As each feature is registered, its position and orientation may be assigned a local coordinate system and transformed to the global coordinate system of the system using standard transformation matrices. Once all features have been transformed into a single global coordinate system, optimization algorithms, such as least squares and gradient descent, may be used to identify the most appropriate vertical and horizontal positions of the robotic arms 300 a′, 300 b′, which may be adjusted to maximize the workspace of the robotic arms relative to their insertion point on the patient via the platform 2700. The optimal workspace may depend on the surgical procedure to be performed and / or the preferred position of the surgeon.

[0282] When platform 2700 is in its desired position relative to patient table PT such that wheels 104' are locked, as shown in Figure 27B, robotic arms 300a', 300b' may be extended away from their respective stowed configurations. The vertical position of the robotic arms relative to platform 2700 may be adjusted to a desired position, as shown in Figure 27C, and the horizontal position of the robotic arms relative to platform 2700 may be adjusted to a desired position, as shown in Figure 27D.

[0283] 28A-28D, screenshots of an exemplary graphical user interface 2800 are provided. The exemplary graphical user interface 2800 may be user-configurable and integrated with the display 110. FIG. 28A illustrates an exemplary start menu. The operator may initiate operation of the collaborative system by activating the "Start" option. FIG. 28B illustrates an exemplary system setup screen. As shown in FIG. 28B, when the system includes two robotic arms, the graphical user interface 2800 may identify which robotic arms should be used with which instruments, e.g., retractor arm 2806 and endoscope arm 2808, and the procedure to be completed. The graphical user interface 2800 may allow the user to pre-load specific calibration files or preset joint positions based on the procedure being performed and / or the surgeon performing the procedure. For example, if a user inputs that a procedure is a laparoscopic cholecystectomy, the system may preload tool types known to be associated with that procedure. Populating these preloaded settings may be accomplished by monitoring the tools the user manually selects for a given procedure. If a given tool is consistently selected for a predetermined number of procedures, the system may automatically preload that tool the next time the procedure is selected by the user.

[0284] Additionally, the operator may adjust the vertical and horizontal position of each robotic arm, as shown in FIGS. 27C and 27D above. As shown in FIG. 28B , to adjust the vertical and / or horizontal position of a robotic arm that will be or is currently coupled to a retractor device, the operator may toggle adjustment actuator 2802, and to adjust the vertical and / or horizontal position of a robotic arm that will be or is currently coupled to an endoscopic device, the operator may toggle adjustment actuator 2804. In some embodiments, a user may adjust the horizontal and vertical position of a robotic arm by using the robotic arm as a force-sensitive input device. For example, the robotic arm may be configured to sense the user's intention by measuring the force applied by the user on the robotic arm. If the user applies a force in the positive horizontal direction, the platform may move the robotic arm in that direction until the user no longer applies force. Similar approaches are taken for other directions, e.g., negative horizontal, positive vertical, and negative vertical. As shown in FIG. 28B, the graphical user interface 2800 may indicate, via error notification 2810, whether an error, e.g., a fault condition, is detected by the system during setup or operation of the system.

[0285] As shown in FIG. 28C , the graphical user interface 2800 may display information associated with a selected surgical instrument, as described above. For example, the graphical user interface 2800 may display, for each instrument to be coupled to each robotic arm, the instrument type, overall length, distance between the coupler body and the instrument tip, distance between the center of mass and the instrument tip, mass, and a preset unlock force required to unlock the instrument. As shown in FIG. 28C , the graphical user interface 2800 may allow the operator to select between a high and low unlock force for the surgical instrument. Additionally, the graphical user interface 2800 may allow the operator to initiate a surgical instrument calibration, for example, for a new surgical instrument that does not already have an associated calibration file stored within the system. FIG. 28D illustrates an example screen during operation of the system, for example, during a surgical procedure. As shown in FIG. 28D, the graphical user interface 2800 may display the trocar force, for example, the force being applied to the tip of the surgical instrument by tissue within the patient's body.

[0286] 29, an alternative cooperatively manipulated surgical robotic system is provided. System 2900 may be constructed similarly to system 200 of FIG. 2. For example, platform 1400′, base portion 302′, shoulder portion 304′, encoders E1′, E2′, E3′, E5′, E6′, and E7′, motor M1′, shoulder joint 318′, shoulder linkage 305′, elbow joint 322′, elbow linkage 310′, wrist portion 311′, and coupler interface 400′ for coupling surgical instrument SI to the robotic arm may be constructed similarly to platform 1400, base portion 302, shoulder portion 304, encoders E1, E2, E3, E5, E6, and E7, motor M1, shoulder joint 318, shoulder linkage 305, elbow joint 322, elbow linkage 310, wrist portion 311, and coupler interface 400, respectively. System 2900 differs from system 200 in that system 2900 includes motors disposed at the joints of the robotic arm. For example, system 2900 may include motor M2' disposed at elbow joint 318' and motor M3' disposed at elbow joint 322' configured to rotate associated linkages and manipulate the robotic arm. Additionally, encoder E4' may be positioned on or adjacent to elbow joint 322'.

[0287] Some implementations of the systems described herein may be configured to be controlled or operated remotely, for example, via a joystick or other suitable remote control device, computer vision algorithms, force measurement algorithms, and / or other means. However, in preferred embodiments, the systems described herein operate without any telemetry; for example, the robotic arm is not remotely controlled via a remote surgeon console separate from the robotic arm; instead, the robotic arm moves in response to movements applied to a surgical instrument coupled thereto. For example, in robotically assisted mode, any robotically assisted movements applied to the surgical instrument by the system are not responsive to user input received at the remote surgeon console.

[0288] FIG. 30A illustrates a top view of a coupler 3000 f...

Claims

1. 1. A coupler device for removably coupling a surgical instrument having a handle and an elongated shaft to a distal end of a robotic arm of a collaborative surgical system to assist in a laparoscopic surgical procedure performed using the surgical instrument, the distal end of the robotic arm comprising a coupler interface configured to be removably coupled to the coupler device, the coupler device comprising: a lumen sized and shaped to receive the elongate shaft of the surgical instrument therein; a coupler body configured to transition between an open state in which the elongate shaft is slidably movable within the lumen and a closed state in which longitudinal movement of the elongate shaft relative to the coupler body is prevented while rotational movement of the elongate shaft relative to the coupler body is permitted in response to movement at the handle of the surgical instrument; Equipped with a coupler device configured such that when the coupler body is coupled to the coupler interface, the coupler body rotates relative to the distal end of the robotic arm via the coupler interface as the elongate shaft is inserted into the lumen, self-aligning the lumen with the elongate shaft.

2. 1. A coupler device for removably coupling a surgical instrument having an elongated shaft to a distal end of a robotic arm of a collaborative surgical system to assist in a laparoscopic surgical procedure performed using the surgical instrument, the distal end of the robotic arm comprising a coupler interface having a protrusion, the coupler device comprising: a coupler body configured to be removably coupled to the coupler interface and the elongate shaft of the surgical instrument, the coupler body comprising: a groove configured to receive the protrusion of the coupler interface; a lumen sized and shaped to receive the elongate shaft therein; a switch configured to transition between an unlocked position and a locked position, the switch including an engagement portion configured to engage the elongate shaft when the elongate shaft is disposed within the lumen and the switch is in the locked position, thereby securing the elongate shaft within the lumen; a coupler body comprising: A coupler device configured such that when the coupler body is coupled to the coupler interface and the elongate shaft is positioned within the lumen, the robotic arm is freely movable in response to movement at the handle of the surgical instrument.

3. 10. The coupler device of claim 1, further comprising a switch configured to transition between an unlocked position and a locked position, the switch comprising an engagement portion configured to apply a frictional force to the elongate shaft when the elongate shaft is disposed within the lumen and the switch is in the locked position, thereby allowing rotational movement of the elongate shaft within the lumen while prohibiting translational movement of the elongate shaft relative to the coupler body.

4. The beveled surface further comprises: a first valley configured to engage the switch in the unlocked position; a second valley configured to engage the switch in the locked position; a ridge disposed between the first valley and the second valley, the ridge configured to allow the switch to transition between the unlocked position and the locked position when a force applied to the switch exceeds a predetermined force threshold; 4. A coupler device according to claim 2 or 3, comprising:

5. A coupler device according to any of claims 2-4, wherein the switch comprises a handle configured to be actuated to transition the switch between the unlocked and locked positions.

6. 6. The coupler device of claim 3, wherein when the coupler body is coupled to the coupler interface and the elongate shaft is positioned within the lumen, the robotic arm is configured to be freely movable in response to movement at the handle of the surgical instrument.

7. 10. The coupler device of claim 9, wherein the coupler body comprises one or more tapered surfaces configured to guide the elongate shaft into the lumen by rotating the coupler body relative to the distal end of the robotic arm via the coupler interface as the elongate shaft is inserted into the lumen along the one or more tapered surfaces, and to promote self-alignment of the elongate shaft with the lumen.

8. 10. The coupler device of any preceding claim, further comprising a clamp configured to transition between an unlocked state, in which the lumen is permitted to receive the elongate shaft, and a locked state, in which the clamp secures the elongate shaft within the lumen.

9. The coupler device of claim 8 , wherein the clamp is configured to be biased toward the locked state.

10. 10. The coupler device of claim 8 or 9, wherein at least a portion of the clamp comprises a tapered surface configured to guide the elongate shaft into the lumen and facilitate transition of the clamp from the locked state to the unlocked state in response to a force applied to the tapered surface by the elongate shaft as the elongate shaft is inserted into the lumen.

11. A coupler device according to any of claims 8-10, wherein the clamp comprises a handle portion configured to be actuated to transition the clamp from the locked state to the unlocked state.

12. a holder slidably disposed within the coupler body, the holder comprising a friction pad configured to define at least a portion of the lumen; Furthermore, 10. The coupler device of claim 1, wherein the holder is configured to be biased in a direction toward the lumen such that the friction pad is configured to engage the elongate shaft when the elongate shaft is positioned within the lumen.

13. 13. The coupler device of claim 12, wherein the coupler interface comprises a repulsive magnet and the holder comprises a magnet, such that the repulsive magnet is configured to apply a magnetic force to the magnet, thereby biasing the holder in a direction toward the lumen.

14. The coupler device of claim 13 , wherein the holder comprises a harness configured to be coupled to the magnet, the harness being sized and shaped to be slidably disposed within a channel in the coupler body.

15. a clamp pivotally coupled to the coupler body via a rod, the clamp configured to transition between an unlocked state, in which the lumen is permitted to receive the elongate shaft, and a locked state, in which the clamp secures the elongate shaft within the lumen. Furthermore, A coupler device as described in any of claims 12-14, wherein the holder comprises one or more cradles coupled to the friction pad, each of the one or more cradles comprising a channel sized and shaped to slidably receive the rod therethrough, such that the holder is configured to be slidably positioned within the coupler body along the rod.

16. The coupler device of claim 1 , wherein the coupler body comprises a groove configured to receive a protrusion of the coupler interface.

17. one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends into the groove of the coupler body and an unlocked configuration in which the one or more locking arms do not extend into the groove such that the protrusion of the coupler interface is allowed to be received by the groove when the one or more locking arms are in the unlocked configuration; Furthermore, 17. A coupler device as described in any of claims 2-16, wherein the at least a portion of the one or more locking arms extends into one or more recesses in the protrusion of the coupler interface when the protrusion is disposed in the groove and the locking arms are in the locked configuration, thereby securing the coupler body to the coupler interface.

18. The coupler device of claim 17 , wherein the one or more locking arms are biased toward the locking configuration.

19. 19. The coupler device of claim 17 or 18, wherein each of the one or more locking arms comprises a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration.

20. A coupler device as described in any of claims 2-19, wherein the protrusion of the coupler interface has a first geometric shape and the groove of the coupler body has a second geometric shape corresponding to the first geometric shape, such that when the protrusion is received by the groove, rotational movement between the coupler body and the coupler interface is prohibited.

21. 10. The coupler device of claim 1, wherein the coupler body is configured to receive a sterile drape between the coupler body and the coupler interface when the coupler body is coupled to the coupler interface.

22. 10. A collaborative surgical system comprising a robotic arm and a coupler device according to any of the preceding claims.

23. The collaborative surgical system of claim 22, wherein the collaborative surgical system has two robotic arms, each robotic arm configured to be coupled to a coupler device.

24. 1. A co-operative surgical system for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the co-operative surgical system comprising: a robotic arm comprising a proximal end, a distal end configured to be removably coupled to the surgical instrument, a plurality of linkages, and a plurality of joints between the proximal end and the distal end, wherein the distal end of the robotic arm comprises a coupler interface; a coupler body configured to be removably coupled to the coupler interface, the coupler body including a lumen sized and shaped to receive the elongate shaft of the surgical instrument therein, the coupler body configured to transition between an open state in which the elongate shaft is slidably movable within the lumen and a closed state in which longitudinal movement of the elongate shaft relative to the coupler body is prevented while rotational movement of the elongate shaft relative to the coupler body is permitted in response to movement of the handle of the surgical instrument; Equipped with when the coupler body is coupled to the coupler interface, the coupler body is configured to rotate relative to the distal end of the robotic arm via the coupler interface to self-align the lumen with the elongate shaft as the elongate shaft is inserted into the lumen; When the coupler body is coupled to the coupler interface in the closed state, the robotic arm is enabled to be freely movable in response to movement of the handle of the surgical instrument to perform laparoscopic surgery.

25. The cooperative surgical system of claim 24, wherein the coupler body is disposable after a single laparoscopic surgical procedure.

26. a switch configured to transition between an unlocked position and a locked position, the switch including an engagement portion configured to engage the elongate shaft when the elongate shaft is disposed within the lumen and the switch is in the locked position, thereby securing the elongate shaft within the lumen. Furthermore, 25. The cooperatively operated surgical system of claim 24, wherein when the elongate shaft is disposed within the lumen and the switch is in the locked position, the engagement portion is configured to apply a frictional force to the elongate shaft, the frictional force being configured to allow rotational movement of the elongate shaft relative to the coupler body while preventing longitudinal movement of the elongate shaft relative to the coupler body.

27. 27. The collaborative surgical system of claim 26, wherein when the coupler body is coupled to the coupler interface and the elongate shaft is disposed within the lumen, the robotic arm is configured to be freely movable in response to movement at the handle of the surgical instrument.

28. The cooperative surgical system of claim 26, wherein the switch comprises a handle portion configured to be actuated to transition the switch between the unlocked and locked positions.

29. The coupler body includes a beveled surface, the beveled surface comprising: a first valley configured to engage the switch in the unlocked position; a second valley configured to engage the switch in the locked position; a peak between the first valley and the second valley, the peak configured to allow the switch to transition between the unlocked and locked positions when a force applied to the switch exceeds a predetermined force threshold; and 27. The cooperative surgical system of claim 26, comprising:

30. 25. The cooperatively manipulated surgical system of claim 24, wherein the coupler body further comprises a holder slidably disposed within the coupler body, the holder comprising a contact surface configured to define at least a portion of the lumen, the holder configured to be biased in a direction toward the lumen such that the contact surface is configured to engage the elongate shaft when the elongate shaft is disposed within the lumen.

31. 31. The cooperatively manipulated surgical system of claim 30, wherein the coupler interface comprises a repulsive magnet and the holder comprises a magnet such that the repulsive magnet is configured to apply a magnetic force to the magnet, thereby biasing the holder in a direction toward the lumen.

32. 32. The cooperatively manipulated surgical system of claim 31 , wherein the holder includes a harness configured to be coupled to the magnet, the harness sized and shaped to be slidably disposed within a channel of the coupler body.

33. a clamp pivotally coupled to the coupler body via a rod, the clamp configured to transition between an unlocked state, in which the lumen is permitted to receive the elongate shaft, and a locked state, in which the clamp secures the elongate shaft within the lumen. Furthermore, 31. The cooperatively manipulated surgical system of claim 30, wherein the holder comprises one or more cradles coupled to the contact surface, each of the one or more cradles comprising a channel sized and shaped to slidably receive the rod therethrough such that the holder is configured to be slidably disposed within the coupler body along the rod.

34. The cooperatively manipulated surgical system of claim 24, wherein the coupler interface includes a protrusion, and the coupler body includes a groove configured to receive the protrusion of the coupler interface.

35. The protrusion comprises one or more recesses, and the coupler body comprises: one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends into the groove of the coupler body and an unlocked configuration in which the one or more locking arms do not extend into the groove of the coupler body; Equipped with the protrusion of the coupler interface is configured to be received by the groove of the coupler body when the one or more locking arms are in the unlocked configuration; 35. The cooperatively operated surgical system of claim 34, wherein the at least a portion of the one or more locking arms extends into the one or more recesses of the protrusion when the protrusion is disposed in the groove and the locking arms are in the locked configuration, thereby securing the coupler body to the coupler interface.

36. The cooperative surgical system of claim 35, wherein the one or more locking arms are biased toward the locked configuration.

37. 36. The cooperative surgical system of claim 35, wherein each of the one or more locking arms includes a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration.

38. 35. The cooperatively manipulated surgical system of claim 34, wherein the protrusion of the coupler interface has a first geometric shape and the groove of the coupler body has a second geometric shape that corresponds to the first geometric shape, such that when the protrusion is received by the groove, rotational movement between the coupler body and the coupler interface is prohibited.

39. 35. The cooperatively manipulated surgical system of claim 34, wherein the coupler interface includes one or more additional protrusions having a first geometric shape and the coupler body includes one or more additional grooves having a second geometric shape, such that when the one or more additional protrusions are received by the one or more additional grooves, rotational movement between the coupler body and the coupler interface is inhibited.

40. 25. The collaborative surgical system of claim 24, wherein the coupler body and the coupler interface are configured to receive a sterile drape therebetween, such that the sterile drape prevents contact between the surgical instruments and the robotic arm during the laparoscopic surgical procedure.

41. 25. The collaborative surgical system of claim 24, wherein the coupler body includes one or more tapered surfaces configured to guide the elongate shaft into the lumen and promote self-alignment of the lumen and the elongate shaft by rotating the coupler body relative to the distal end of the robotic arm via the coupler interface as the elongate shaft is inserted into the lumen along the one or more tapered surfaces.

42. 25. The cooperative surgical system of claim 24, wherein the coupler body includes a clamp configured to transition between an unlocked state in which the lumen is permitted to receive the elongate shaft and a locked state in which the clamp secures the elongate shaft within the lumen.

43. The cooperative surgical system of claim 42, wherein the clamp is configured to be biased toward the locked state.

44. 44. The cooperatively manipulated surgical system of claim 43, wherein at least a portion of the clamp includes a tapered surface configured to guide the elongate shaft into the lumen and facilitate transition of the clamp from the locked state to the unlocked state in response to a force applied to the tapered surface by the elongate shaft as the elongate shaft is inserted into the lumen.

45. 1. A method for using a robotic arm configured to be removably coupled to a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the method comprising: removably coupling a coupler body to a coupler interface at a distal end of the robotic arm; inserting the elongate shaft of the surgical instrument into a lumen of the coupler body; transitioning the coupler body from an open state in which the elongate shaft is slidably movable within the lumen to a closed state in which longitudinal movement of the elongate shaft relative to the coupler body is prevented while rotational movement of the elongate shaft relative to the coupler body is permitted in response to movement of the handle of the surgical instrument; When the coupler body is coupled to the coupler interface in the closed state, moving the handle of the surgical instrument allows the robotic arm to move freely. Including, The coupler body rotates relative to the distal end of the robotic arm via the coupler interface as the elongate shaft is inserted into the lumen, self-aligning the lumen with the elongate shaft.

46. Removably coupling the coupler body to the coupler interface includes: actuating one or more locking arms of the coupler body to transition the one or more locking arms from a locked configuration in which at least a portion of the one or more locking arms extends into a groove of the coupler body to an unlocked configuration in which the one or more locking arms do not extend into the groove; Inserting a protrusion of the coupler interface into a groove of the coupler body; releasing the one or more locking arms and transitioning the one or more locking arms from the unlocked configuration to the locked configuration such that the at least a portion of the one or more locking arms extends into one or more recesses of the protrusion, thereby securing the coupler body to the coupler interface; 46. ​​The method of claim 45, comprising:

47. 46. ​​The method of claim 45, wherein inserting the elongate shaft of the surgical instrument into the lumen of the coupler body comprises guiding the elongate shaft into the lumen along one or more tapered surfaces of the coupler body.

48. 48. The method of claim 47, wherein guiding the elongate shaft into the lumen along one or more tapered surfaces of the coupler body rotates the coupler body relative to the distal end of the robotic arm via the coupler interface as the elongate shaft is inserted into the lumen along the one or more tapered surfaces, causing the lumen to self-align with the elongate shaft.

49. Inserting the elongate shaft of the surgical instrument into the lumen of the coupler body comprises: actuating a clamp of the coupler body to transition the clamp from a locked state to an unlocked state in which the lumen is permitted to receive the elongate shaft; inserting the elongate shaft of the surgical instrument into the lumen; releasing the clamp and transitioning the clamp from the unlocked state to the locked state, such that the clamp secures the elongate shaft within the lumen; 46. ​​The method of claim 45, comprising:

50. 46. ​​The method of claim 45, wherein transitioning the coupler body from the open state to the closed state includes transitioning a switch of the coupler body from an unlocked position, in which the elongate shaft is slidably movable within the lumen, to a locked position, in which an engagement portion of a switch engages the elongate shaft disposed within the lumen, thereby preventing longitudinal movement of the elongate shaft relative to the coupler body while allowing rotational movement of the elongate shaft relative to the coupler body.

51. 51. The method of claim 50, wherein when the elongate shaft is disposed within the lumen and the switch is in the locked position, the engagement portion applies a frictional force to the elongate shaft, the frictional force configured to allow rotational movement of the elongate shaft relative to the coupler body while preventing longitudinal movement of the elongate shaft relative to the coupler body.

52. 46. ​​The method of claim 45, wherein the coupler interface comprises a repulsive magnet, and the coupler body comprises a holder slidably disposed within the coupler body, the holder comprising a magnet and a contact surface configured to define at least a portion of the lumen, such that when the coupler body is removably coupled to the coupler interface, the repulsive magnet applies a magnetic force to the magnet, urging the holder in a direction toward the lumen.

53. 46. ​​The method of claim 45, further comprising positioning a sterile drape between the coupler body and the coupler interface prior to removably coupling the coupler body to the coupler interface.

54. 1. A co-operative surgical system for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the co-operative surgical system comprising: a robotic arm comprising a proximal end, a distal end configured to be removably coupled to the surgical instrument, a plurality of linkages, and a plurality of joints between the proximal end and the distal end, wherein the distal end of the robotic arm comprises a coupler interface; a coupler body configured to be removably coupled to the coupler interface, the coupler body including a lumen sized and shaped to receive the elongate shaft of the surgical instrument therein, the coupler body configured to transition between an open state in which the elongate shaft is slidably movable within the lumen and a closed state in which longitudinal movement of the elongate shaft relative to the coupler body is prevented while rotational movement of the elongate shaft relative to the coupler body is permitted in response to movement of the handle of the surgical instrument; Equipped with When the coupler body is coupled to the coupler interface in the closed state, the robotic arm is enabled to be freely movable in response to movement of the handle of the surgical instrument to perform laparoscopic surgery.

55. The cooperative surgical system of claim 54, wherein the coupler body is disposable after a single laparoscopic surgical procedure.

56. a switch configured to transition between an unlocked position and a locked position, the switch including an engagement portion configured to engage the elongate shaft when the elongate shaft is disposed within the lumen and the switch is in the locked position, thereby securing the elongate shaft within the lumen. Furthermore, 55. The collaborative surgical system of claim 54, wherein when the coupler body is coupled to the coupler interface, the elongate shaft is disposed within the lumen, and the switch is in the locked position, the robotic arm is configured to be freely movable in response to movement at the handle of the surgical instrument.

57. 57. The cooperatively operated surgical system of claim 56, wherein when the elongate shaft is disposed within the lumen and the switch is in the locked position, the engagement portion is configured to apply a frictional force to the elongate shaft, the frictional force being configured to allow rotational movement of the elongate shaft relative to the coupler body while preventing longitudinal movement of the elongate shaft relative to the coupler body.

58. The cooperative surgical system of claim 56, wherein the switch comprises a handle portion configured to be actuated to transition the switch between the unlocked and locked positions.

59. 57. The cooperatively manipulated surgical system of claim 56, wherein the coupler body further comprises a holder slidably disposed within the coupler body, the holder comprising a contact surface configured to define at least a portion of the lumen, the holder configured to be biased in a direction toward the lumen such that the contact surface is configured to engage the elongate shaft when the elongate shaft is disposed within the lumen.

60. 60. The cooperatively manipulated surgical system of claim 59, wherein the coupler interface comprises a repulsive magnet and the holder comprises a magnet such that the repulsive magnet is configured to apply a magnetic force to the magnet, thereby biasing the holder in a direction toward the lumen.

61. 61. The cooperatively manipulated surgical system of claim 60, wherein the holder includes a harness configured to be coupled to the magnet, the harness sized and shaped to be slidably disposed within a channel of the coupler body.

62. 60. The cooperatively operated surgical system of claim 59, wherein the contact surface is configured to apply a frictional force to the elongate shaft when the elongate shaft is disposed within the lumen and the switch is in the locked position, the frictional force being configured to facilitate allowing rotational movement of the elongate shaft relative to the coupler body while preventing translational movement of the elongate shaft relative to the coupler body.

63. a clamp pivotally coupled to the coupler body via a rod, the clamp configured to transition between an unlocked state, in which the lumen is permitted to receive the elongate shaft, and a locked state, in which the clamp secures the elongate shaft within the lumen. Furthermore, 60. The cooperatively operated surgical system of claim 59, wherein the holder comprises one or more cradles coupled to the contact surface, each of the one or more cradles comprising a channel sized and shaped to slidably receive the rod therethrough such that the holder is configured to be slidably disposed within the coupler body along the rod.

64. The cooperatively manipulated surgical system of claim 54, wherein the coupler interface includes a protrusion, and the coupler body includes a groove configured to receive the protrusion of the coupler interface.

65. The protrusion comprises one or more recesses, and the coupler body comprises: one or more locking arms configured to transition between a locked configuration in which at least a portion of the one or more locking arms extends into the groove of the coupler body and an unlocked configuration in which the one or more locking arms do not extend into the groove of the coupler body; Equipped with the protrusion of the coupler interface is configured to be received by the groove of the coupler body when the one or more locking arms are in the unlocked configuration; 65. The cooperatively operated surgical system of claim 64, wherein the at least a portion of the one or more locking arms extends into the one or more recesses of the protrusion when the protrusion is disposed in the groove and the locking arms are in the locked configuration, thereby securing the coupler body to the coupler interface.

66. The cooperative surgical system of claim 65, wherein the one or more locking arms are biased toward the locked configuration.

67. 66. The cooperative surgical system according to claim 65, wherein each of the one or more locking arms includes a handle portion configured to be actuated to transition the one or more locking arms from the locked configuration to the unlocked configuration.

68. 65. The cooperatively manipulated surgical system of claim 64, wherein the protrusion of the coupler interface has a first geometric shape and the groove of the coupler body has a second geometric shape that corresponds to the first geometric shape, such that when the protrusion is received by the groove, rotational movement between the coupler body and the coupler interface is prohibited.

69. 65. The cooperatively operated surgical system of claim 64, wherein the coupler interface includes one or more additional protrusions having a first geometric shape and the coupler body includes one or more additional grooves having a second geometric shape, such that when the one or more additional protrusions are received by the one or more additional grooves, rotational movement between the coupler body and the coupler interface is inhibited.

70. 55. The collaborative surgical system of claim 54, wherein the coupler body and the coupler interface are configured to receive a sterile drape therebetween, such that the sterile drape prevents contact between the surgical instruments and the robotic arm during the laparoscopic surgical procedure.

71. 55. The cooperatively manipulated surgical system of claim 54, wherein the coupler body includes one or more tapered surfaces configured to guide the elongate shaft into the lumen by rotating the coupler body and the coupler interface to align the lumen with the elongate shaft as the elongate shaft is inserted into the lumen along the one or more tapered surfaces, thereby facilitating self-alignment of the distal end of the robotic arm with the surgical instrument.

72. 55. The cooperative surgical system of claim 54, wherein the coupler body includes a clamp configured to transition between an unlocked state in which the lumen is permitted to receive the elongate shaft and a locked state in which the clamp secures the elongate shaft within the lumen.

73. The cooperative surgical system of claim 72, wherein the clamp is configured to be biased toward the locked state.

74. 74. The cooperatively manipulated surgical system of claim 73, wherein at least a portion of the clamp includes a tapered surface configured to guide the elongate shaft into the lumen and facilitate transition of the clamp from the locked state to the unlocked state in response to a force applied to the tapered surface by the elongate shaft as the elongate shaft is inserted into the lumen.

75. 1. A method for using a robotic arm configured to be removably coupled to a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the method comprising: removably coupling a coupler body to a coupler interface at a distal end of the robotic arm; inserting the elongate shaft of the surgical instrument into a lumen of the coupler body; transitioning the coupler body from an open state in which the elongate shaft is slidably movable within the lumen to a closed state in which longitudinal movement of the elongate shaft relative to the coupler body is prevented while rotational movement of the elongate shaft relative to the coupler body is permitted in response to movement of the handle of the surgical instrument; When the coupler body is coupled to the coupler interface in the closed state, moving the handle of the surgical instrument allows the robotic arm to move freely. A method comprising:

76. Removably coupling the coupler body to the coupler interface includes: actuating one or more locking arms of the coupler body to transition the one or more locking arms from a locked configuration in which at least a portion of the one or more locking arms extends into a groove of the coupler body to an unlocked configuration in which the one or more locking arms do not extend into the groove; Inserting a protrusion of the coupler interface into a groove of the coupler body; releasing the one or more locking arms and transitioning the one or more locking arms from the unlocked configuration to the locked configuration such that the at least a portion of the one or more locking arms extends into one or more recesses of the protrusion, thereby securing the coupler body to the coupler interface; 76. The method of claim 75, comprising:

77. 76. The method of claim 75, wherein inserting the elongate shaft of the surgical instrument into the lumen of the coupler body comprises guiding the elongate shaft into the lumen along one or more tapered surfaces of the coupler body.

78. 78. The method of claim 77, wherein guiding the elongate shaft into the lumen along one or more tapered surfaces of the coupler body comprises rotating the coupler body and the coupler interface as the elongate shaft is inserted into the lumen along the one or more tapered surfaces to promote self-alignment of the elongate shaft with the lumen.

79. Inserting the elongate shaft of the surgical instrument into the lumen of the coupler body comprises: actuating a clamp of the coupler body to transition the clamp from a locked state to an unlocked state in which the lumen is permitted to receive the elongate shaft; inserting the elongate shaft of the surgical instrument into the lumen; releasing the clamp and transitioning the clamp from the unlocked state to the locked state, such that the clamp secures the elongate shaft within the lumen; 76. The method of claim 75, comprising:

80. 76. The method of claim 75, wherein transitioning the coupler body from the open state to the closed state includes transitioning a switch of the coupler body from an unlocked position, in which the elongate shaft is slidably movable within the lumen, to a locked position, in which an engagement portion of a switch engages the elongate shaft disposed within the lumen, thereby preventing longitudinal movement of the elongate shaft relative to the coupler body while allowing rotational movement of the elongate shaft relative to the coupler body.

81. 81. The method of claim 80, wherein when the elongate shaft is disposed within the lumen and the switch is in the locked position, the engagement portion applies a frictional force to the elongate shaft, the frictional force configured to allow rotational movement of the elongate shaft relative to the coupler body while preventing longitudinal movement of the elongate shaft relative to the coupler body.

82. 76. The method of claim 75, wherein the coupler interface comprises a repulsive magnet, and the coupler body comprises a holder slidably disposed within the coupler body, the holder comprising a magnet and a contact surface configured to define at least a portion of the lumen, such that when the coupler body is removably coupled to the coupler interface, the repulsive magnet applies a magnetic force to the magnet, urging the holder in a direction toward the lumen.

83. 76. The method of claim 75, further comprising positioning a sterile drape between the coupler body and the coupler interface prior to removably coupling the coupler body to the coupler interface.

84. 1. A method for using a robotic arm configured to be removably coupled to a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the method being performed at a bedside adjacent to a bed holding a patient, the method comprising: positioning the robotic arm at a bedside; coupling the surgical instrument to the robotic arm, the robotic arm comprising a proximal end, a distal end, and a plurality of connections and joints between the proximal end and the distal end; freely moving the robotic arm by moving a handle of the surgical instrument coupled to the robotic arm while the robotic arm is positioned at the bedside; A method comprising:

85. 85. The method of claim 84, wherein the robotic arm remains at the bedside during use of the surgical instrument coupled to the robotic arm.

86. 85. The method of claim 84, wherein coupling the surgical instrument to the robotic arm comprises coupling the surgical instrument to the robotic arm using a purely mechanical linkage.

87. 85. The method of claim 84, wherein coupling the surgical instrument to the robotic arm comprises coupling the robotic arm only to the elongate shaft of the surgical instrument.

88. 85. The method of claim 84, wherein coupling the surgical instrument to the robotic arm comprises coupling the surgical instrument to the robotic arm while keeping a handle of the surgical instrument fully exposed for contact with a surgeon's hand.

89. 85. The method of claim 84, wherein coupling the surgical instrument to the robotic arm comprises coupling the surgical instrument to the robotic arm while the robotic arm is positioned at a bedside.

90. 85. The method of claim 84, wherein coupling the surgical instrument to the robotic arm comprises removably coupling a coupler body to a coupler interface disposed on the distal end of the robotic arm and removably coupling the surgical instrument to the coupler body.

91. 91. The method of claim 90, further comprising transitioning the coupler body from an open state, in which the elongate shaft is slidably moveable within a lumen of the coupler body, to a closed state, in which the robotic arm is allowed to move freely in response to movement at the handle of the surgical instrument.

92. 92. The method of claim 91, wherein when the coupler body is coupled to the coupler interface in the closed state, longitudinal movement of the elongate shaft relative to the coupler body is prevented, while rotational movement of the elongate shaft relative to the coupler body is allowed in response to movement at the handle of the surgical instrument.

93. 93. The method of claim 92, wherein when the coupler body is coupled to the coupler interface in the closed state, the coupler body applies a frictional force to the elongate shaft sufficient to permit rotational movement of the elongate shaft relative to the coupler body while preventing longitudinal movement of the elongate shaft relative to the coupler body.

94. 91. The method of claim 90, wherein removably coupling the surgical instrument to the coupler body comprises removably coupling the coupler body to a fixed point along the elongate shaft to provide a consistent point of reference for force calculations of the surgical instrument.

95. 91. The method of claim 90, further comprising positioning a sterile drape between the coupler body and the coupler interface prior to removably coupling the coupler body to the coupler interface.

96. 91. The method of claim 90, further comprising discarding the coupler body after a single laparoscopic surgical procedure.

97. 85. The method of claim 84, wherein the robotic arm is not remotely controlled via user input received at a remote surgeon console.

98. 85. The method of claim 84, wherein freely moving the robotic arm by moving the handle of the surgical instrument coupled to the robotic arm comprises applying a force to the robotic arm via the surgical instrument that exceeds a predetermined threshold, thereby automatically switching the robotic arm into a cooperative manipulation mode in which an impedance is applied to the robotic arm to account for the weight of the surgical instrument and the robotic arm.

99. 99. The method of claim 98, further comprising adjusting the predetermined threshold of force applied to the robotic arm via a graphical user interface operatively coupled to the robotic arm to automatically switch the robotic arm into the collaborative manipulation mode.

100. 99. The method of claim 98, wherein the plurality of joints of the robotic arm comprise one or more motorized joints operably coupled to one or more motors disposed in a base coupled to the proximal end of the robotic arm, the method further comprising measuring a current in the one or more motors, the current indicative of the force applied to the robotic arm via the surgical instrument.

101. 101. The method of claim 100, wherein the impedance applied to the robotic arm to account for the weight of the surgical instrument and the robotic arm is applied through the one or more motorized joints of the robotic arm.

102. 85. The method of claim 84, further comprising automatically switching the robotic arm to a passive mode in which the movement of the robotic arm is maintained within a predetermined amount for at least a predetermined dwell period, thereby maintaining the robotic arm in a static position.

103. 103. The method of claim 102, further comprising adjusting at least one of the predetermined amount of movement of the robotic arm or the predetermined dwell period via a graphical user interface operably coupled to the robotic arm to automatically switch the robotic arm to the passive mode.

104. 85. The method of claim 84, further comprising moving the robotic arm by moving the handle of the surgical instrument outside a predefined haptic barrier, thereby automatically switching the robotic arm to a haptic mode in which an impedance is applied to the robotic arm, making movement of the robotic arm in response to movement at the handle of the surgical instrument more viscoelastic in the haptic mode.

105. 105. The method of claim 104, further comprising adjusting a position of the predefined haptic barrier via a graphical user interface operably coupled to the robotic arm.

106. 85. The method of claim 84, further comprising selecting an identification of the surgical instrument coupled to the robotic arm via a graphical user interface operably coupled to the robotic arm.

107. 85. The method of claim 84, wherein the proximal end of the robotic arm is coupled to a base coupled to the platform via a stage assembly configured to move the base relative to the platform with at least two degrees of freedom, and the method further comprises adjusting at least one of a vertical height or a horizontal position of the robotic arm relative to the platform via the stage assembly.

108. 108. The method of claim 107, wherein adjusting at least one of the vertical height or the horizontal position of the robotic arm via the stage assembly includes providing user input via at least one of a graphical user interface operably coupled to the stage assembly or application of a force by a user at a distal region of the robotic arm in at least one of the at least two degrees of freedom.

109. 108. The method of claim 107, wherein the platform comprises a plurality of wheels, and wherein positioning the robotic arm at a bedside to perform the laparoscopic surgery comprises moving the platform relative to the bed via the plurality of wheels.

110. 110. The method of claim 109, further comprising: disengaging braking mechanisms of the plurality of wheels to allow movement of the platform via the plurality of wheels.

111. 108. The method of claim 107, further comprising displaying a virtual map comprising a graphical representation of the platform relative to the bed in an area surrounding the platform to facilitate positioning the robotic arm at the bedside to perform the laparoscopic surgical procedure.

112. 85. The method of claim 84, further comprising adjusting at least one of a height or an orientation of an optical sensor having a field of view of a surgical scene comprising at least one of the robotic arm, the surgical instrument coupled to the robotic arm, or the bed, thereby optimizing the field of view of the optical sensor.

113. further comprising selecting a laparoscope, a retractor tool, a grasper tool, or a surgical cutting tool; coupling the surgical instrument to the robotic arm includes coupling the laparoscope, the retractor tool, the grasper tool, or the surgical cutting tool to the robotic arm.

85. The method of claim 84.

114. 1. A co-operative surgical system for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the co-operative surgical system comprising: a robotic arm having a proximal end operably coupled to a base, a distal end configured to be removably coupled to the surgical instrument, a plurality of links, and a plurality of joints; a platform coupled to the base, the platform configured to move the base in at least one degree of freedom; a plurality of motors operably coupled to at least some of the plurality of joints; one or more sensors configured to collect sensor data comprising at least one of 3D depth data or pixel image data; a controller operably coupled to the robotic arm and the one or more sensors, the controller configured to enable the robotic arm to be freely movable in response to movement of the handle of the surgical instrument to perform a laparoscopic surgical procedure using the surgical instrument, the controller comprising: identifying at least one of a position or an orientation of one or more objects in a surgical room based on the sensor data from the one or more sensors; estimating relative distances between the one or more objects and the at least one of the base or the robotic arm as the at least one of the base or the robotic arm moves within the operating room; applying torque or impedance to at least some of the joints of the robot arm via the motors when the estimated relative distance approaches a predetermined threshold to reposition the robot arm or stop movement of the robot arm and avoid a collision between the one or more objects and the at least one of the base or the robot arm; a controller programmed to A collaborative surgical system comprising:

115. The controller detecting movement at the distal end of the robotic arm in a first direction in response to a first force applied to the distal end of the robotic arm by a user; causing the platform to move the base in the first direction in response to the detection of the movement at the distal end of the robotic arm in the first direction; causing the platform to stop movement of the base in the first direction when the first force applied by the user to the distal end of the robotic arm falls below a predetermined threshold.

115. The cooperatively operated surgical system of claim 114, configured to:

116. 116. The collaborative surgical system of claim 115, wherein the controller is configured to cause the platform to move the base in the first direction when the first force applied to the distal end of the robotic arm exceeds a predetermined force threshold.

117. 115. The collaborative surgical system of claim 114, wherein the controller is configured to identify planes of the one or more objects in the operating room based on the sensor data from the one or more sensors, and the controller is configured to estimate a relative distance between the one or more objects and the at least one of the base or the robotic arm based on the planes of the one or more objects.

118. The controller Determining the type of laparoscopic surgery to be performed; identifying at least one of a position or an orientation of a trocar port based on the sensor data from the one or more sensors; applying torques to at least some of the joints of the robotic arm via the plurality of motors to automatically position the robotic arm in a predetermined configuration relative to the trocar port based on the type of laparoscopic surgical procedure to be performed; 115. The cooperatively operated surgical system of claim 114, configured to:

119. The collaborative surgical system of claim 114, wherein the one or more objects comprises a surgical bed.

120. 1. A co-operative surgical system for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the co-operative surgical system comprising: a robotic arm having a proximal end, a distal end configured to be removably coupled to the surgical instrument, a plurality of linkages, and a plurality of joints; one or more sensors configured to collect sensor data comprising at least one of 3D depth data or pixel image data; a controller operably coupled to the robotic arm and the one or more sensors, the controller configured to enable the robotic arm to be freely movable in response to movement of the handle of the surgical instrument to perform a laparoscopic surgical procedure using the surgical instrument, the controller comprising: determining at least one of a position or orientation of a trocar port relative to the robotic arm based on the sensor data from the one or more sensors; detecting movement of the trocar port based on the sensor data from the one or more sensors when the working end of the surgical instrument is inserted through the trocar port; repositioning the robotic arm during movement of the trocar port to maintain the position of the working end of the surgical instrument relative to the trocar port; a controller programmed to A collaborative surgical system comprising:

121. 121. The collaborative surgical system of claim 120, wherein the controller is configured to detect movement of the trocar port in response to movement of a surgical bed.

122. 121. The collaborative surgical system of claim 120, wherein the controller is configured to detect movement of the trocar port in response to movement of the patient's body in response to breathing by the patient.

123. 121. The collaborative surgical system of claim 120, wherein the controller is configured to cause a distal end of the robotic arm to retract the working end of the surgical instrument within the trocar port prior to repositioning the robotic arm and maintaining the position of the working end of the surgical instrument relative to the trocar port during the movement of the trocar port.

124. 1. A co-operative surgical system for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the co-operative surgical system comprising: a robotic arm having a proximal end configured to be removably coupled to a cart, a distal end configured to be removably coupled to the surgical instrument, a plurality of linkages, and a plurality of joints; an optical scanner configured to measure depth data; a controller operably coupled to the robotic arm and the optical scanner, the controller configured to enable the robotic arm to be freely movable in response to movement of the handle of the surgical instrument to perform a laparoscopic surgical procedure using the surgical instrument, the controller comprising: moving the robotic arm in an intended predefined pattern of movement relative to the cart according to a preprogrammed routine; comparing depth data from the optical scanner indicative of actual movement of the robotic arm in response to the pre-programmed routine with an intended pre-defined pattern of movement, and generating a measure of error indicative of a deviation between the actual movement of the robotic arm and the intended pre-defined pattern of movement; executing an optimization algorithm configured to reduce the degree of error, such that deviation between the actual movement of the robot arm and an intended predefined pattern of movement is reduced; and a controller programmed to A collaborative surgical system comprising:

125. 125. The collaborative surgical system of claim 124, wherein the controller is configured to allow the robotic arm to be freely movable in response to movement of the handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument when the degree of error is below a predetermined threshold.

126. 1. A co-operative surgical system for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the co-operative surgical system comprising: a robotic arm having a proximal end, a distal end configured to be removably coupled to the surgical instrument, a plurality of linkages, and a plurality of joints; a controller operably coupled to the robotic arm and a laparoscope configured to generate a video feed, the controller configured to enable the robotic arm to be freely movable in response to movement at the handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument, the controller comprising: overlaying a virtual menu over the video feed displayed on a display; tracking movement of the working end of the surgical instrument in response to movement at the handle of the surgical instrument in the video feed and detecting one or more predetermined gesture patterns of movement of the working end; activating a function of the collaborative surgical system associated with the virtual menu based on the detection of the one or more predetermined gesture patterns of movement of the working end relative to the virtual menu; a controller programmed to A collaborative surgical system comprising:

127. The collaborative surgical system of claim 126, wherein the virtual menu comprises one or more menu options overlaid on at least one corner of the video feed.

128. 127. The collaborative surgical system of claim 126, wherein the functions of the collaborative surgical system associated with the virtual menu include adjusting a holding force threshold that must be exceeded to switch the robotic arm from a passive mode, in which the controller causes the robotic arm to maintain a static position, to a collaborative mode, in which the controller allows the robotic arm to be freely movable in response to movement at the handle of the surgical instrument to perform laparoscopic surgery using the surgical instrument.

129. 127. The collaborative surgical system of claim 126, wherein functions associated with the collaborative surgical system with the virtual menu include activation of an auxiliary speculum mode in which the controller causes the laparoscope to automatically adjust at least one of field of view or position to assist in the laparoscopic surgery.

130. 127. The collaborative surgical system of claim 126, wherein the controller is configured to overlay the virtual menu on the video feed displayed on the display in response to user input received via a graphical user interface operably coupled to the controller.

131. 127. The collaborative surgical system of claim 126, wherein the controller is configured to overlay the virtual menu on the video feed displayed on the display in response to a voice command by a user.

132. 127. The collaborative surgical system of claim 126, wherein the controller is configured to overlay the virtual menu on the video feed displayed on the display in response to actuation of an actuator disposed on the robotic arm.

133. 127. The collaborative surgical system of claim 126, wherein the controller is configured to track movement of the working end of the surgical instrument in response to user input received via a graphical user interface operably coupled to the controller.

134. 127. The collaborative surgical system of claim 126, wherein the controller is configured to track movement of the working end of the surgical instrument in response to voice commands by a user.

135. 127. The collaborative surgical system of claim 126, wherein the controller is configured to track movement of the working end of the surgical instrument in response to actuation of an actuator disposed on the robotic arm.

136. 1. A co-operative surgical system for assisting in laparoscopic surgery performed using a surgical instrument having a handle, a working end, and an elongated shaft therebetween, the co-operative surgical system comprising: a robotic arm having a proximal end, a distal end configured to be removably coupled to the surgical instrument, a plurality of linkages, and a plurality of joints; a controller operably coupled to the robotic arm, the controller comprising: causing the robotic arm to automatically switch between a cooperative manipulation mode, in which the controller allows the robotic arm to be freely movable in response to movement of the handle of the surgical instrument, to perform laparoscopic surgery using the surgical instrument, and a passive mode, in which the controller causes the robotic arm to maintain a static position; causing a vibration at the distal end of the robot arm, the vibration indicating that the robot arm is switching from the collaborative manipulation mode to the passive mode; a controller programmed to A collaborative surgical system comprising:

137. 137. The cooperatively operated surgical system of claim 136, wherein the vibration is configured to cause negligible movement in the working end of the surgical instrument while being perceptible by a user holding the handle of the surgical instrument.

138. 137. The collaborative surgical system of claim 136, wherein the controller is configured to induce a second vibration in the distal end of the robotic arm when the surgical instrument is coupled to the distal end of the robotic arm, the second vibration indicating that the surgical instrument has been coupled to the distal end of the robotic arm.

139. 137. The collaborative surgical system of claim 136, wherein the controller is configured to cause the robotic arm to switch to the passive mode in response to determining that movement of the robotic arm due to movement at the handle of the surgical instrument is less than a predetermined amount for at least a predetermined dwell period.

140. 137. The collaborative surgical system of claim 136, wherein the controller is configured to cause the robotic arm to switch to the collaborative manipulation mode in response to determining that a force applied to the robotic arm due to a force applied to the handle of the surgical instrument exceeds a predetermined threshold.

141. 137. The collaborative surgical system of claim 136, wherein the controller is configured to apply a first impedance to the robotic arm in the collaborative mode and to account for a weight of the surgical instrument and the robotic arm.

142. 137. The collaborative surgical system of claim 136, wherein the controller is configured to generate an audible alert, the audible alert indicating that the robotic arm has switched from the collaborative mode to the passive mode.

143. The robotic arm includes a base operably coupled to the proximal end of the robotic arm, and the system further includes: a plurality of motors disposed within the base, the plurality of motors operably coupled to at least some of the plurality of joints; Equipped with 137. The collaborative surgical system of claim 136, wherein the controller is programmed to actuate at least one motor of the plurality of motors to induce the vibrations in the distal end of the robotic arm.

144. 1. A co-manipulation surgical system for assisting in laparoscopic surgery performed using a laparoscope having a handle, a working end configured to collect a video feed constituting a field of view, and an elongated shaft therebetween, the co-manipulation surgical system comprising: a robotic arm having a proximal end, a distal end configured to be removably coupled to the laparoscope, a plurality of linkages, and a plurality of joints; an optical scanner configured to measure depth data; a controller operably coupled to the robotic arm and the optical scanner, the controller comprising: comparing the movement of the laparoscope based on the depth data from the optical scanner with the movement of the laparoscope's field of view during the movement of the laparoscope based on the video feed collected from the working end of the laparoscope; identifying a type of the laparoscope based on the movement of the field of view of the laparoscope during the movement of the laparoscope; a controller programmed to A collaborative surgical system comprising:

145. 145. The collaborative surgical system of claim 144, wherein the controller is configured to execute a pre-programmed routine in a calibration mode and cause the movement of the laparoscope in a pre-defined pattern of movement according to the pre-programmed routine.

146. 146. The collaborative surgical system of claim 145, wherein the predefined pattern of movement includes a circular motion.

147. 145. The collaborative surgical system of claim 144, wherein the movement of the laparoscope is in response to movement of the handle of the laparoscope by a user.

148. 145. The collaborative surgical system of claim 144, wherein the type of laparoscope comprises an angle of the working end of the laparoscope.

149. 149. The collaborative surgical system of claim 148, wherein the controller is configured to identify the type of the laparoscope as a flat-tipped laparoscope when the movement of the laparoscope includes a circular movement and the movement of the field of view of the laparoscope during the circular movement of the laparoscope includes a corresponding circular movement.

150. 149. The collaborative surgical system of claim 148, wherein the controller is configured to identify the type of the laparoscope as a flat-tipped laparoscope when the movement of the laparoscope includes a circular motion and the movement of the field of view of the laparoscope during the circular movement of the laparoscope does not include a change in depth of the field of view.

151. 149. The collaborative surgical system of claim 148, wherein the controller is configured to identify the type of the laparoscope as an angled tip laparoscope when the movement of the laparoscope includes a circular motion and the movement of the field of view of the laparoscope during the circular movement of the laparoscope includes a change in depth of the field of view.

152. 145. The collaborative surgical system of claim 144, wherein the controller is configured to enable the robotic arm to be freely movable in response to movement of the handle of the laparoscope to perform laparoscopic surgery using the laparoscope.

Images